WildDragonLLC/dragonflight
1
0
Fork 0
Code Issues 18 Pull requests 3 Projects Releases Packages Wiki Activity Actions
dragonflight/services/premiere-plugin/FEATURE_SPEC.md
ZGaetano a03c85f08a feat: server-side filmstrip worker + fix scheduler crash + fix clip freeze 
Root causes found:
1. Scheduler crashing every 15s: assets table has no error_message column.
   Fix: remove error_message from UPDATE in scheduler.js (#66 regression).

2. Clip freezing: client-side filmstrip seek loop runs on main thread,
   seeks same proxy the player is streaming → both stall → freeze.
   Fix: replace browser seek loop entirely with server-side FFmpeg worker.

3. No dedicated filmstrip worker: filmstrip was never pre-built server-side.

Changes:
- services/mam-api/src/db/migrations/018-add-filmstrip-s3-key.sql
  Add filmstrip_s3_key TEXT column to assets table

- services/worker/src/workers/filmstrip.js (new)
  BullMQ worker: downloads proxy, runs FFmpeg fps filter to extract
  28 evenly-spaced JPEG frames, base64-encodes them, uploads JSON
  array to S3 at filmstrips/<assetId>.json, stores key in DB

- services/worker/src/workers/thumbnail.js
  Queue filmstrip job automatically after thumbnail completes

- services/worker/src/index.js
  Register filmstrip worker (concurrency=2), export filmstripQueue
  singleton, close it on SIGTERM

- services/mam-api/src/routes/assets.js
  - filmstripQueue added
  - POST /reprocess?type=filmstrip now supported
  - GET /:id/filmstrip returns signed S3 URL for JSON frames

- services/mam-api/src/routes/jobs.js
  filmstrip queue visible in Jobs UI

- services/web-ui/public/screens-asset.jsx
  Replace browser seek loop with fetch of /assets/:id/filmstrip
  → fetch S3 JSON → render frames. Zero browser-side video seeking.
  Right-click and Files tab re-generate via API endpoint.
2026-05-26 16:39:44 +00:00

9.2 KiB
Raw Blame History

Premiere Pro Plugin - Complete Workflow Specification

Overview

The Wild Dragon MAM Premiere Pro plugin provides three primary workflows for proxy-based editing with hi-res delivery, plus a dedicated interface for growing files (live capture).

Workflow 1: Proxy Edit with Server-Side Conform (Export to MAM)

Status: ✅ Already implemented (FCP XML Export & Conform)

User Story

Editor works with proxy files in Premiere, completes the edit, then exports the timeline to the MAM server. The server renders the hi-res master using FFmpeg and stores it as a new MAM asset.

Workflow Steps

  1. Editor imports proxy assets from MAM into Premiere via plugin
  2. Editor completes timeline edit using proxies
  3. Editor clicks Advanced > Export & Conform Timeline
  4. Plugin generates FCP XML from active sequence
  5. Plugin uploads XML to MAM API endpoint POST /api/sequences/conform
  6. Server-side FFmpeg worker:
    • Parses FCP XML
    • Fetches hi-res originals from S3
    • Renders timeline with selected codec/quality preset
    • Uploads rendered file to S3
    • Creates new asset in MAM with name "Conformed: [Sequence Name]"
  7. Plugin polls job status and shows progress
  8. On completion, conformed asset appears in MAM library

Technical Details

  • Codec presets: ProRes, H.264, H.265/HEVC
  • Quality levels: Low, High, Broadcast
  • Resolution options: Match sequence, 4K, 1080p, 720p
  • Audio: Optional stereo/AAC
  • Job queue: BullMQ FIFO, concurrency 1
  • Storage: S3 at projects/<projectId>/conformed/<filename>

Current Implementation

  • File: services/premiere-plugin/jsx/premiere.jsx (FCP XML export)
  • File: services/premiere-plugin/js/main.js (conform UI)
  • API: services/mam-api/src/routes/sequences.js
  • Worker: services/worker/src/workers/conform.js

Workflow 2: Local Export with Hi-Res Segment Fetch

Status: ✅ Already implemented (Hi-Res Auto-Relink)

User Story

Editor works with proxy files in Premiere on a remote machine. When ready for final export, editor requests hi-res segments for only the used portions of each clip. Server renders frame-accurate trimmed segments, sends them back to the local machine, and plugin relinks them in Premiere. Editor then exports locally at full quality.

Workflow Steps

  1. Editor imports proxy assets from MAM into Premiere via plugin
  2. Editor completes timeline edit using proxies
  3. Editor clicks Advanced > Fetch & Relink All
  4. Plugin analyzes active sequence and lists all proxy clips with in/out times
  5. Editor reviews clip list and deselects any clips to keep as proxy
  6. Editor clicks Fetch & Relink
  7. For each selected clip:
    • Plugin sends trim request to POST /api/assets/batch-trim
    • Server-side FFmpeg worker fetches hi-res original from S3
    • Worker renders frame-accurate segment (no handles) using stream copy where possible
    • Worker uploads segment to S3 at temp-segments/<clipInstanceId>.mov
    • Worker returns presigned download URL
  8. Plugin downloads each segment to local temp directory
  9. Plugin calls ProjectItem.changeMediaPath() to relink each clip in place
  10. Timeline cuts are preserved; clips now reference hi-res segments
  11. Editor exports timeline locally from Premiere (File > Export)

Technical Details

  • Segment storage: S3 prefix temp-segments/
  • TTL: 24 hours (automatic cleanup via hourly task)
  • Job queue: BullMQ FIFO, concurrency 4
  • Relink method: ExtendScript changeMediaPath()
  • Cleanup: services/mam-api/src/tasks/cleanupTempSegments.js

Current Implementation

  • File: services/premiere-plugin/jsx/premiere.jsx (timeline analysis, relink)
  • File: services/premiere-plugin/js/main.js (relink UI)
  • API: services/mam-api/src/routes/assets.js (batch-trim endpoint)
  • Worker: services/worker/src/workers/trimWorker.js
  • DB table: temp_segments

Workflow 3: Growing Files Tab (NEW)

Status: ❌ Not implemented

User Story

Editor wants to see and work with clips that are currently being captured to the SMB landing zone (growing files) separately from the main S3-backed asset library. A dedicated "Growing" tab shows live captures, allows mounting them for live editing, and shows promotion status.

Requirements

UI Changes

  1. New tab in plugin: Add "Growing" tab alongside existing asset grid
  2. Tab navigation: Toggle between "Library" (S3 assets) and "Growing" (SMB assets)
  3. Growing tab layout:
    • Grid view similar to main asset grid
    • Show assets with status=live or status=ingesting
    • Display live preview thumbnail (from HLS stream if available)
    • Show recording indicator (red dot + duration counter)
    • Show promotion status badge:
      • "Recording" (red) - actively capturing
      • "Idle" (yellow) - capture stopped, waiting for promotion
      • "Promoting" (blue) - uploading to S3
      • "Ready" (green) - promoted to S3, ready to relink

Functionality

  1. Mount Live: Button to import growing file from SMB share into Premiere
    • Calls GET /api/assets/:id/live-path
    • Resolves SMB UNC path (e.g., \\10.0.0.25\mam-growing\<projectId>\<filename>)
    • Calls app.project.importFiles([uncPath])
    • Premiere imports still-growing file for live editing
  2. Relink to Hi-Res: Button appears when asset transitions to status=ready
    • Downloads finalized hi-res from S3
    • Calls ProjectItem.changeMediaPath() to relink
    • Timeline cuts preserved
  3. Auto-refresh: Poll every 5 seconds to update status badges and promotion progress
  4. Filter by project: Same project filter as main library tab

API Requirements

  • GET /api/assets?status=live - fetch growing files
  • GET /api/assets/:id/live-path - resolve SMB UNC path
  • Existing promotion worker already handles status transitions

Technical Details

  • SMB mount: Editor must mount smb://10.0.0.25/mam-growing at OS level
  • Polling interval: 5 seconds
  • Status transitions: live → ingesting → ready
  • Promotion trigger: growing_promote_after_seconds of mtime inactivity (default 300s)

Settings Integration

All workflows respect the following MAM settings (Settings → Growing files):

  • growing_enabled - Master switch for growing files feature
  • growing_path - Container mount path (default /growing)
  • growing_smb_url - SMB URL for plugin (e.g., smb://10.0.0.25/mam-growing)
  • growing_promote_after_seconds - Idle threshold before promotion (default 300)

Infrastructure

Deployment Targets

  • ZAMPP1 (online) - MAM API + worker container
  • ZAMPP2 (online) - MAM API + worker container
  • BMG-PC-Edit (online) - Premiere Pro workstation with plugin installed

Storage

  • S3: Hi-res masters at projects/<projectId>/masters/
  • S3: Proxies at projects/<projectId>/proxies/
  • S3: Conformed exports at projects/<projectId>/conformed/
  • S3: Temp segments at temp-segments/ (24h TTL)
  • SMB: Growing files at /mnt/NVME/MAM-growing/<projectId>/ (TrueNAS)

Network Requirements

  • Plugin → MAM API: HTTP/HTTPS
  • Plugin → SMB share: SMB 2.0+ (passwordless on LAN)
  • MAM API → S3: HTTPS (presigned URLs)

Implementation Checklist

Workflow 1: Proxy Edit with Server-Side Conform

  • FCP XML export from Premiere
  • Server-side conform worker
  • Codec presets (ProRes, H.264, H.265)
  • Job queue and progress tracking
  • Asset creation in MAM

Workflow 2: Local Export with Hi-Res Segments

  • Timeline analysis and clip detection
  • Batch trim API endpoint
  • Frame-accurate segment rendering
  • Temp segment storage with TTL
  • Auto-relink in Premiere
  • Cleanup task for expired segments

Workflow 3: Growing Files Tab

  • Add "Growing" tab to plugin UI
  • Fetch and display status=live assets
  • Show recording indicator and duration
  • Show promotion status badges
  • Implement "Mount Live" button (existing, now exposed via Growing tab)
  • Implement "Relink to Hi-Res" button (existing, now exposed via Growing tab)
  • Auto-refresh polling (5s interval)
  • Project filter for growing tab
  • Handle SMB path resolution (existing live-path endpoint)
  • Error handling for unmounted SMB share (existing)

Testing Plan

Workflow 1 Testing

  1. Import proxies, edit timeline, export via plugin
  2. Verify conform job completes
  3. Verify conformed asset appears in MAM
  4. Test all codec presets
  5. Test error handling (missing hi-res, worker offline)

Workflow 2 Testing

  1. Import proxies, edit timeline, fetch hi-res segments
  2. Verify segments download and relink
  3. Verify timeline cuts preserved
  4. Export locally and verify quality
  5. Verify temp segments cleaned up after 24h

Workflow 3 Testing

  1. Start recorder, verify asset appears in Growing tab
  2. Click "Mount Live", verify import into Premiere
  3. Edit while recording continues
  4. Stop recorder, verify promotion status updates
  5. Click "Relink to Hi-Res", verify relink completes
  6. Verify timeline cuts preserved after relink
  7. Test with unmounted SMB share (error handling)

Related Issues

  • Issue #92: Growing Files: Per-upload toggle, retention controls, and seamless relinking
  • Commit c312991: feat: implement advanced features (conform, auto-relink, GUI redesign)