dragonflight/services/premiere-plugin/FEATURE_SPEC.md

231 lines
9.2 KiB
Markdown
Raw Permalink Normal View History

feat: server-side filmstrip worker + fix scheduler crash + fix clip freeze Root causes found: 1. Scheduler crashing every 15s: assets table has no error_message column. Fix: remove error_message from UPDATE in scheduler.js (#66 regression). 2. Clip freezing: client-side filmstrip seek loop runs on main thread, seeks same proxy the player is streaming → both stall → freeze. Fix: replace browser seek loop entirely with server-side FFmpeg worker. 3. No dedicated filmstrip worker: filmstrip was never pre-built server-side. Changes: - services/mam-api/src/db/migrations/018-add-filmstrip-s3-key.sql Add filmstrip_s3_key TEXT column to assets table - services/worker/src/workers/filmstrip.js (new) BullMQ worker: downloads proxy, runs FFmpeg fps filter to extract 28 evenly-spaced JPEG frames, base64-encodes them, uploads JSON array to S3 at filmstrips/<assetId>.json, stores key in DB - services/worker/src/workers/thumbnail.js Queue filmstrip job automatically after thumbnail completes - services/worker/src/index.js Register filmstrip worker (concurrency=2), export filmstripQueue singleton, close it on SIGTERM - services/mam-api/src/routes/assets.js - filmstripQueue added - POST /reprocess?type=filmstrip now supported - GET /:id/filmstrip returns signed S3 URL for JSON frames - services/mam-api/src/routes/jobs.js filmstrip queue visible in Jobs UI - services/web-ui/public/screens-asset.jsx Replace browser seek loop with fetch of /assets/:id/filmstrip → fetch S3 JSON → render frames. Zero browser-side video seeking. Right-click and Files tab re-generate via API endpoint.
2026-05-26 12:39:44 -04:00
# Premiere Pro Plugin - Complete Workflow Specification
## Overview
The Wild Dragon MAM Premiere Pro plugin provides three primary workflows for proxy-based editing with hi-res delivery, plus a dedicated interface for growing files (live capture).
---
## Workflow 1: Proxy Edit with Server-Side Conform (Export to MAM)
**Status:** ✅ Already implemented (FCP XML Export & Conform)
### User Story
Editor works with proxy files in Premiere, completes the edit, then exports the timeline to the MAM server. The server renders the hi-res master using FFmpeg and stores it as a new MAM asset.
### Workflow Steps
1. Editor imports proxy assets from MAM into Premiere via plugin
2. Editor completes timeline edit using proxies
3. Editor clicks **Advanced > Export & Conform Timeline**
4. Plugin generates FCP XML from active sequence
5. Plugin uploads XML to MAM API endpoint `POST /api/sequences/conform`
6. Server-side FFmpeg worker:
- Parses FCP XML
- Fetches hi-res originals from S3
- Renders timeline with selected codec/quality preset
- Uploads rendered file to S3
- Creates new asset in MAM with name "Conformed: [Sequence Name]"
7. Plugin polls job status and shows progress
8. On completion, conformed asset appears in MAM library
### Technical Details
- **Codec presets:** ProRes, H.264, H.265/HEVC
- **Quality levels:** Low, High, Broadcast
- **Resolution options:** Match sequence, 4K, 1080p, 720p
- **Audio:** Optional stereo/AAC
- **Job queue:** BullMQ FIFO, concurrency 1
- **Storage:** S3 at `projects/<projectId>/conformed/<filename>`
### Current Implementation
- File: `services/premiere-plugin/jsx/premiere.jsx` (FCP XML export)
- File: `services/premiere-plugin/js/main.js` (conform UI)
- API: `services/mam-api/src/routes/sequences.js`
- Worker: `services/worker/src/workers/conform.js`
---
## Workflow 2: Local Export with Hi-Res Segment Fetch
**Status:** ✅ Already implemented (Hi-Res Auto-Relink)
### User Story
Editor works with proxy files in Premiere on a remote machine. When ready for final export, editor requests hi-res segments for only the used portions of each clip. Server renders frame-accurate trimmed segments, sends them back to the local machine, and plugin relinks them in Premiere. Editor then exports locally at full quality.
### Workflow Steps
1. Editor imports proxy assets from MAM into Premiere via plugin
2. Editor completes timeline edit using proxies
3. Editor clicks **Advanced > Fetch & Relink All**
4. Plugin analyzes active sequence and lists all proxy clips with in/out times
5. Editor reviews clip list and deselects any clips to keep as proxy
6. Editor clicks **Fetch & Relink**
7. For each selected clip:
- Plugin sends trim request to `POST /api/assets/batch-trim`
- Server-side FFmpeg worker fetches hi-res original from S3
- Worker renders frame-accurate segment (no handles) using stream copy where possible
- Worker uploads segment to S3 at `temp-segments/<clipInstanceId>.mov`
- Worker returns presigned download URL
8. Plugin downloads each segment to local temp directory
9. Plugin calls `ProjectItem.changeMediaPath()` to relink each clip in place
10. Timeline cuts are preserved; clips now reference hi-res segments
11. Editor exports timeline locally from Premiere (File > Export)
### Technical Details
- **Segment storage:** S3 prefix `temp-segments/`
- **TTL:** 24 hours (automatic cleanup via hourly task)
- **Job queue:** BullMQ FIFO, concurrency 4
- **Relink method:** ExtendScript `changeMediaPath()`
- **Cleanup:** `services/mam-api/src/tasks/cleanupTempSegments.js`
### Current Implementation
- File: `services/premiere-plugin/jsx/premiere.jsx` (timeline analysis, relink)
- File: `services/premiere-plugin/js/main.js` (relink UI)
- API: `services/mam-api/src/routes/assets.js` (batch-trim endpoint)
- Worker: `services/worker/src/workers/trimWorker.js`
- DB table: `temp_segments`
---
## Workflow 3: Growing Files Tab (NEW)
**Status:** ❌ Not implemented
### User Story
Editor wants to see and work with clips that are currently being captured to the SMB landing zone (growing files) separately from the main S3-backed asset library. A dedicated "Growing" tab shows live captures, allows mounting them for live editing, and shows promotion status.
### Requirements
#### UI Changes
1. **New tab in plugin:** Add "Growing" tab alongside existing asset grid
2. **Tab navigation:** Toggle between "Library" (S3 assets) and "Growing" (SMB assets)
3. **Growing tab layout:**
- Grid view similar to main asset grid
- Show assets with `status=live` or `status=ingesting`
- Display live preview thumbnail (from HLS stream if available)
- Show recording indicator (red dot + duration counter)
- Show promotion status badge:
- "Recording" (red) - actively capturing
- "Idle" (yellow) - capture stopped, waiting for promotion
- "Promoting" (blue) - uploading to S3
- "Ready" (green) - promoted to S3, ready to relink
#### Functionality
1. **Mount Live:** Button to import growing file from SMB share into Premiere
- Calls `GET /api/assets/:id/live-path`
- Resolves SMB UNC path (e.g., `\\10.0.0.25\mam-growing\<projectId>\<filename>`)
- Calls `app.project.importFiles([uncPath])`
- Premiere imports still-growing file for live editing
2. **Relink to Hi-Res:** Button appears when asset transitions to `status=ready`
- Downloads finalized hi-res from S3
- Calls `ProjectItem.changeMediaPath()` to relink
- Timeline cuts preserved
3. **Auto-refresh:** Poll every 5 seconds to update status badges and promotion progress
4. **Filter by project:** Same project filter as main library tab
#### API Requirements
- `GET /api/assets?status=live` - fetch growing files
- `GET /api/assets/:id/live-path` - resolve SMB UNC path
- Existing promotion worker already handles status transitions
#### Technical Details
- **SMB mount:** Editor must mount `smb://10.0.0.25/mam-growing` at OS level
- **Polling interval:** 5 seconds
- **Status transitions:** `live``ingesting``ready`
- **Promotion trigger:** `growing_promote_after_seconds` of mtime inactivity (default 300s)
---
## Settings Integration
All workflows respect the following MAM settings (Settings → Growing files):
- `growing_enabled` - Master switch for growing files feature
- `growing_path` - Container mount path (default `/growing`)
- `growing_smb_url` - SMB URL for plugin (e.g., `smb://10.0.0.25/mam-growing`)
- `growing_promote_after_seconds` - Idle threshold before promotion (default 300)
---
## Infrastructure
### Deployment Targets
- **ZAMPP1** (online) - MAM API + worker container
- **ZAMPP2** (online) - MAM API + worker container
- **BMG-PC-Edit** (online) - Premiere Pro workstation with plugin installed
### Storage
- **S3:** Hi-res masters at `projects/<projectId>/masters/`
- **S3:** Proxies at `projects/<projectId>/proxies/`
- **S3:** Conformed exports at `projects/<projectId>/conformed/`
- **S3:** Temp segments at `temp-segments/` (24h TTL)
- **SMB:** Growing files at `/mnt/NVME/MAM-growing/<projectId>/` (TrueNAS)
### Network Requirements
- Plugin → MAM API: HTTP/HTTPS
- Plugin → SMB share: SMB 2.0+ (passwordless on LAN)
- MAM API → S3: HTTPS (presigned URLs)
---
## Implementation Checklist
### Workflow 1: Proxy Edit with Server-Side Conform
- [x] FCP XML export from Premiere
- [x] Server-side conform worker
- [x] Codec presets (ProRes, H.264, H.265)
- [x] Job queue and progress tracking
- [x] Asset creation in MAM
### Workflow 2: Local Export with Hi-Res Segments
- [x] Timeline analysis and clip detection
- [x] Batch trim API endpoint
- [x] Frame-accurate segment rendering
- [x] Temp segment storage with TTL
- [x] Auto-relink in Premiere
- [x] Cleanup task for expired segments
### Workflow 3: Growing Files Tab
- [x] Add "Growing" tab to plugin UI
- [x] Fetch and display `status=live` assets
- [x] Show recording indicator and duration
- [x] Show promotion status badges
- [x] Implement "Mount Live" button (existing, now exposed via Growing tab)
- [x] Implement "Relink to Hi-Res" button (existing, now exposed via Growing tab)
- [x] Auto-refresh polling (5s interval)
- [x] Project filter for growing tab
- [x] Handle SMB path resolution (existing live-path endpoint)
- [x] Error handling for unmounted SMB share (existing)
---
## Testing Plan
### Workflow 1 Testing
1. Import proxies, edit timeline, export via plugin
2. Verify conform job completes
3. Verify conformed asset appears in MAM
4. Test all codec presets
5. Test error handling (missing hi-res, worker offline)
### Workflow 2 Testing
1. Import proxies, edit timeline, fetch hi-res segments
2. Verify segments download and relink
3. Verify timeline cuts preserved
4. Export locally and verify quality
5. Verify temp segments cleaned up after 24h
### Workflow 3 Testing
1. Start recorder, verify asset appears in Growing tab
2. Click "Mount Live", verify import into Premiere
3. Edit while recording continues
4. Stop recorder, verify promotion status updates
5. Click "Relink to Hi-Res", verify relink completes
6. Verify timeline cuts preserved after relink
7. Test with unmounted SMB share (error handling)
---
## Related Issues
- Issue #92: Growing Files: Per-upload toggle, retention controls, and seamless relinking
- Commit c312991: feat: implement advanced features (conform, auto-relink, GUI redesign)