WildDragonLLC/dragonflight
1
0
Fork 0
Code Issues 18 Pull requests 3 Projects Releases Packages Wiki Activity Actions

docs: finalize spec with user decisions (auto-save, multi-sequence, 59.94fps, gap-ok)

Browse source
This commit is contained in:
Zac Gaetano 2026-05-18 19:23:01 -04:00
parent 67251a0dcd
commit 9032853629
1 changed files with 188 additions and 222 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

410
docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md
View file

@ -1,16 +1,28 @@
# Wild Dragon MAM — Editor, Growing File Preview & Feature Expansion # Wild Dragon MAM — Editor, Growing File Preview & Feature Expansion
**Date:** 2026-05-18 **Date:** 2026-05-18
**Status:** Draft — awaiting user approval before implementation begins **Status:** APPROVED — ready for implementation planning
---
## Resolved Decisions
| Question | Decision |
|----------|----------|
| Save strategy | **Auto-save** (debounced 2s after any change) |
| Sequences per project | **Multiple named sequences** (like Premiere's project panel) |
| HLS temp disk | **3 TB available** — no constraint |
| Primary frame rate | **59.94 fps** |
| Program monitor fidelity | **Brief gap at clip boundaries is acceptable for v1** |
--- ---
## Overview ## Overview
Three interconnected areas, sequenced by priority: Three sequenced phases:
1. **NLE Editor** (`editor.html`) — Premiere-style timeline editor with razor blade, in/out marking, and EDL export 1. **NLE Editor** (`editor.html`) — Premiere-style timeline editor with razor blade, in/out marking, and EDL export
2. **Growing File Workflow** — HLS live preview during SDI/SRT/RTMP capture, with rewind support 2. **Growing File Workflow** — HLS live preview during SDI/SRT/RTMP capture, with rewind support
3. **Feature Additions** — Prioritized list of high-value additions (subclips, player improvements, bulk ops, waveform, etc.) 3. **Feature Additions** — Subclips, improved player, bulk ops, waveform, timecoded comments, smart bins, metadata templates
--- ---
@ -18,84 +30,85 @@ Three interconnected areas, sequenced by priority:
### 1.1 Layout ### 1.1 Layout
A new page `editor.html` replaces the role of `player.html` for editorial work. `player.html` is kept as the lightweight browse/metadata view; the editor is the full-screen creative environment. A new page `editor.html`. `player.html` is kept as the lightweight browse/metadata view; the editor is the full-screen creative environment.
``` ```
┌─ SOURCE MONITOR ──────┬─ PROGRAM MONITOR ──────┐ ← 40vh ┌─ SOURCE MONITOR ──────┬─ PROGRAM MONITOR ──────┐ ← 40vh
│ [video preview] │ [video preview] │ │ [video preview] │ [video preview] │
│ TC: 00:00:00:00 │ TC: 00:00:00:00 │ │ TC: 00:00:00;00 │ TC: 00:00:00;00 │
│ [────scrub bar────] │ [────scrub bar────] │ │ [────scrub bar────] │ [────scrub bar────] │
│ [IN] [OUT] [Insert] │ [J] [K] [L] [⏩] │ │ [IN] [OUT] [Insert] │ [J] [K] [L] [⏩] │
├─ MEDIA PANEL ─────────┴─ TIMELINE ─────────────┤ ← 60vh ├─ MEDIA PANEL ─────────┴─ TIMELINE ─────────────┤ ← 60vh
│ [bin tree] │ [V] [C] [H] [──zoom──] │ │ [sequence picker] │ [V] [C] [H] [zoom] │
│ [asset list] │ ruler: 00:00 00:05 00:10 … │ │ [bin tree] │ ruler: 00:00 00:05 … │
│ │ V1 ░░░░[clip A]░░░░[clip B]░░ │ │ [asset list] │ V1 ░░░░[clip A]░░░░ │
│ │ V2 │ │ │ V2 │
│ │ A1 ░░░░[clip A]░░░░[clip B]░░ │ │ │ A1 ░░░░[clip A]░░░░ │
│ │ A2 │ │ │ A2 │
└───────────────┴──────────────────────────────────┘ └───────────────────────┴─────────────────────────┘
``` ```
The editor is accessed from the library: each asset card gains an "Open in Editor" action (next to the existing delete button). Clicking opens `editor.html?project=<id>&asset=<id>` — loads the asset into the source monitor and creates or resumes the project's active sequence. Accessed from the library via an "Open in Editor" action on each asset card. Opens `editor.html?project=<id>&asset=<id>` — loads the asset into the source monitor and opens the project's most-recent sequence (or creates one named "Sequence 1" if none exists).
The sidebar nav gains an **Editor** link (between Library and Ingest). The sidebar nav gains an **Editor** link (between Library and Ingest).
### 1.2 Source Monitor ### 1.2 Source Monitor
- Displays the currently "loaded" clip (double-click asset in the media panel, or opened via `?asset=` URL param) - Displays the currently loaded clip (double-click in media panel, or via `?asset=` param)
- Native `<video>` element with a custom transport bar (scrub slider, current-time / duration label, play/pause button) - Native `<video>` element with a **custom** transport bar: scrub slider, current-time / duration label, play/pause button
- **Mark In:** keyboard `I` or button — stores `sourceIn` in seconds - **Mark In (`I`):** stores `sourceIn` as seconds; shown as left handle on the scrub bar
- **Mark Out:** keyboard `O` or button — stores `sourceOut` in seconds - **Mark Out (`O`):** stores `sourceOut` as seconds; shown as right handle on the scrub bar
- In/out range shown as a highlighted region on the scrub bar - In/out range highlighted on scrub bar (accent color tint)
- **Insert:** drops the marked range into the timeline at the playhead, shifting downstream clips right - **Insert:** drops marked range at timeline playhead, shifts downstream clips right
- **Overwrite:** drops the marked range at the playhead, overwriting whatever is there - **Overwrite:** drops marked range at timeline playhead, overwrites what's there
- If no in/out marks are set, the full clip duration is used - No marks set → uses full clip duration
### 1.3 Program Monitor ### 1.3 Program Monitor
- Plays back the timeline from the current playhead position - Plays the timeline from the current playhead position
- **Virtual playback:** a single `<video>` element cycles through clips — at each clip's source in-point, the video src is set to that clip's signed proxy URL and `currentTime` is seeked to `sourceIn`. When `currentTime` reaches `sourceOut`, the next clip loads. This is frame-close (not frame-perfect) and is acceptable for v1. - **Virtual playback:** single `<video>` element. On each clip start, set `src` = that clip's signed proxy URL and `currentTime` = `sourceIn`. When `currentTime` reaches `sourceOut`, load the next clip. Brief load gap at clip boundaries is acceptable for v1.
- Timecode display: `HH:MM:SS:FF` derived from timeline position and the sequence frame rate - Timecode display: `HH:MM:SS;FF` at **59.94 fps** (drop-frame notation, semicolon separator)
- Transport shortcuts: `Space` (play/pause), `J` (reverse), `K` (stop), `L` (forward), `←`/`→` (± 1 frame), `Home` (jump to start) - Transport shortcuts: `Space` (play/pause), `J` (reverse), `K` (stop), `L` (forward), `←`/`→` (± 1 frame at 59.94), `Home` (jump to start)
### 1.4 Timeline ### 1.4 Timeline
**Ruler:** Horizontal ruler showing timecode marks. Default scale: 100px/s. Adjustable via `Ctrl + scroll wheel` or a zoom slider. **Ruler:** Timecode-marked horizontal ruler. Default scale: 100px/s. Zoom: `Ctrl + scroll wheel` or zoom slider (range: 20px/s → 500px/s).
**Tracks:** Four track rows — V1, V2 (video), A1, A2 (audio). Each track row has a fixed height (48px). **Tracks:** V1, V2 (video), A1, A2 (audio). Each track row: 48px tall. Track header (40px wide, left): track label, lock toggle.
**Clips:** Absolutely positioned `<div>` elements within track rows. **Clips:** Absolutely positioned `<div>` elements within track rows.
- `left` = `timelineIn * scale` - `left` = `timelineIn_seconds × scale`
- `width` = `(timelineOut - timelineIn) * scale` - `width` = `(timelineOut - timelineIn)_seconds × scale`
- Clip body shows: clip name (truncated), timecode range - Shows: clip name (truncated), source TC range
- If clip is wide enough (> 120px), shows the asset thumbnail centered - If width > 120px: asset thumbnail centered in clip body
**Playhead:** Red vertical line spanning all tracks. Draggable. Clicking the ruler jumps the playhead. **Playhead:** Red vertical line spanning all tracks. Draggable. Clicking the ruler jumps playhead.
**Tools (keyboard shortcuts):** **Tools:**
| Tool | Key | Behavior | | Tool | Key | Behavior |
|------|-----|----------| |------|-----|----------|
| Select | `V` | Click to select (highlight), drag to move horizontally within track, drag left/right edge to trim source in/out | | Select | `V` | Click to select (accent border). Drag body to move horizontally. Drag left/right edge to trim source in/out. |
| Razor | `C` | Click anywhere on a clip to split it into two clips at that exact frame | | Razor | `C` | Click on a clip → splits into two clips at that exact frame. |
| Hand | `H` | Click-drag to pan the timeline horizontally | | Hand | `H` | Click-drag to pan timeline horizontally. |
**Selection + editing:** **Editing:**
- Delete key removes selected clip(s) - `Delete` / `Backspace`: remove selected clip(s)
- Ctrl+Z / Ctrl+Shift+Z: undo / redo (state stored as a local history stack, max 50 steps) - `Ctrl+Z` / `Ctrl+Shift+Z`: undo / redo (local history stack, max 50 steps)
- Clips cannot overlap within the same track (razor and move operations enforce this) - Clips cannot overlap within the same track (enforced on move/razor)
**Auto-save:** Debounced 2s after any timeline change. Visual indicator: subtle "Saving…" / "Saved" text in the timeline toolbar.
### 1.5 Data Model ### 1.5 Data Model
**New migration: `schema_patch_editor.sql`** **New migration: `schema_patch_editor.sql`**
```sql ```sql
-- Sequences (named timelines within a project)
CREATE TABLE sequences ( CREATE TABLE sequences (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
project_id UUID NOT NULL REFERENCES projects ON DELETE CASCADE, project_id UUID NOT NULL REFERENCES projects ON DELETE CASCADE,
name TEXT NOT NULL DEFAULT 'Sequence 1', name TEXT NOT NULL DEFAULT 'Sequence 1',
frame_rate NUMERIC(6,3) NOT NULL DEFAULT 29.97, frame_rate NUMERIC(6,3) NOT NULL DEFAULT 59.94,
width INTEGER NOT NULL DEFAULT 1920, width INTEGER NOT NULL DEFAULT 1920,
height INTEGER NOT NULL DEFAULT 1080, height INTEGER NOT NULL DEFAULT 1080,
created_at TIMESTAMPTZ DEFAULT NOW(), created_at TIMESTAMPTZ DEFAULT NOW(),
@ -104,64 +117,65 @@ CREATE TABLE sequences (
CREATE INDEX idx_sequences_project_id ON sequences(project_id); CREATE INDEX idx_sequences_project_id ON sequences(project_id);
-- Clips on a sequence timeline
CREATE TABLE sequence_clips ( CREATE TABLE sequence_clips (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
sequence_id UUID NOT NULL REFERENCES sequences ON DELETE CASCADE, sequence_id UUID NOT NULL REFERENCES sequences ON DELETE CASCADE,
asset_id UUID NOT NULL REFERENCES assets ON DELETE CASCADE, asset_id UUID NOT NULL REFERENCES assets ON DELETE CASCADE,
track INTEGER NOT NULL DEFAULT 0, track INTEGER NOT NULL DEFAULT 0,
-- 0=V1, 1=V2, 100=A1, 101=A2 -- track encoding: 0=V1, 1=V2, 100=A1, 101=A2
timeline_in_frames BIGINT NOT NULL, timeline_in_frames BIGINT NOT NULL,
timeline_out_frames BIGINT NOT NULL, timeline_out_frames BIGINT NOT NULL,
source_in_frames BIGINT NOT NULL DEFAULT 0, source_in_frames BIGINT NOT NULL DEFAULT 0,
source_out_frames BIGINT NOT NULL, source_out_frames BIGINT NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW(), created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW() updated_at TIMESTAMPTZ DEFAULT NOW()
); );
CREATE INDEX idx_sequence_clips_sequence_id ON sequence_clips(sequence_id); CREATE INDEX idx_sequence_clips_sequence_id ON sequence_clips(sequence_id);
``` ```
**Frame math:** All frame counts use **59.94 fps** (= 60000/1001). Timecode display uses SMPTE drop-frame (`;` separator). Conversion helpers:
- `framesToSeconds(f)` = `f / 59.94`
- `secondsToFrames(s)` = `Math.round(s * 59.94)`
- Drop-frame TC calculation uses standard SMPTE DF algorithm
### 1.6 API Routes ### 1.6 API Routes
All new routes live in `services/mam-api/src/routes/sequences.js`. New file: `services/mam-api/src/routes/sequences.js`
| Method | Path | Description | | Method | Path | Body / Params | Notes |
|--------|------|-------------| |--------|------|--------------|-------|
| GET | `/api/v1/sequences?project_id=X` | List sequences for project | | GET | `/api/v1/sequences` | `?project_id=` | List sequences, ordered by `updated_at DESC` |
| POST | `/api/v1/sequences` | Create sequence (`{ project_id, name, frame_rate, width, height }`) | | POST | `/api/v1/sequences` | `{ project_id, name, frame_rate?, width?, height? }` | Create |
| GET | `/api/v1/sequences/:id` | Get sequence + all clips (joined with asset thumbnail/name/proxy/fps) | | GET | `/api/v1/sequences/:id` | — | Sequence + all clips joined with asset (`display_name`, `fps`, `duration_ms`, `proxy_s3_key`, `thumbnail_s3_key`) |
| PUT | `/api/v1/sequences/:id` | Update sequence metadata | | PUT | `/api/v1/sequences/:id` | `{ name?, frame_rate?, width?, height? }` | Update metadata |
| DELETE | `/api/v1/sequences/:id` | Delete sequence and its clips | | DELETE | `/api/v1/sequences/:id` | — | Cascade deletes clips |
| PUT | `/api/v1/sequences/:id/clips` | Replace entire clip array (full sync) | | PUT | `/api/v1/sequences/:id/clips` | `[ { asset_id, track, timeline_in_frames, timeline_out_frames, source_in_frames, source_out_frames } ]` | **Full replace** — deletes existing clips, inserts new array in one transaction |
| POST | `/api/v1/sequences/:id/export/edl` | Generate CMX3600 EDL string, return as `text/plain` download | | POST | `/api/v1/sequences/:id/export/edl` | — | Returns CMX3600 EDL as `text/plain; charset=utf-8`, `Content-Disposition: attachment` |
**Save strategy:** The editor auto-saves (debounces 2s after any change). Ctrl+S triggers immediate save. The full clip array is sent on each save — simple and avoids fine-grained diff logic.
### 1.7 EDL Export ### 1.7 EDL Export
A new `generateEDL(sequence, clips)` function added to a shared utility (or inline in the export route) using the existing `secondsToTimecode` helper from `worker/src/edl/parser.js` (the logic is duplicated into the API since they're separate services — or promoted to a shared `edl-utils` module). `generateEDL(sequenceName, clips, fps)` function produces CMX3600. Timecode math reuses the logic from `worker/src/edl/parser.js` (copied into a shared util or duplicated in the API route).
Output format: CMX3600 compatible with the existing `conform.js` worker.
``` ```
TITLE: Sequence 1 TITLE: My Sequence
001 clip_filename V C 00:00:00:00 00:00:05:00 00:00:00:00 00:00:05:00 001 clip_filename V C 00:00:00;00 00:00:05;00 00:00:00;00 00:00:05;00
002 other_clip V C 00:00:02:00 00:00:08:00 00:00:05:00 00:00:11:00 002 other_clip V C 00:00:02;00 00:00:08;00 00:00:05;00 00:00:11;00
``` ```
### 1.8 api.js Additions Conforms via the existing `conform.js` BullMQ worker unchanged.
### 1.8 `api.js` Additions
```js ```js
// Sequences
getSequences(projectId) getSequences(projectId)
createSequence(projectId, data) createSequence(data)
getSequence(sequenceId) getSequence(sequenceId)
updateSequence(sequenceId, data) updateSequence(sequenceId, data)
deleteSequence(sequenceId) deleteSequence(sequenceId)
syncSequenceClips(sequenceId, clips) syncSequenceClips(sequenceId, clipsArray)
exportSequenceEDL(sequenceId) // triggers download exportSequenceEDL(sequenceId) // triggers file download
``` ```
--- ---
@ -170,221 +184,173 @@ exportSequenceEDL(sequenceId) // triggers download
### 2.1 Problem ### 2.1 Problem
During SDI capture, two fragmented MP4 streams are piped to S3 via multipart upload. Multipart uploads are invisible in S3 until `CompleteMultipartUpload` — so the proxy doesn't exist until recording stops. The UI has no way to preview an in-progress capture, and there's no way to rewind to review what was just captured. During SDI capture, proxy is piped to S3 via multipart upload — invisible until `CompleteMultipartUpload`. No live preview is possible. For SRT/RTMP, no proxy exists at all until the BullMQ worker runs post-stop.
For SRT/RTMP sources, there is no proxy at all until the BullMQ worker processes the file post-stop. ### 2.2 Solution: HLS Segments During Capture
### 2.2 Solution: HLS Segmentation During Capture Write proxy as HLS segments to local disk during recording. Serve them live from the capture service. On stop: stitch segments → single MP4 → upload to S3 as proxy.
Replace the pipe-to-S3 proxy step with **local HLS segment writing** during capture. After the recording stops, stitch the segments into a single MP4 and upload that to S3 as the proxy. **Why HLS:** Growing playlists are the browser-native live video mechanism. Accumulated segments give free rewind. `hls.js` is a small CDN-loadable polyfill for Chrome/Firefox. FFmpeg's `hls` muxer is proven in production.
**Why HLS:** ### 2.3 FFmpeg Args Change
- Growing playlists are the standard browser mechanism for live video
- Segments accumulate on disk — seeking back to the start of the capture (rewind) is natural
- `hls.js` provides a tiny CDN-loadable library for Chrome/Firefox
- FFmpeg's `hls` muxer is well-tested in production
**FFmpeg proxy args change (SDI — currently spawned as a second process):**
**SDI (second FFmpeg process — proxy process):**
```bash ```bash
# Before (fragmented MP4 → pipe → S3): # Before: fragmented MP4 → pipe → S3
ffmpeg [decklink input] \ ffmpeg [decklink] -c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
-c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
-movflags +frag_keyframe+empty_moov -f mp4 pipe:1 -movflags +frag_keyframe+empty_moov -f mp4 pipe:1
# After (HLS segments → local temp dir): # After: HLS segments → local dir
ffmpeg [decklink input] \ ffmpeg [decklink] -c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
-c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \ -hls_time 2 -hls_list_size 0 -hls_flags append_list \
-hls_time 2 \ -hls_segment_filename '$HLS_DIR/<sessionId>/seg%05d.ts' \
-hls_list_size 0 \ $HLS_DIR/<sessionId>/index.m3u8
-hls_flags append_list \
-hls_segment_filename '/tmp/wd-hls/<sessionId>/seg%05d.ts' \
/tmp/wd-hls/<sessionId>/index.m3u8
``` ```
**Disk budget:** 2s segments at 10 Mbps ≈ 2.5 MB/segment. A 1-hour capture ≈ 9 GB temp disk. The capture host needs ≥ 10 GB free on the temp partition. **SRT/RTMP (single FFmpeg process — two output pads):**
```bash
ffmpeg [srt/rtmp input] \
-c:v prores_ks -profile:v 3 -c:a pcm_s24le \
-movflags +frag_keyframe+empty_moov -f mov pipe:1 \
-c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
-hls_time 2 -hls_list_size 0 -hls_flags append_list \
-hls_segment_filename '$HLS_DIR/<sessionId>/seg%05d.ts' \
$HLS_DIR/<sessionId>/index.m3u8
```
### 2.3 Capture Service Changes ### 2.4 Capture Service Changes
**`capture-manager.js`:** **`capture-manager.js`:**
- New `HLS_SESSION_DIR` env var (default: `/tmp/wd-hls`) - New env var: `HLS_SESSION_DIR` (default: `/tmp/wd-hls`). 3 TB disk — no constraint.
- On `start()`: create `/tmp/wd-hls/<sessionId>/`, write HLS instead of piping to S3 - `start()`: `mkdir -p $HLS_SESSION_DIR/<sessionId>/`, spawn FFmpeg with HLS output. Add `liveUrl: /capture/live/<sessionId>/index.m3u8` to session state.
- `proxyKey` is still generated for the final S3 path; the HLS dir is an intermediate - `stop()`:
- `liveUrl` added to session state: `/capture/live/<sessionId>/index.m3u8` 1. `SIGINT` FFmpeg (existing)
- On `stop()`: 2. Wait for process exit (FFmpeg writes final segment + closes playlist before exiting)
1. `SIGINT` the FFmpeg process (as now) 3. `ffmpeg -i $HLS_DIR/<sessionId>/index.m3u8 -c copy <tmp>.mp4` (stitch)
2. Wait for FFmpeg to flush the final segment and close the playlist 4. Upload stitched MP4 to S3 as proxy key (existing `uploadToS3` helper)
3. Run `ffmpeg -i /tmp/wd-hls/<sessionId>/index.m3u8 -c copy <tmp>.mp4` to stitch 5. `rm -rf $HLS_DIR/<sessionId>/`
4. Upload stitched MP4 to S3 as proxy (using existing `createUploadStream` or direct upload) - Startup: scan `$HLS_SESSION_DIR/` and delete dirs older than 24h
5. `rimraf(/tmp/wd-hls/<sessionId>/)` to clean up
- On startup: purge any `/tmp/wd-hls/` dirs older than 24h (crash recovery)
**New capture routes (`services/capture/src/routes/`):** **New capture routes:**
``` ```
GET /capture/live/:sessionId/index.m3u8 → stream the HLS playlist file GET /capture/live/:sessionId/index.m3u8 → serve HLS playlist file
GET /capture/live/:sessionId/:segment → stream a .ts segment file GET /capture/live/:sessionId/:file → serve .ts segment files
``` ```
Both routes check that the session is active (or was recently active) before serving. Both routes: check session exists (active or recently stopped with dir still present), stream file from disk with correct `Content-Type` (`application/vnd.apple.mpegurl` / `video/mp2t`).
**Updated `/capture/status` response:** **Updated `/capture/status` response:**
```json ```json
{ {
"recording": true, "recording": true,
"sessionId": "...", "sessionId": "abc123",
"liveUrl": "/capture/live/<sessionId>/index.m3u8", "liveUrl": "/capture/live/abc123/index.m3u8",
...existing fields... "startedAt": "...",
"duration": 42,
...
} }
``` ```
**nginx proxy:** Add location block to proxy `/capture/live/` through to the capture service (same pattern as existing `/capture/` pass-through). **nginx:** Add location block for `/capture/live/` → proxy_pass to capture service (same pattern as existing `/capture/` block).
### 2.4 SRT/RTMP Sources ### 2.5 Live Preview in `capture.html`
Single FFmpeg process limitation: network streams can only be opened once. Use FFmpeg's multiple-output capability: When status poll returns `recording: true` with `liveUrl`:
1. Show collapsible **Live Preview** panel beneath capture controls
```bash 2. Load hls.js from CDN: `cdnjs.cloudflare.com/ajax/libs/hls.js/1.5.15/hls.min.js`
ffmpeg [srt/rtmp input] \
-c:v prores_ks -profile:v 3 -c:a pcm_s24le \
-movflags +frag_keyframe+empty_moov -f mov pipe:1 \ ← hires → S3 (unchanged)
-c:v libx264 -preset fast -b:v 10M -c:a aac \
-hls_time 2 -hls_list_size 0 -hls_flags append_list \
-hls_segment_filename '/tmp/wd-hls/<sessionId>/seg%05d.ts' \
/tmp/wd-hls/<sessionId>/index.m3u8 ← proxy → local HLS
```
The hires upload stream continues unchanged. The proxy no longer goes to S3 during capture — it goes to HLS segments, then stitched on stop.
### 2.5 Live Preview UI (`capture.html`)
When the status poll returns `recording: true` with a `liveUrl`:
1. Show a collapsible **Live Preview** panel below the capture controls
2. Load `hls.js` from CDN: `cdnjs.cloudflare.com/ajax/libs/hls.js/1.5.7/hls.min.js`
3. `new Hls()` → `hls.loadSource(status.liveUrl)` → `hls.attachMedia(videoEl)` 3. `new Hls()` → `hls.loadSource(status.liveUrl)` → `hls.attachMedia(videoEl)`
4. `video.play()` on `MANIFEST_PARSED` event — plays at the live edge by default 4. `videoEl.play()` on `MANIFEST_PARSED` (plays at live edge)
5. **"⏮ Rewind"** button: `video.currentTime = 0` — jumps to the start of the capture 5. **⏮ Rewind** button: `videoEl.currentTime = 0`
6. Elapsed time counter ticking from `startedAt` 6. Elapsed time counter from `status.startedAt`
When recording stops: destroy the HLS instance, hide the preview panel. The asset appears in the library once the proxy upload and thumbnail job complete. On recording stop: `hls.destroy()`, hide preview panel. Asset appears in library after proxy upload + thumbnail job complete.
--- ---
## 3. Feature Additions (Prioritized) ## 3. Feature Additions (Prioritized)
After Phase 1 (editor) and Phase 2 (growing file) are done, implement in this order:
### P1 — Improved Player (`player.html` rebuild) ### P1 — Improved Player (`player.html` rebuild)
Current `player.html` uses the old CSS variable naming convention (`--color-bg-tertiary`, etc.) and doesn't match the redesigned system. Rebuild it to match `index.html`'s design tokens and add: - Migrate from old CSS variables (`--color-bg-tertiary`) to current design tokens (`--bg-panel`, etc.)
- Custom transport bar with scrub slider (replace browser defaults) - Replace browser default controls with custom transport bar: scrub slider, timecode display `HH:MM:SS;FF` at 59.94 fps, frame-step buttons, J/K/L shortcuts
- Timecode display `HH:MM:SS:FF` (using `fps` from asset metadata) - Inline rename: click `display_name` to edit in place → auto-save
- Frame-step buttons (± 1 frame) - "Open in Editor" button → `editor.html?asset=<id>`
- J/K/L keyboard shortcuts
- Inline rename: click `display_name` to edit in place
- "Open in Editor" button linking to `editor.html?asset=<id>`
### P2 — Subclips ### P2 — Subclips
In `player.html`, after marking in/out points: - Player shows In/Out markers → "Create Subclip" button
- "Create Subclip" button → `POST /api/v1/assets` with: - `POST /api/v1/assets` with `{ parent_asset_id, subclip_in_ms, subclip_out_ms, display_name }`
```json - New columns: `parent_asset_id UUID REFERENCES assets`, `subclip_in_ms BIGINT`, `subclip_out_ms BIGINT`
{ "parent_asset_id": "...", "subclip_in_ms": 12000, "subclip_out_ms": 45000, ... } - Library shows subclip cards with ✂ badge; player pre-seeks to `subclip_in_ms`
``` - Subclips use parent's proxy S3 key — no re-transcode
- New columns on `assets`: `parent_asset_id UUID REFERENCES assets`, `subclip_in_ms BIGINT`, `subclip_out_ms BIGINT`
- Library shows subclip cards with a "✂" badge; clicking opens the player pre-seeked to `subclip_in_ms`
- Subclips use the same proxy S3 key as the parent; no extra transcoding needed
### P3 — Multi-select & Bulk Ops ### P3 — Multi-select & Bulk Ops
In `index.html`: - Shift-click or checkbox (visible on hover) to select multiple asset cards
- Shift-click or checkbox (appears on hover) to select multiple asset cards - Floating action bar: **Move to bin | Add tags | Delete**
- Floating action bar appears at bottom when ≥1 card selected: **Move to bin | Add tags | Delete** - Move to bin: slide panel with bin tree, `PATCH /assets/:id { bin_id }` for each
- "Move to bin": slide panel with bin tree, moves selected assets via `PATCH /assets/:id { bin_id }` - Add tags: appends to all selected assets
- "Add tags": text input, appends tags to all selected assets - Bulk delete: confirm modal → soft-delete
- Bulk delete: confirm modal, then soft-delete all
### P4 — Waveform Display in Editor ### P4 — Waveform Display in Editor
During proxy generation (`worker/src/workers/proxy.js`): - In proxy worker (`proxy.js`): after transcode, run FFmpeg `astats` filter to generate per-second peak arrays → store as `waveforms/<assetId>.json` in S3
- After transcoding, run FFmpeg's `volumedetect` or `astats` filter to produce per-second peak arrays - New `waveform_s3_key TEXT` column on `assets`
- Alternatively: `ffmpeg -i proxy.mp4 -af astats=metadata=1:reset=1,ametadata=print:file=peaks.txt -f null -` - Editor timeline: fetch waveform JSON for audio track clips, render as SVG polyline within clip block
- Parse and store as `{ peaks: [0.2, 0.8, ...] }` JSON → upload to `waveforms/<assetId>.json` in S3
- New column on `assets`: `waveform_s3_key TEXT`
- Editor timeline: for audio track clips, fetch the waveform JSON and render an SVG polyline within the clip block
### P5 — Timecoded Comments ### P5 — Timecoded Comments
New table:
```sql ```sql
CREATE TABLE asset_comments ( CREATE TABLE asset_comments (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
asset_id UUID NOT NULL REFERENCES assets ON DELETE CASCADE, asset_id UUID NOT NULL REFERENCES assets ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users, user_id UUID NOT NULL REFERENCES users,
timecode_seconds NUMERIC(10,3), timecode_seconds NUMERIC(10,3),
body TEXT NOT NULL, body TEXT NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW() created_at TIMESTAMPTZ DEFAULT NOW()
); );
``` ```
In `player.html` sidebar: "Comments" tab. Input field with "Post at [current TC]" button. Renders as a list of clickable entries that seek the video to that timecode. Player sidebar: "Comments" tab with "Post at current TC" button. Clickable entries seek video to that timecode.
### P6 — Smart Bins ### P6 — Smart Bins
Extend `bins` table:
```sql ```sql
ALTER TABLE bins ADD COLUMN is_smart BOOLEAN DEFAULT false; ALTER TABLE bins ADD COLUMN is_smart BOOLEAN DEFAULT false;
ALTER TABLE bins ADD COLUMN smart_query JSONB; ALTER TABLE bins ADD COLUMN smart_query JSONB;
-- smart_query shape: { "tags": ["interview"], "media_type": "video", "created_after": "2026-01-01" }
``` ```
Smart bin `smart_query` shape: `{ "tags": ["interview"], "media_type": "video", "created_after": "2026-01-01" }`. Smart bin assets: dynamically queried from `assets` table. Shows ✦ icon in bin tree.
When loading a smart bin's assets, run a dynamically built query against the `assets` table.
In the UI, smart bins show a ✦ icon in the bin tree and their contents cannot be manually modified.
### P7 — Metadata Templates ### P7 — Metadata Templates
New table:
```sql ```sql
CREATE TABLE project_metadata_fields ( CREATE TABLE project_metadata_fields (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
project_id UUID NOT NULL REFERENCES projects ON DELETE CASCADE, project_id UUID NOT NULL REFERENCES projects ON DELETE CASCADE,
field_key TEXT NOT NULL, field_key TEXT NOT NULL,
label TEXT NOT NULL, label TEXT NOT NULL,
field_type TEXT NOT NULL DEFAULT 'text', -- text, number, date, select field_type TEXT NOT NULL DEFAULT 'text',
options JSONB, -- for select type options JSONB,
required BOOLEAN DEFAULT false, required BOOLEAN DEFAULT false,
sort_order INTEGER DEFAULT 0 sort_order INTEGER DEFAULT 0
); );
ALTER TABLE assets ADD COLUMN metadata JSONB DEFAULT '{}';
``` ```
New `metadata JSONB` column on `assets`. Player shows project-defined fields in the sidebar; values saved via `PATCH /assets/:id { metadata: {...} }`.
--- ---
## Implementation Sequencing ## Implementation Sequencing
### Phase 1 — Editor (estimated 3–4 dev sessions) ### Phase 1 — Editor
1. DB migration: `sequences` + `sequence_clips` 1. DB migration (`schema_patch_editor.sql`)
2. API: `sequences.js` route file (CRUD + clip sync + EDL export) 2. API (`sequences.js` route — CRUD + clip sync + EDL export)
3. `api.js`: add sequence helper functions 3. `api.js` — sequence helpers
4. `editor.html`: shell layout + CSS (4-panel, sidebar nav update) 4. `editor.html` — 4-panel shell + CSS (sidebar nav update)
5. Timeline engine: ruler, playhead, track rows, clip rendering 5. Timeline engine — ruler, playhead, track rows, clip rendering
6. Select tool: click-select, drag-move, drag-edge trim 6. Select tool — click-select, drag-move, drag-edge trim
7. Razor tool: click-to-split 7. Razor tool — click-to-split
8. Source monitor: video + transport + in/out marking + Insert/Overwrite 8. Source monitor — video + transport + in/out marking + Insert/Overwrite
9. Program monitor: virtual playback + timecode display 9. Program monitor — virtual playback + 59.94 drop-frame timecode
10. Library integration: "Open in Editor" card action + sidebar link 10. Auto-save (debounced 2s) + sequence picker in media panel
11. Library "Open in Editor" action + sidebar link
### Phase 2 — Growing File (estimated 2 dev sessions) ### Phase 2 — Growing File
1. Capture service: HLS output instead of proxy pipe-to-S3 1. Capture service: HLS output, `$HLS_SESSION_DIR` env var
2. Capture service: new `/capture/live/:sessionId/` HTTP routes 2. Capture service: `/capture/live/:sessionId/` routes + nginx config
3. Capture service: on-stop stitch + S3 upload, cleanup 3. Capture service: on-stop stitch + S3 upload + cleanup
4. nginx: add `/capture/live/` location block 4. `capture.html`: live preview panel (hls.js), rewind button
5. `capture.html`: live preview panel with hls.js, rewind button
6. `api.js`: handle `liveUrl` in status response
### Phase 3 — Feature Additions (ongoing, in priority order) ### Phase 3 — Feature Additions
P1 → P2 → P3 → P4 → P5 → P6 → P7 P1 → P2 → P3 → P4 → P5 → P6 → P7
---
## Open Questions (for user review before implementation begins)
1. **Auto-save vs. explicit save:** The design calls for auto-save (debounced 2s). Do you want an explicit Ctrl+S only mode, or both (auto-save + Ctrl+S for immediate flush)?
2. **Multiple sequences per project:** The design allows multiple named sequences per project (like Premiere's Project panel). Or would you prefer one active sequence per project to keep things simpler?
3. **HLS temp disk:** Is there ≥ 10 GB spare disk available on the capture service host for HLS segments during recording?
4. **Frame rates in use:** What frame rates are you primarily working in? (23.976, 25, 29.97, 30, 59.94?) The EDL timecode generator needs to know.
5. **Program monitor fidelity:** For v1, is clip-by-clip virtual playback (brief gap at clip boundaries while the next video src loads) acceptable? Or do you need gapless/frame-accurate playback from day one?