From 67251a0dcda42ae210db507ceafc97b1dcc1063e Mon Sep 17 00:00:00 2001
From: ZGaetano <zgaetano@wilddragon.net>
Date: Mon, 18 May 2026 19:14:00 -0400
Subject: [PATCH] docs: add editor + growing file + feature expansion design
 spec

---
 ...-18-editor-growing-file-features-design.md | 390 ++++++++++++++++++
 1 file changed, 390 insertions(+)
 create mode 100644 docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md

diff --git a/docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md b/docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md
new file mode 100644
index 0000000..4618f3d
--- /dev/null
+++ b/docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md
@@ -0,0 +1,390 @@
+# Wild Dragon MAM — Editor, Growing File Preview & Feature Expansion
+**Date:** 2026-05-18  
+**Status:** Draft — awaiting user approval before implementation begins
+
+---
+
+## Overview
+
+Three interconnected areas, sequenced by priority:
+
+1. **NLE Editor** (`editor.html`) — Premiere-style timeline editor with razor blade, in/out marking, and EDL export
+2. **Growing File Workflow** — HLS live preview during SDI/SRT/RTMP capture, with rewind support
+3. **Feature Additions** — Prioritized list of high-value additions (subclips, player improvements, bulk ops, waveform, etc.)
+
+---
+
+## 1. NLE Editor
+
+### 1.1 Layout
+
+A new page `editor.html` replaces the role of `player.html` for editorial work. `player.html` is kept as the lightweight browse/metadata view; the editor is the full-screen creative environment.
+
+```
+┌─ SOURCE MONITOR ──────┬─ PROGRAM MONITOR ──────┐   ← 40vh
+│  [video preview]      │  [video preview]        │
+│  TC: 00:00:00:00      │  TC: 00:00:00:00        │
+│  [────scrub bar────]  │  [────scrub bar────]    │
+│  [IN] [OUT] [Insert]  │  [J] [K] [L]  [⏩]     │
+├─ MEDIA PANEL ─────────┴─ TIMELINE ─────────────┤   ← 60vh
+│ [bin tree]    │  [V] [C] [H]  [──zoom──]        │
+│ [asset list]  │  ruler: 00:00  00:05  00:10 …   │
+│               │  V1 ░░░░[clip A]░░░░[clip B]░░  │
+│               │  V2                              │
+│               │  A1 ░░░░[clip A]░░░░[clip B]░░  │
+│               │  A2                              │
+└───────────────┴──────────────────────────────────┘
+```
+
+The editor is accessed from the library: each asset card gains an "Open in Editor" action (next to the existing delete button). Clicking opens `editor.html?project=<id>&asset=<id>` — loads the asset into the source monitor and creates or resumes the project's active sequence.
+
+The sidebar nav gains an **Editor** link (between Library and Ingest).
+
+### 1.2 Source Monitor
+
+- Displays the currently "loaded" clip (double-click asset in the media panel, or opened via `?asset=` URL param)
+- Native `<video>` element with a custom transport bar (scrub slider, current-time / duration label, play/pause button)
+- **Mark In:** keyboard `I` or button — stores `sourceIn` in seconds
+- **Mark Out:** keyboard `O` or button — stores `sourceOut` in seconds
+- In/out range shown as a highlighted region on the scrub bar
+- **Insert:** drops the marked range into the timeline at the playhead, shifting downstream clips right
+- **Overwrite:** drops the marked range at the playhead, overwriting whatever is there
+- If no in/out marks are set, the full clip duration is used
+
+### 1.3 Program Monitor
+
+- Plays back the timeline from the current playhead position
+- **Virtual playback:** a single `<video>` element cycles through clips — at each clip's source in-point, the video src is set to that clip's signed proxy URL and `currentTime` is seeked to `sourceIn`. When `currentTime` reaches `sourceOut`, the next clip loads. This is frame-close (not frame-perfect) and is acceptable for v1.
+- Timecode display: `HH:MM:SS:FF` derived from timeline position and the sequence frame rate
+- Transport shortcuts: `Space` (play/pause), `J` (reverse), `K` (stop), `L` (forward), `←`/`→` (± 1 frame), `Home` (jump to start)
+
+### 1.4 Timeline
+
+**Ruler:** Horizontal ruler showing timecode marks. Default scale: 100px/s. Adjustable via `Ctrl + scroll wheel` or a zoom slider.
+
+**Tracks:** Four track rows — V1, V2 (video), A1, A2 (audio). Each track row has a fixed height (48px).
+
+**Clips:** Absolutely positioned `<div>` elements within track rows.
+- `left` = `timelineIn * scale`
+- `width` = `(timelineOut - timelineIn) * scale`
+- Clip body shows: clip name (truncated), timecode range
+- If clip is wide enough (> 120px), shows the asset thumbnail centered
+
+**Playhead:** Red vertical line spanning all tracks. Draggable. Clicking the ruler jumps the playhead.
+
+**Tools (keyboard shortcuts):**
+
+| Tool | Key | Behavior |
+|------|-----|----------|
+| Select | `V` | Click to select (highlight), drag to move horizontally within track, drag left/right edge to trim source in/out |
+| Razor | `C` | Click anywhere on a clip to split it into two clips at that exact frame |
+| Hand | `H` | Click-drag to pan the timeline horizontally |
+
+**Selection + editing:**
+- Delete key removes selected clip(s)
+- Ctrl+Z / Ctrl+Shift+Z: undo / redo (state stored as a local history stack, max 50 steps)
+- Clips cannot overlap within the same track (razor and move operations enforce this)
+
+### 1.5 Data Model
+
+**New migration: `schema_patch_editor.sql`**
+
+```sql
+-- Sequences (named timelines within a project)
+CREATE TABLE sequences (
+  id          UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  project_id  UUID NOT NULL REFERENCES projects ON DELETE CASCADE,
+  name        TEXT NOT NULL DEFAULT 'Sequence 1',
+  frame_rate  NUMERIC(6,3) NOT NULL DEFAULT 29.97,
+  width       INTEGER NOT NULL DEFAULT 1920,
+  height      INTEGER NOT NULL DEFAULT 1080,
+  created_at  TIMESTAMPTZ DEFAULT NOW(),
+  updated_at  TIMESTAMPTZ DEFAULT NOW()
+);
+
+CREATE INDEX idx_sequences_project_id ON sequences(project_id);
+
+-- Clips on a sequence timeline
+CREATE TABLE sequence_clips (
+  id                    UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  sequence_id           UUID NOT NULL REFERENCES sequences ON DELETE CASCADE,
+  asset_id              UUID NOT NULL REFERENCES assets ON DELETE CASCADE,
+  track                 INTEGER NOT NULL DEFAULT 0,
+  -- 0=V1, 1=V2, 100=A1, 101=A2
+  timeline_in_frames    BIGINT NOT NULL,
+  timeline_out_frames   BIGINT NOT NULL,
+  source_in_frames      BIGINT NOT NULL DEFAULT 0,
+  source_out_frames     BIGINT NOT NULL,
+  created_at            TIMESTAMPTZ DEFAULT NOW(),
+  updated_at            TIMESTAMPTZ DEFAULT NOW()
+);
+
+CREATE INDEX idx_sequence_clips_sequence_id ON sequence_clips(sequence_id);
+```
+
+### 1.6 API Routes
+
+All new routes live in `services/mam-api/src/routes/sequences.js`.
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/v1/sequences?project_id=X` | List sequences for project |
+| POST | `/api/v1/sequences` | Create sequence (`{ project_id, name, frame_rate, width, height }`) |
+| GET | `/api/v1/sequences/:id` | Get sequence + all clips (joined with asset thumbnail/name/proxy/fps) |
+| PUT | `/api/v1/sequences/:id` | Update sequence metadata |
+| DELETE | `/api/v1/sequences/:id` | Delete sequence and its clips |
+| PUT | `/api/v1/sequences/:id/clips` | Replace entire clip array (full sync) |
+| POST | `/api/v1/sequences/:id/export/edl` | Generate CMX3600 EDL string, return as `text/plain` download |
+
+**Save strategy:** The editor auto-saves (debounces 2s after any change). Ctrl+S triggers immediate save. The full clip array is sent on each save — simple and avoids fine-grained diff logic.
+
+### 1.7 EDL Export
+
+A new `generateEDL(sequence, clips)` function added to a shared utility (or inline in the export route) using the existing `secondsToTimecode` helper from `worker/src/edl/parser.js` (the logic is duplicated into the API since they're separate services — or promoted to a shared `edl-utils` module).
+
+Output format: CMX3600 compatible with the existing `conform.js` worker.
+
+```
+TITLE: Sequence 1
+
+001  clip_filename  V  C  00:00:00:00 00:00:05:00 00:00:00:00 00:00:05:00
+002  other_clip     V  C  00:00:02:00 00:00:08:00 00:00:05:00 00:00:11:00
+```
+
+### 1.8 api.js Additions
+
+```js
+// Sequences
+getSequences(projectId)
+createSequence(projectId, data)
+getSequence(sequenceId)
+updateSequence(sequenceId, data)
+deleteSequence(sequenceId)
+syncSequenceClips(sequenceId, clips)
+exportSequenceEDL(sequenceId)  // triggers download
+```
+
+---
+
+## 2. Growing File Workflow
+
+### 2.1 Problem
+
+During SDI capture, two fragmented MP4 streams are piped to S3 via multipart upload. Multipart uploads are invisible in S3 until `CompleteMultipartUpload` — so the proxy doesn't exist until recording stops. The UI has no way to preview an in-progress capture, and there's no way to rewind to review what was just captured.
+
+For SRT/RTMP sources, there is no proxy at all until the BullMQ worker processes the file post-stop.
+
+### 2.2 Solution: HLS Segmentation During Capture
+
+Replace the pipe-to-S3 proxy step with **local HLS segment writing** during capture. After the recording stops, stitch the segments into a single MP4 and upload that to S3 as the proxy.
+
+**Why HLS:**
+- Growing playlists are the standard browser mechanism for live video
+- Segments accumulate on disk — seeking back to the start of the capture (rewind) is natural
+- `hls.js` provides a tiny CDN-loadable library for Chrome/Firefox
+- FFmpeg's `hls` muxer is well-tested in production
+
+**FFmpeg proxy args change (SDI — currently spawned as a second process):**
+
+```bash
+# Before (fragmented MP4 → pipe → S3):
+ffmpeg [decklink input] \
+  -c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
+  -movflags +frag_keyframe+empty_moov -f mp4 pipe:1
+
+# After (HLS segments → local temp dir):
+ffmpeg [decklink input] \
+  -c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
+  -hls_time 2 \
+  -hls_list_size 0 \
+  -hls_flags append_list \
+  -hls_segment_filename '/tmp/wd-hls/<sessionId>/seg%05d.ts' \
+  /tmp/wd-hls/<sessionId>/index.m3u8
+```
+
+**Disk budget:** 2s segments at 10 Mbps ≈ 2.5 MB/segment. A 1-hour capture ≈ 9 GB temp disk. The capture host needs ≥ 10 GB free on the temp partition.
+
+### 2.3 Capture Service Changes
+
+**`capture-manager.js`:**
+- New `HLS_SESSION_DIR` env var (default: `/tmp/wd-hls`)
+- On `start()`: create `/tmp/wd-hls/<sessionId>/`, write HLS instead of piping to S3
+- `proxyKey` is still generated for the final S3 path; the HLS dir is an intermediate
+- `liveUrl` added to session state: `/capture/live/<sessionId>/index.m3u8`
+- On `stop()`:
+  1. `SIGINT` the FFmpeg process (as now)
+  2. Wait for FFmpeg to flush the final segment and close the playlist
+  3. Run `ffmpeg -i /tmp/wd-hls/<sessionId>/index.m3u8 -c copy <tmp>.mp4` to stitch
+  4. Upload stitched MP4 to S3 as proxy (using existing `createUploadStream` or direct upload)
+  5. `rimraf(/tmp/wd-hls/<sessionId>/)` to clean up
+- On startup: purge any `/tmp/wd-hls/` dirs older than 24h (crash recovery)
+
+**New capture routes (`services/capture/src/routes/`):**
+```
+GET /capture/live/:sessionId/index.m3u8   → stream the HLS playlist file
+GET /capture/live/:sessionId/:segment     → stream a .ts segment file
+```
+Both routes check that the session is active (or was recently active) before serving.
+
+**Updated `/capture/status` response:**
+```json
+{
+  "recording": true,
+  "sessionId": "...",
+  "liveUrl": "/capture/live/<sessionId>/index.m3u8",
+  ...existing fields...
+}
+```
+
+**nginx proxy:** Add location block to proxy `/capture/live/` through to the capture service (same pattern as existing `/capture/` pass-through).
+
+### 2.4 SRT/RTMP Sources
+
+Single FFmpeg process limitation: network streams can only be opened once. Use FFmpeg's multiple-output capability:
+
+```bash
+ffmpeg [srt/rtmp input] \
+  -c:v prores_ks -profile:v 3 -c:a pcm_s24le \
+  -movflags +frag_keyframe+empty_moov -f mov pipe:1 \   ← hires → S3 (unchanged)
+  -c:v libx264 -preset fast -b:v 10M -c:a aac \
+  -hls_time 2 -hls_list_size 0 -hls_flags append_list \
+  -hls_segment_filename '/tmp/wd-hls/<sessionId>/seg%05d.ts' \
+  /tmp/wd-hls/<sessionId>/index.m3u8                   ← proxy → local HLS
+```
+
+The hires upload stream continues unchanged. The proxy no longer goes to S3 during capture — it goes to HLS segments, then stitched on stop.
+
+### 2.5 Live Preview UI (`capture.html`)
+
+When the status poll returns `recording: true` with a `liveUrl`:
+
+1. Show a collapsible **Live Preview** panel below the capture controls
+2. Load `hls.js` from CDN: `cdnjs.cloudflare.com/ajax/libs/hls.js/1.5.7/hls.min.js`
+3. `new Hls()` → `hls.loadSource(status.liveUrl)` → `hls.attachMedia(videoEl)`
+4. `video.play()` on `MANIFEST_PARSED` event — plays at the live edge by default
+5. **"⏮ Rewind"** button: `video.currentTime = 0` — jumps to the start of the capture
+6. Elapsed time counter ticking from `startedAt`
+
+When recording stops: destroy the HLS instance, hide the preview panel. The asset appears in the library once the proxy upload and thumbnail job complete.
+
+---
+
+## 3. Feature Additions (Prioritized)
+
+After Phase 1 (editor) and Phase 2 (growing file) are done, implement in this order:
+
+### P1 — Improved Player (`player.html` rebuild)
+Current `player.html` uses the old CSS variable naming convention (`--color-bg-tertiary`, etc.) and doesn't match the redesigned system. Rebuild it to match `index.html`'s design tokens and add:
+- Custom transport bar with scrub slider (replace browser defaults)
+- Timecode display `HH:MM:SS:FF` (using `fps` from asset metadata)
+- Frame-step buttons (± 1 frame)
+- J/K/L keyboard shortcuts
+- Inline rename: click `display_name` to edit in place
+- "Open in Editor" button linking to `editor.html?asset=<id>`
+
+### P2 — Subclips
+In `player.html`, after marking in/out points:
+- "Create Subclip" button → `POST /api/v1/assets` with:
+  ```json
+  { "parent_asset_id": "...", "subclip_in_ms": 12000, "subclip_out_ms": 45000, ... }
+  ```
+- New columns on `assets`: `parent_asset_id UUID REFERENCES assets`, `subclip_in_ms BIGINT`, `subclip_out_ms BIGINT`
+- Library shows subclip cards with a "✂" badge; clicking opens the player pre-seeked to `subclip_in_ms`
+- Subclips use the same proxy S3 key as the parent; no extra transcoding needed
+
+### P3 — Multi-select & Bulk Ops
+In `index.html`:
+- Shift-click or checkbox (appears on hover) to select multiple asset cards
+- Floating action bar appears at bottom when ≥1 card selected: **Move to bin | Add tags | Delete**
+- "Move to bin": slide panel with bin tree, moves selected assets via `PATCH /assets/:id { bin_id }`
+- "Add tags": text input, appends tags to all selected assets
+- Bulk delete: confirm modal, then soft-delete all
+
+### P4 — Waveform Display in Editor
+During proxy generation (`worker/src/workers/proxy.js`):
+- After transcoding, run FFmpeg's `volumedetect` or `astats` filter to produce per-second peak arrays
+- Alternatively: `ffmpeg -i proxy.mp4 -af astats=metadata=1:reset=1,ametadata=print:file=peaks.txt -f null -`
+- Parse and store as `{ peaks: [0.2, 0.8, ...] }` JSON → upload to `waveforms/<assetId>.json` in S3
+- New column on `assets`: `waveform_s3_key TEXT`
+- Editor timeline: for audio track clips, fetch the waveform JSON and render an SVG polyline within the clip block
+
+### P5 — Timecoded Comments
+New table:
+```sql
+CREATE TABLE asset_comments (
+  id              UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  asset_id        UUID NOT NULL REFERENCES assets ON DELETE CASCADE,
+  user_id         UUID NOT NULL REFERENCES users,
+  timecode_seconds NUMERIC(10,3),
+  body            TEXT NOT NULL,
+  created_at      TIMESTAMPTZ DEFAULT NOW()
+);
+```
+In `player.html` sidebar: "Comments" tab. Input field with "Post at [current TC]" button. Renders as a list of clickable entries that seek the video to that timecode.
+
+### P6 — Smart Bins
+Extend `bins` table:
+```sql
+ALTER TABLE bins ADD COLUMN is_smart BOOLEAN DEFAULT false;
+ALTER TABLE bins ADD COLUMN smart_query JSONB;
+```
+Smart bin `smart_query` shape: `{ "tags": ["interview"], "media_type": "video", "created_after": "2026-01-01" }`.
+When loading a smart bin's assets, run a dynamically built query against the `assets` table.
+In the UI, smart bins show a ✦ icon in the bin tree and their contents cannot be manually modified.
+
+### P7 — Metadata Templates
+New table:
+```sql
+CREATE TABLE project_metadata_fields (
+  id          UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
+  project_id  UUID NOT NULL REFERENCES projects ON DELETE CASCADE,
+  field_key   TEXT NOT NULL,
+  label       TEXT NOT NULL,
+  field_type  TEXT NOT NULL DEFAULT 'text', -- text, number, date, select
+  options     JSONB,                         -- for select type
+  required    BOOLEAN DEFAULT false,
+  sort_order  INTEGER DEFAULT 0
+);
+```
+New `metadata JSONB` column on `assets`. Player shows project-defined fields in the sidebar; values saved via `PATCH /assets/:id { metadata: {...} }`.
+
+---
+
+## Implementation Sequencing
+
+### Phase 1 — Editor (estimated 3–4 dev sessions)
+1. DB migration: `sequences` + `sequence_clips`
+2. API: `sequences.js` route file (CRUD + clip sync + EDL export)
+3. `api.js`: add sequence helper functions
+4. `editor.html`: shell layout + CSS (4-panel, sidebar nav update)
+5. Timeline engine: ruler, playhead, track rows, clip rendering
+6. Select tool: click-select, drag-move, drag-edge trim
+7. Razor tool: click-to-split
+8. Source monitor: video + transport + in/out marking + Insert/Overwrite
+9. Program monitor: virtual playback + timecode display
+10. Library integration: "Open in Editor" card action + sidebar link
+
+### Phase 2 — Growing File (estimated 2 dev sessions)
+1. Capture service: HLS output instead of proxy pipe-to-S3
+2. Capture service: new `/capture/live/:sessionId/` HTTP routes
+3. Capture service: on-stop stitch + S3 upload, cleanup
+4. nginx: add `/capture/live/` location block
+5. `capture.html`: live preview panel with hls.js, rewind button
+6. `api.js`: handle `liveUrl` in status response
+
+### Phase 3 — Feature Additions (ongoing, in priority order)
+P1 → P2 → P3 → P4 → P5 → P6 → P7
+
+---
+
+## Open Questions (for user review before implementation begins)
+
+1. **Auto-save vs. explicit save:** The design calls for auto-save (debounced 2s). Do you want an explicit Ctrl+S only mode, or both (auto-save + Ctrl+S for immediate flush)?
+
+2. **Multiple sequences per project:** The design allows multiple named sequences per project (like Premiere's Project panel). Or would you prefer one active sequence per project to keep things simpler?
+
+3. **HLS temp disk:** Is there ≥ 10 GB spare disk available on the capture service host for HLS segments during recording?
+
+4. **Frame rates in use:** What frame rates are you primarily working in? (23.976, 25, 29.97, 30, 59.94?) The EDL timecode generator needs to know.
+
+5. **Program monitor fidelity:** For v1, is clip-by-clip virtual playback (brief gap at clip boundaries while the next video src loads) acceptable? Or do you need gapless/frame-accurate playback from day one?