dragonflight/docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md

# Wild Dragon MAM — Editor, Growing File Preview & Feature Expansion
**Date:** 2026-05-18  
**Status:** APPROVED — ready for implementation planning

---

## Resolved Decisions

| Question | Decision |
|----------|----------|
| Save strategy | **Auto-save** (debounced 2s after any change) |
| Sequences per project | **Multiple named sequences** (like Premiere's project panel) |
| HLS temp disk | **3 TB available** — no constraint |
| Primary frame rate | **59.94 fps** |
| Program monitor fidelity | **Brief gap at clip boundaries is acceptable for v1** |

---

## Overview

Three sequenced phases:

1. **NLE Editor** (`editor.html`) — Premiere-style timeline editor with razor blade, in/out marking, and EDL export
2. **Growing File Workflow** — HLS live preview during SDI/SRT/RTMP capture, with rewind support
3. **Feature Additions** — Subclips, improved player, bulk ops, waveform, timecoded comments, smart bins, metadata templates

---

## 1. NLE Editor

### 1.1 Layout

A new page `editor.html`. `player.html` is kept as the lightweight browse/metadata view; the editor is the full-screen creative environment.

```
┌─ SOURCE MONITOR ──────┬─ PROGRAM MONITOR ──────┐   ← 40vh
│  [video preview]      │  [video preview]        │
│  TC: 00:00:00;00      │  TC: 00:00:00;00        │
│  [────scrub bar────]  │  [────scrub bar────]    │
│  [IN] [OUT] [Insert]  │  [J] [K] [L]  [⏩]     │
├─ MEDIA PANEL ─────────┴─ TIMELINE ─────────────┤   ← 60vh
│ [sequence picker]     │  [V] [C] [H]  [zoom]    │
│ [bin tree]            │  ruler: 00:00  00:05 …  │
│ [asset list]          │  V1 ░░░░[clip A]░░░░    │
│                       │  V2                     │
│                       │  A1 ░░░░[clip A]░░░░    │
│                       │  A2                     │
└───────────────────────┴─────────────────────────┘
```

Accessed from the library via an "Open in Editor" action on each asset card. Opens `editor.html?project=<id>&asset=<id>` — loads the asset into the source monitor and opens the project's most-recent sequence (or creates one named "Sequence 1" if none exists).

The sidebar nav gains an **Editor** link (between Library and Ingest).

### 1.2 Source Monitor

- Displays the currently loaded clip (double-click in media panel, or via `?asset=` param)
- Native `<video>` element with a **custom** transport bar: scrub slider, current-time / duration label, play/pause button
- **Mark In (`I`):** stores `sourceIn` as seconds; shown as left handle on the scrub bar
- **Mark Out (`O`):** stores `sourceOut` as seconds; shown as right handle on the scrub bar
- In/out range highlighted on scrub bar (accent color tint)
- **Insert:** drops marked range at timeline playhead, shifts downstream clips right
- **Overwrite:** drops marked range at timeline playhead, overwrites what's there
- No marks set → uses full clip duration

### 1.3 Program Monitor

- Plays the timeline from the current playhead position
- **Virtual playback:** single `<video>` element. On each clip start, set `src` = that clip's signed proxy URL and `currentTime` = `sourceIn`. When `currentTime` reaches `sourceOut`, load the next clip. Brief load gap at clip boundaries is acceptable for v1.
- Timecode display: `HH:MM:SS;FF` at **59.94 fps** (drop-frame notation, semicolon separator)
- Transport shortcuts: `Space` (play/pause), `J` (reverse), `K` (stop), `L` (forward), `←`/`→` (± 1 frame at 59.94), `Home` (jump to start)

### 1.4 Timeline

**Ruler:** Timecode-marked horizontal ruler. Default scale: 100px/s. Zoom: `Ctrl + scroll wheel` or zoom slider (range: 20px/s → 500px/s).

**Tracks:** V1, V2 (video), A1, A2 (audio). Each track row: 48px tall. Track header (40px wide, left): track label, lock toggle.

**Clips:** Absolutely positioned `<div>` elements within track rows.
- `left` = `timelineIn_seconds × scale`
- `width` = `(timelineOut - timelineIn)_seconds × scale`
- Shows: clip name (truncated), source TC range
- If width > 120px: asset thumbnail centered in clip body

**Playhead:** Red vertical line spanning all tracks. Draggable. Clicking the ruler jumps playhead.

**Tools:**

| Tool | Key | Behavior |
|------|-----|----------|
| Select | `V` | Click to select (accent border). Drag body to move horizontally. Drag left/right edge to trim source in/out. |
| Razor | `C` | Click on a clip → splits into two clips at that exact frame. |
| Hand | `H` | Click-drag to pan timeline horizontally. |

**Editing:**
- `Delete` / `Backspace`: remove selected clip(s)
- `Ctrl+Z` / `Ctrl+Shift+Z`: undo / redo (local history stack, max 50 steps)
- Clips cannot overlap within the same track (enforced on move/razor)

**Auto-save:** Debounced 2s after any timeline change. Visual indicator: subtle "Saving…" / "Saved" text in the timeline toolbar.

### 1.5 Data Model

**New migration: `schema_patch_editor.sql`**

```sql
CREATE TABLE sequences (
  id          UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  project_id  UUID NOT NULL REFERENCES projects ON DELETE CASCADE,
  name        TEXT NOT NULL DEFAULT 'Sequence 1',
  frame_rate  NUMERIC(6,3) NOT NULL DEFAULT 59.94,
  width       INTEGER NOT NULL DEFAULT 1920,
  height      INTEGER NOT NULL DEFAULT 1080,
  created_at  TIMESTAMPTZ DEFAULT NOW(),
  updated_at  TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_sequences_project_id ON sequences(project_id);

CREATE TABLE sequence_clips (
  id                   UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  sequence_id          UUID NOT NULL REFERENCES sequences ON DELETE CASCADE,
  asset_id             UUID NOT NULL REFERENCES assets ON DELETE CASCADE,
  track                INTEGER NOT NULL DEFAULT 0,
  -- track encoding: 0=V1, 1=V2, 100=A1, 101=A2
  timeline_in_frames   BIGINT NOT NULL,
  timeline_out_frames  BIGINT NOT NULL,
  source_in_frames     BIGINT NOT NULL DEFAULT 0,
  source_out_frames    BIGINT NOT NULL,
  created_at           TIMESTAMPTZ DEFAULT NOW(),
  updated_at           TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_sequence_clips_sequence_id ON sequence_clips(sequence_id);
```

**Frame math:** All frame counts use **59.94 fps** (= 60000/1001). Timecode display uses SMPTE drop-frame (`;` separator). Conversion helpers:
- `framesToSeconds(f)` = `f / 59.94`
- `secondsToFrames(s)` = `Math.round(s * 59.94)`
- Drop-frame TC calculation uses standard SMPTE DF algorithm

### 1.6 API Routes

New file: `services/mam-api/src/routes/sequences.js`

| Method | Path | Body / Params | Notes |
|--------|------|--------------|-------|
| GET | `/api/v1/sequences` | `?project_id=` | List sequences, ordered by `updated_at DESC` |
| POST | `/api/v1/sequences` | `{ project_id, name, frame_rate?, width?, height? }` | Create |
| GET | `/api/v1/sequences/:id` | — | Sequence + all clips joined with asset (`display_name`, `fps`, `duration_ms`, `proxy_s3_key`, `thumbnail_s3_key`) |
| PUT | `/api/v1/sequences/:id` | `{ name?, frame_rate?, width?, height? }` | Update metadata |
| DELETE | `/api/v1/sequences/:id` | — | Cascade deletes clips |
| PUT | `/api/v1/sequences/:id/clips` | `[ { asset_id, track, timeline_in_frames, timeline_out_frames, source_in_frames, source_out_frames } ]` | **Full replace** — deletes existing clips, inserts new array in one transaction |
| POST | `/api/v1/sequences/:id/export/edl` | — | Returns CMX3600 EDL as `text/plain; charset=utf-8`, `Content-Disposition: attachment` |

### 1.7 EDL Export

`generateEDL(sequenceName, clips, fps)` function produces CMX3600. Timecode math reuses the logic from `worker/src/edl/parser.js` (copied into a shared util or duplicated in the API route).

```
TITLE: My Sequence

001  clip_filename  V  C  00:00:00;00 00:00:05;00 00:00:00;00 00:00:05;00
002  other_clip     V  C  00:00:02;00 00:00:08;00 00:00:05;00 00:00:11;00
```

Conforms via the existing `conform.js` BullMQ worker unchanged.

### 1.8 `api.js` Additions

```js
getSequences(projectId)
createSequence(data)
getSequence(sequenceId)
updateSequence(sequenceId, data)
deleteSequence(sequenceId)
syncSequenceClips(sequenceId, clipsArray)
exportSequenceEDL(sequenceId)   // triggers file download
```

---

## 2. Growing File Workflow

### 2.1 Problem

During SDI capture, proxy is piped to S3 via multipart upload — invisible until `CompleteMultipartUpload`. No live preview is possible. For SRT/RTMP, no proxy exists at all until the BullMQ worker runs post-stop.

### 2.2 Solution: HLS Segments During Capture

Write proxy as HLS segments to local disk during recording. Serve them live from the capture service. On stop: stitch segments → single MP4 → upload to S3 as proxy.

**Why HLS:** Growing playlists are the browser-native live video mechanism. Accumulated segments give free rewind. `hls.js` is a small CDN-loadable polyfill for Chrome/Firefox. FFmpeg's `hls` muxer is proven in production.

### 2.3 FFmpeg Args Change

**SDI (second FFmpeg process — proxy process):**
```bash
# Before: fragmented MP4 → pipe → S3
ffmpeg [decklink] -c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
  -movflags +frag_keyframe+empty_moov -f mp4 pipe:1

# After: HLS segments → local dir
ffmpeg [decklink] -c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
  -hls_time 2 -hls_list_size 0 -hls_flags append_list \
  -hls_segment_filename '$HLS_DIR/<sessionId>/seg%05d.ts' \
  $HLS_DIR/<sessionId>/index.m3u8
```

**SRT/RTMP (single FFmpeg process — two output pads):**
```bash
ffmpeg [srt/rtmp input] \
  -c:v prores_ks -profile:v 3 -c:a pcm_s24le \
  -movflags +frag_keyframe+empty_moov -f mov pipe:1 \
  -c:v libx264 -preset fast -b:v 10M -c:a aac -b:a 192k \
  -hls_time 2 -hls_list_size 0 -hls_flags append_list \
  -hls_segment_filename '$HLS_DIR/<sessionId>/seg%05d.ts' \
  $HLS_DIR/<sessionId>/index.m3u8
```

### 2.4 Capture Service Changes

**`capture-manager.js`:**
- New env var: `HLS_SESSION_DIR` (default: `/tmp/wd-hls`). 3 TB disk — no constraint.
- `start()`: `mkdir -p $HLS_SESSION_DIR/<sessionId>/`, spawn FFmpeg with HLS output. Add `liveUrl: /capture/live/<sessionId>/index.m3u8` to session state.
- `stop()`:
  1. `SIGINT` FFmpeg (existing)
  2. Wait for process exit (FFmpeg writes final segment + closes playlist before exiting)
  3. `ffmpeg -i $HLS_DIR/<sessionId>/index.m3u8 -c copy <tmp>.mp4` (stitch)
  4. Upload stitched MP4 to S3 as proxy key (existing `uploadToS3` helper)
  5. `rm -rf $HLS_DIR/<sessionId>/`
- Startup: scan `$HLS_SESSION_DIR/` and delete dirs older than 24h

**New capture routes:**
```
GET /capture/live/:sessionId/index.m3u8  → serve HLS playlist file
GET /capture/live/:sessionId/:file       → serve .ts segment files
```
Both routes: check session exists (active or recently stopped with dir still present), stream file from disk with correct `Content-Type` (`application/vnd.apple.mpegurl` / `video/mp2t`).

**Updated `/capture/status` response:**
```json
{
  "recording": true,
  "sessionId": "abc123",
  "liveUrl": "/capture/live/abc123/index.m3u8",
  "startedAt": "...",
  "duration": 42,
  ...
}
```

**nginx:** Add location block for `/capture/live/` → proxy_pass to capture service (same pattern as existing `/capture/` block).

### 2.5 Live Preview in `capture.html`

When status poll returns `recording: true` with `liveUrl`:
1. Show collapsible **Live Preview** panel beneath capture controls
2. Load hls.js from CDN: `cdnjs.cloudflare.com/ajax/libs/hls.js/1.5.15/hls.min.js`
3. `new Hls()` → `hls.loadSource(status.liveUrl)` → `hls.attachMedia(videoEl)`
4. `videoEl.play()` on `MANIFEST_PARSED` (plays at live edge)
5. **⏮ Rewind** button: `videoEl.currentTime = 0`
6. Elapsed time counter from `status.startedAt`

On recording stop: `hls.destroy()`, hide preview panel. Asset appears in library after proxy upload + thumbnail job complete.

---

## 3. Feature Additions (Prioritized)

### P1 — Improved Player (`player.html` rebuild)
- Migrate from old CSS variables (`--color-bg-tertiary`) to current design tokens (`--bg-panel`, etc.)
- Replace browser default controls with custom transport bar: scrub slider, timecode display `HH:MM:SS;FF` at 59.94 fps, frame-step buttons, J/K/L shortcuts
- Inline rename: click `display_name` to edit in place → auto-save
- "Open in Editor" button → `editor.html?asset=<id>`

### P2 — Subclips
- Player shows In/Out markers → "Create Subclip" button
- `POST /api/v1/assets` with `{ parent_asset_id, subclip_in_ms, subclip_out_ms, display_name }`
- New columns: `parent_asset_id UUID REFERENCES assets`, `subclip_in_ms BIGINT`, `subclip_out_ms BIGINT`
- Library shows subclip cards with ✂ badge; player pre-seeks to `subclip_in_ms`
- Subclips use parent's proxy S3 key — no re-transcode

### P3 — Multi-select & Bulk Ops
- Shift-click or checkbox (visible on hover) to select multiple asset cards
- Floating action bar: **Move to bin | Add tags | Delete**
- Move to bin: slide panel with bin tree, `PATCH /assets/:id { bin_id }` for each
- Add tags: appends to all selected assets
- Bulk delete: confirm modal → soft-delete

### P4 — Waveform Display in Editor
- In proxy worker (`proxy.js`): after transcode, run FFmpeg `astats` filter to generate per-second peak arrays → store as `waveforms/<assetId>.json` in S3
- New `waveform_s3_key TEXT` column on `assets`
- Editor timeline: fetch waveform JSON for audio track clips, render as SVG polyline within clip block

### P5 — Timecoded Comments
```sql
CREATE TABLE asset_comments (
  id               UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  asset_id         UUID NOT NULL REFERENCES assets ON DELETE CASCADE,
  user_id          UUID NOT NULL REFERENCES users,
  timecode_seconds NUMERIC(10,3),
  body             TEXT NOT NULL,
  created_at       TIMESTAMPTZ DEFAULT NOW()
);
```
Player sidebar: "Comments" tab with "Post at current TC" button. Clickable entries seek video to that timecode.

### P6 — Smart Bins
```sql
ALTER TABLE bins ADD COLUMN is_smart BOOLEAN DEFAULT false;
ALTER TABLE bins ADD COLUMN smart_query JSONB;
-- smart_query shape: { "tags": ["interview"], "media_type": "video", "created_after": "2026-01-01" }
```
Smart bin assets: dynamically queried from `assets` table. Shows ✦ icon in bin tree.

### P7 — Metadata Templates
```sql
CREATE TABLE project_metadata_fields (
  id          UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  project_id  UUID NOT NULL REFERENCES projects ON DELETE CASCADE,
  field_key   TEXT NOT NULL,
  label       TEXT NOT NULL,
  field_type  TEXT NOT NULL DEFAULT 'text',
  options     JSONB,
  required    BOOLEAN DEFAULT false,
  sort_order  INTEGER DEFAULT 0
);
ALTER TABLE assets ADD COLUMN metadata JSONB DEFAULT '{}';
```

---

## Implementation Sequencing

### Phase 1 — Editor
1. DB migration (`schema_patch_editor.sql`)
2. API (`sequences.js` route — CRUD + clip sync + EDL export)
3. `api.js` — sequence helpers
4. `editor.html` — 4-panel shell + CSS (sidebar nav update)
5. Timeline engine — ruler, playhead, track rows, clip rendering
6. Select tool — click-select, drag-move, drag-edge trim
7. Razor tool — click-to-split
8. Source monitor — video + transport + in/out marking + Insert/Overwrite
9. Program monitor — virtual playback + 59.94 drop-frame timecode
10. Auto-save (debounced 2s) + sequence picker in media panel
11. Library "Open in Editor" action + sidebar link

### Phase 2 — Growing File
1. Capture service: HLS output, `$HLS_SESSION_DIR` env var
2. Capture service: `/capture/live/:sessionId/` routes + nginx config
3. Capture service: on-stop stitch + S3 upload + cleanup
4. `capture.html`: live preview panel (hls.js), rewind button

### Phase 3 — Feature Additions
P1 → P2 → P3 → P4 → P5 → P6 → P7