docs: add NLE editor React polish plan (phases 1-3)
This commit is contained in:
parent
f21157f3c7
commit
7189df7957
1 changed files with 797 additions and 0 deletions
|
|
@ -0,0 +1,797 @@
|
|||
# NLE Editor: React SPA Polish — Phases 1–3 Implementation Plan
|
||||
|
||||
> **Date:** 2026-05-24
|
||||
> **Status:** Draft — awaiting user review before execution
|
||||
> **Reference spec:** `docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md`
|
||||
> **Architecture note:** The original NLE editor plan (`2026-05-18-nle-editor.md`) was written for standalone HTML pages. Commit `6322b61` deleted those pages — the React SPA is now the only entry. This plan adapts Phase 1 for the React SPA architecture. Phases 2–3 follow the original design spec.
|
||||
|
||||
---
|
||||
|
||||
## Architectural Context
|
||||
|
||||
**Backend (complete, no changes needed):**
|
||||
- `003-editor-sequences.sql` migration applied — `sequences` + `sequence_clips` tables live
|
||||
- `routes/sequences.js` — full CRUD, clip sync (`PUT /:id/clips`), EDL export — registered in `index.js`
|
||||
|
||||
**Frontend (React SPA):**
|
||||
- `screens-editor.jsx` — skeleton with all features disabled behind "In Development" overlay
|
||||
- `shell.jsx` — Editor nav item with `DEV` badge
|
||||
- `app.jsx` — `case 'editor': content = <Editor />`
|
||||
- `data.jsx` — **no** sequence API helpers exist
|
||||
- `js/timecode.js` — 59.94 DF timecode (vanilla JS, unused by SPA)
|
||||
- `js/timeline.js` — DOM timeline engine (vanilla JS, unused by SPA)
|
||||
|
||||
**Other services:**
|
||||
- `services/editor/` — OpenReel Video (separate WebCodecs/WebGPU NLE, proxied at `/editor/`). Not part of this plan's scope — the React SPA editor is the primary in-app NLE.
|
||||
- `services/premiere-plugin/` — Premiere Pro CEP panel (external integration, not in scope)
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
### Phase 1 — Core Editor
|
||||
|
||||
| Action | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| MODIFY | `services/web-ui/public/data.jsx` | Add `getSequences`, `createSequence`, `getSequence`, `updateSequence`, `deleteSequence`, `syncSequenceClips`, `exportSequenceEDL` |
|
||||
| MODIFY | `services/web-ui/public/screens-editor.jsx` | Full rewrite: source monitor, media panel, timeline, program monitor, sequence management, auto-save, undo/redo, EDL export |
|
||||
| MODIFY | `services/web-ui/public/shell.jsx` | Remove `DEV` badge from Editor nav item |
|
||||
| NO CHANGE | `services/web-ui/public/js/timecode.js` | Imported by screens-editor.jsx via `<script>` tag or inlined |
|
||||
| NO CHANGE | `services/web-ui/public/js/timeline.js` | Same as above |
|
||||
|
||||
### Phase 2 — UX & Growing File
|
||||
|
||||
| Action | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| MODIFY | `services/web-ui/public/screens-editor.jsx` | Multi-track refinements, zoom slider, JKL transport, waveform display, inspector wiring |
|
||||
| MODIFY | `services/web-ui/public/screens-editor.jsx` | Style migration from legacy CSS variables to `/dist/app.css` Tailwind primitives |
|
||||
| MODIFY | `services/capture/src/capture-manager.js` | HLS segment output during recording (`hls_time 2`, `hls_list_size 0`) |
|
||||
| CREATE | `services/capture/src/routes/live.js` | Serve HLS playlists + segments from `$HLS_SESSION_DIR/<sessionId>/` |
|
||||
| MODIFY | `services/capture/src/index.js` | Register live routes |
|
||||
| MODIFY | `services/web-ui/nginx.conf` | Proxy `/capture/live/` → capture service |
|
||||
| MODIFY | `services/web-ui/public/screens-ingest.jsx` | Add hls.js live preview panel to capture/recorder screens |
|
||||
|
||||
### Phase 3 — Export, Conform & Features
|
||||
|
||||
| Action | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| MODIFY | `services/mam-api/src/routes/sequences.js` | Add FCP XML export endpoint |
|
||||
| MODIFY | `services/web-ui/public/screens-editor.jsx` | Conform integration, hi-res relink from timeline |
|
||||
| MODIFY | `services/web-ui/public/screens-asset.jsx` | Player rebuild: custom transport, 59.94 TC, JKL, "Open in Editor" |
|
||||
| MODIFY | `services/web-ui/public/screens-asset.jsx` | Subclip creation from player in/out markers |
|
||||
| MODIFY | `services/web-ui/public/screens-library.jsx` | Multi-select (shift-click/checkbox), floating action bar |
|
||||
| MODIFY | `services/mam-api/src/routes/bins.js` | Smart bins: `is_smart`, `smart_query` columns, dynamic query |
|
||||
| MODIFY | `services/web-ui/public/screens-library.jsx` | Smart bin UI (icon, dynamic content) |
|
||||
| MODIFY | `services/mam-api/src/routes/projects.js` | Metadata templates CRUD |
|
||||
| MODIFY | `services/mam-api/src/db/migrations/` | Migration(s) for new columns/tables |
|
||||
| MODIFY | `services/web-ui/public/screens-asset.jsx` | Metadata form in asset detail |
|
||||
| MODIFY | `services/web-ui/public/screens-library.jsx` | Bulk operations UI |
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Tasks
|
||||
|
||||
### Task 1.1: Add Sequence API Helpers to data.jsx
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/data.jsx`
|
||||
|
||||
Add the following functions after the existing `async function loadData()` block (before the final `window.ZAMPP_API` export):
|
||||
|
||||
```js
|
||||
// ── Sequence API ────────────────────────────────────────────────
|
||||
|
||||
async function getSequences(projectId) {
|
||||
return apiFetch('/sequences?project_id=' + projectId);
|
||||
}
|
||||
|
||||
async function createSequence(data) {
|
||||
return apiFetch('/sequences', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify(data),
|
||||
});
|
||||
}
|
||||
|
||||
async function getSequence(sequenceId) {
|
||||
return apiFetch('/sequences/' + sequenceId);
|
||||
}
|
||||
|
||||
async function updateSequence(sequenceId, data) {
|
||||
return apiFetch('/sequences/' + sequenceId, {
|
||||
method: 'PUT',
|
||||
body: JSON.stringify(data),
|
||||
});
|
||||
}
|
||||
|
||||
async function deleteSequence(sequenceId) {
|
||||
return apiFetch('/sequences/' + sequenceId, { method: 'DELETE' });
|
||||
}
|
||||
|
||||
async function syncSequenceClips(sequenceId, clips) {
|
||||
return apiFetch('/sequences/' + sequenceId + '/clips', {
|
||||
method: 'PUT',
|
||||
body: JSON.stringify(clips),
|
||||
});
|
||||
}
|
||||
|
||||
async function exportSequenceEDL(sequenceId) {
|
||||
const res = await fetch(API + '/sequences/' + sequenceId + '/export/edl', {
|
||||
method: 'POST',
|
||||
credentials: 'include',
|
||||
});
|
||||
if (!res.ok) throw new Error('EDL export failed: ' + res.status);
|
||||
const blob = await res.blob();
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = 'sequence.edl';
|
||||
a.click();
|
||||
URL.revokeObjectURL(url);
|
||||
}
|
||||
```
|
||||
|
||||
Append to the `window.ZAMPP_API` export:
|
||||
```js
|
||||
window.ZAMPP_API = {
|
||||
fetch: apiFetch, loadData, fmtDuration, fmtSize, fmtRelative,
|
||||
getSequences, createSequence, getSequence, updateSequence,
|
||||
deleteSequence, syncSequenceClips, exportSequenceEDL,
|
||||
};
|
||||
```
|
||||
|
||||
**Verify:**
|
||||
```js
|
||||
// In browser console after rebuild:
|
||||
ZAMPP_API.getSequences('<project-id>').then(r => console.log(r))
|
||||
// Expected: array of sequences (or empty array)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 1.2: Timecode Utility for React
|
||||
|
||||
**Files:** No new file — add to `screens-editor.jsx` or use existing `public/js/timecode.js`.
|
||||
|
||||
The existing `js/timecode.js` exports `window.TC` with `{ framesToTC, tcToFrames, secondsToFrames, framesToSeconds, FPS }`. In the React SPA, we either:
|
||||
|
||||
- **Option A** (preferred): Load via existing `<script>` tag in `index.html` (if not already), access via `window.TC`
|
||||
- **Option B**: Inline the pure functions in `screens-editor.jsx`
|
||||
|
||||
Check `index.html` to see if `timecode.js` is already loaded. If not, add:
|
||||
```html
|
||||
<script src="js/timecode.js"></script>
|
||||
```
|
||||
|
||||
For the React component, create a thin wrapper:
|
||||
```js
|
||||
// Inside screens-editor.jsx, before the Editor component:
|
||||
function useTimecode() {
|
||||
// Ensure TC is loaded
|
||||
if (typeof TC === 'undefined') {
|
||||
console.warn('timecode.js not loaded — timecode functions unavailable');
|
||||
}
|
||||
return {
|
||||
framesToTC: (f) => TC ? TC.framesToTC(f) : String(f),
|
||||
tcToFrames: (tc) => TC ? TC.tcToFrames(tc) : 0,
|
||||
secondsToFrames: (s) => TC ? TC.secondsToFrames(s) : Math.round(s * 60),
|
||||
framesToSeconds: (f) => TC ? TC.framesToSeconds(f) : f / 60,
|
||||
FPS: TC ? TC.FPS : 59.94,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**Verify:** In browser console — `TC.framesToTC(3596)` → `"00:01:00;00"`
|
||||
|
||||
---
|
||||
|
||||
### Task 1.3: React Timeline Component
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx` (add components inline, same file pattern as other screens)
|
||||
|
||||
Build a `TimelinePanel` React component that wraps the existing `timeline.js` engine:
|
||||
|
||||
```js
|
||||
function TimelinePanel({ clips, onClipsChanged, playheadFrames, onPlayheadMoved, fps }) {
|
||||
const containerRef = React.useRef(null);
|
||||
const engineRef = React.useRef(null);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (!containerRef.current || engineRef.current) return;
|
||||
engineRef.current = window.Timeline;
|
||||
window.Timeline.init(containerRef.current, {
|
||||
fps: fps || 59.94,
|
||||
scale: 100,
|
||||
onClipsChanged: onClipsChanged,
|
||||
onPlayheadMoved: onPlayheadMoved,
|
||||
});
|
||||
return () => { /* cleanup */ };
|
||||
}, []);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (engineRef.current && clips) {
|
||||
engineRef.current.render(clips, { fps: fps || 59.94 });
|
||||
}
|
||||
}, [clips, fps]);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (engineRef.current) {
|
||||
engineRef.current.setPlayhead(playheadFrames || 0);
|
||||
}
|
||||
}, [playheadFrames]);
|
||||
|
||||
return (
|
||||
<div className="timeline-panel">
|
||||
<div className="timeline-toolbar">
|
||||
{/* Tool buttons: Select(V), Razor(C), Hand(H) */}
|
||||
{/* Undo/Redo */}
|
||||
{/* Save status indicator */}
|
||||
</div>
|
||||
<div className="timeline-container" ref={containerRef} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
The component mounts the timeline once, then pushes clip/playhead updates via `render()` and `setPlayhead()`. No rebuild needed on every frame.
|
||||
|
||||
**Tool toolbar state:**
|
||||
- Three tool buttons (V/C/H) that call `Timeline.setTool(name)`
|
||||
- Active tool highlighted — sync with keyboard shortcuts
|
||||
- Undo/redo buttons call the editor's history stack (not timeline.js's — timeline.js doesn't have an undo stack, the editor page in the original plan managed it in the app state)
|
||||
|
||||
**Verify:** Timeline renders 4 track rows (V1, V2, A1, A2) with ruler, playhead at frame 0, no clips. Tool buttons switch cursor mode.
|
||||
|
||||
---
|
||||
|
||||
### Task 1.4: Rewrite screens-editor.jsx
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
Full rewrite preserving the `Editor` component on `window.Editor`. The new structure:
|
||||
|
||||
```
|
||||
┌─ SOURCE MONITOR ───────┬─ PROGRAM MONITOR ───────┐
|
||||
│ [video preview] │ [video preview] │
|
||||
│ TC: 00:00:00;00 │ TC: 00:00:00;00 │
|
||||
│ [=====scrub=========] │ [=====scrub=========] │
|
||||
│ [In] [Out] [Insert] │ [▶] [⏮] [⏭] │
|
||||
├─ MEDIA PANEL ──────────┴─ TIMELINE ─────────────┤
|
||||
│ [sequence picker ▼] [+]│ [V] [C] [H] | [↩] [↪] │
|
||||
│ [filter by bin ▼] │ ruler: 00:00 — 00:05 … │
|
||||
│ [asset list] │ V1 ░░░░[clip]░░░░ │
|
||||
│ │ V2 │
|
||||
│ │ A1 ░░░░[clip]░░░░ │
|
||||
│ │ A2 │
|
||||
└────────────────────────┴─────────────────────────┘
|
||||
```
|
||||
|
||||
**App state (React hooks):**
|
||||
```js
|
||||
const [projectId, setProjectId] = React.useState(null);
|
||||
const [sequences, setSequences] = React.useState([]);
|
||||
const [currentSeq, setCurrentSeq] = React.useState(null); // { id, name, clips: [...] }
|
||||
const [assets, setAssets] = React.useState([]); // from ZAMPP_DATA.ASSETS
|
||||
const [sourceAsset, setSourceAsset] = React.useState(null);
|
||||
const [srcIn, setSrcIn] = React.useState(null);
|
||||
const [srcOut, setSrcOut] = React.useState(null);
|
||||
const [playing, setPlaying] = React.useState(false);
|
||||
const [playheadFrames, setPlayheadFrames] = React.useState(0);
|
||||
const [saveStatus, setSaveStatus] = React.useState('');
|
||||
const [history, setHistory] = React.useState([[]]);
|
||||
const [historyIdx, setHistoryIdx] = React.useState(0);
|
||||
```
|
||||
|
||||
**Source monitor:**
|
||||
- `<video>` element with `src` from `apiFetch('/assets/' + assetId + '/stream')` — matches the pattern in `screens-asset.jsx` lines 53-54
|
||||
- Scrub slider, current-time TC display (59.94 DF via `TC.framesToTC`)
|
||||
- Mark In (`I` key or button), Mark Out (`O` key or button)
|
||||
- In/out range highlighted (stored as seconds in state)
|
||||
- Insert button: calls `Timeline.addClip(asset, srcIn, srcOut, 0)` then triggers auto-save
|
||||
- Overwrite button: same but shifts/overwrites existing clips at playhead
|
||||
|
||||
**Media panel:**
|
||||
- Left column in the bottom row
|
||||
- Sequence picker (`<select>` from `getSequences(projectId)`) + "New sequence" (+) button
|
||||
- Bin filter (`<select>` from `ZAMPP_DATA.BINS`)
|
||||
- Asset list: filtered by project, each showing thumbnail (`AssetThumb` component from `visuals.jsx`) + name + duration
|
||||
- Double-click loads asset into source monitor
|
||||
|
||||
**Program monitor:**
|
||||
- Clip-by-clip virtual playback (same algorithm as the original editor.html lines 1746-1864)
|
||||
- On play: gather V1 clips sorted by `timeline_in_frames`, find clip at current playhead, load its signed stream URL, `currentTime = source_in_seconds`
|
||||
- On `timeupdate` event: check if `currentTime >= source_out_seconds` → advance to next clip
|
||||
- Timecode display synced to timeline playhead
|
||||
- Transport: Play/Pause (Space), Stop (K), Forward (L), Reverse (J — v1 can just stop)
|
||||
- `<video>` element with preload=auto
|
||||
|
||||
**Auto-save:**
|
||||
- `markDirty()` → clears previous timer, sets `setTimeout(saveSequence, 2000)`
|
||||
- `saveSequence()` → `ZAMPP_API.syncSequenceClips(seqId, clips)` → show "Saved" / "Save failed"
|
||||
- On mount: load project's sequences, open most-recent (or first)
|
||||
|
||||
**Undo/redo:**
|
||||
- `onClipsChanged` callback from Timeline → push clone of clips onto history stack, cap at 50
|
||||
- `undo()` → pop history, render
|
||||
- `redo()` → advance history index, render
|
||||
- Ctrl+Z / Ctrl+Shift+Z
|
||||
|
||||
**EDL export:**
|
||||
- Button in top toolbar → calls `exportSequenceEDL(currentSeq.id)` → triggers file download
|
||||
|
||||
**Remove "In Development" overlay:**
|
||||
- Delete the `<div style={{ position: 'absolute', inset: 0, backdropFilter: 'blur(6px)' ... }}>` block (lines 98-117)
|
||||
- Delete the `FauxFrame` import/reference
|
||||
- Replace `FauxFrame` with real `<video>` element in the canvas area
|
||||
- Enable all previously disabled buttons
|
||||
|
||||
**Keyboard shortcuts:**
|
||||
- `I` / `O` — mark in/out in source monitor
|
||||
- `V` / `C` / `H` — select/razor/hand tool
|
||||
- `Space` — play/pause program monitor
|
||||
- `J` / `K` / `L` — reverse/stop/forward
|
||||
- `Delete` / `Backspace` — remove selected clip
|
||||
- `Ctrl+Z` / `Ctrl+Shift+Z` — undo/redo
|
||||
- `Ctrl+S` — save
|
||||
- `←` / `→` — frame step (±1 frame at 59.94)
|
||||
|
||||
**Verify:**
|
||||
- Navigation to Editor route renders the 4-panel layout
|
||||
- Sequence picker shows existing sequences from API (or "No sequences")
|
||||
- Double-click asset loads it into source monitor with video playback
|
||||
- Mark In/Out and click Insert places a clip on V1
|
||||
- Clip appears in timeline, auto-save fires within 2s
|
||||
- Program monitor plays clip back
|
||||
- Ctrl+Z undoes the insert
|
||||
- EDL export downloads a `.edl` file
|
||||
- API check: `GET /api/v1/sequences/:id` returns `{ clips: [...] }`
|
||||
|
||||
---
|
||||
|
||||
### Task 1.5: Update Nav Badge
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/shell.jsx`
|
||||
|
||||
Change:
|
||||
```js
|
||||
{ id: "editor", label: "Editor", icon: "editor", badge: { kind: "dev", text: "DEV" } }
|
||||
```
|
||||
To:
|
||||
```js
|
||||
{ id: "editor", label: "Editor", icon: "editor" }
|
||||
```
|
||||
|
||||
**Verify:** Sidebar shows "Editor" without the amber DEV pill badge.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Tasks
|
||||
|
||||
### Task 2.1: Multi-track & Timeline Refinements
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
**Ripple delete:** Add a "ripple" mode / toggle. When enabled, deleting a clip shifts all clips to its right left by the deleted clip's duration. Implemented via `onClipsChanged` callback:
|
||||
```js
|
||||
function deleteWithRipple(clipId) {
|
||||
const clip = clips.find(c => c._id === clipId);
|
||||
if (!clip) return;
|
||||
const duration = clip.timeline_out_frames - clip.timeline_in_frames;
|
||||
// Remove clip
|
||||
let newClips = clips.filter(c => c._id !== clipId);
|
||||
// Shift clips to the right of the gap
|
||||
newClips = newClips.map(c => {
|
||||
if (c.timeline_in_frames >= clip.timeline_in_frames) {
|
||||
return { ...c,
|
||||
timeline_in_frames: c.timeline_in_frames - duration,
|
||||
timeline_out_frames: c.timeline_out_frames - duration,
|
||||
};
|
||||
}
|
||||
return c;
|
||||
});
|
||||
// Update state
|
||||
}
|
||||
```
|
||||
|
||||
**Snap-to-playhead:** When dragging a clip, snap its left edge to the current playhead position if within 5 frames. Implemented in the `_onMoveStart` handler or a post-process in `onClipsChanged`.
|
||||
|
||||
**Track locking:** Add lock toggle to track headers (V1, V2, A1, A2). Locked tracks ignore clip drops, moves, and deletes.
|
||||
|
||||
**Clip overlap prevention:** When moving a clip, clamp `timeline_in_frames` so clips don't overlap on the same track. Trim duration if needed.
|
||||
|
||||
**Thumbnails in clips:** When a clip is wider than 120px, fetch the asset's thumbnail and render it as the clip body background (low opacity, stretched to fit).
|
||||
|
||||
**Verify:** Dragging clips snaps at playhead, ripple delete shifts downstream clips, locked tracks show visual indicator and reject edits.
|
||||
|
||||
---
|
||||
|
||||
### Task 2.2: Zoom Slider & Improved Ruler
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
Add a horizontal zoom slider in the timeline toolbar (range: 20–500 px/s):
|
||||
```jsx
|
||||
<input type="range" min="20" max="500" value={scale}
|
||||
onChange={e => {
|
||||
setScale(Number(e.target.value));
|
||||
Timeline.setScale(Number(e.target.value));
|
||||
}} />
|
||||
```
|
||||
|
||||
The ruler needs to adapt tick intervals based on zoom level:
|
||||
- `< 30 px/s`: tick every 10s, label every 30s
|
||||
- `30–60 px/s`: tick every 5s, label every 15s
|
||||
- `60–120 px/s`: tick every 2s, label every 10s
|
||||
- `> 120 px/s`: tick every 1s, label every 5s
|
||||
|
||||
The existing `timeline.js` `_renderRuler` function already handles this (lines 779). The React wrapper just needs to expose `setScale`.
|
||||
|
||||
**Verify:** Slider changes scale, ruler ticks and labels update accordingly.
|
||||
|
||||
---
|
||||
|
||||
### Task 2.3: JKL Transport & Frame Stepping
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
Full jog/shuttle keyboard:
|
||||
- `J` — play reverse at 1x (press twice for 2x, three times for 4x)
|
||||
- `K` — stop/pause
|
||||
- `L` — play forward at 1x (press twice for 2x, three times for 4x)
|
||||
- `←` — step back 1 frame
|
||||
- `→` — step forward 1 frame
|
||||
- `Shift+←` — step back 1 second (59.94 frames)
|
||||
- `Shift+→` — step forward 1 second
|
||||
- `Home` — jump to timeline start
|
||||
- `End` — jump to timeline end
|
||||
|
||||
**Program monitor bar buttons:**
|
||||
```jsx
|
||||
<div className="monitor-bar">
|
||||
<button onClick={stepBack} title="Step back (←)">⏪</button>
|
||||
<button onClick={stepBack1Sec} title="Step back 1s">◀</button>
|
||||
<button onClick={togglePlay} title="Play (Space)">{playing ? '⏸' : '▶'}</button>
|
||||
<button onClick={stepForward1Sec} title="Step forward 1s">▶</button>
|
||||
<button onClick={stepForward} title="Step forward (→)">⏩</button>
|
||||
<span className="monitor-tc">{TC.framesToTC(playheadFrames)}</span>
|
||||
<input type="range" className="monitor-scrub" ... />
|
||||
</div>
|
||||
```
|
||||
|
||||
**Verify:** JKL play/stop/forward works, arrow keys step precisely 1 frame at 59.94.
|
||||
|
||||
---
|
||||
|
||||
### Task 2.4: Waveform Display in Audio Tracks
|
||||
|
||||
This requires both backend and frontend changes.
|
||||
|
||||
**Backend (proxy worker):**
|
||||
MODIFY `services/worker/src/workers/proxy.js` — after transcode, run:
|
||||
```bash
|
||||
ffmpeg -i <proxied-file> -filter_complex "aformat=channel_layouts=mono,astats=metadata=1:reset=1" -f null -
|
||||
```
|
||||
Parse the `stderr` for `pts_time` and `Bit_depth` to extract per-second peak arrays. Store as `waveforms/<assetId>.json` in S3.
|
||||
|
||||
**New column:** `waveform_s3_key TEXT` on `assets` table (migration 013).
|
||||
|
||||
**Frontend:**
|
||||
MODIFY `screens-editor.jsx` — for audio track clips (A1, A2), fetch waveform JSON and render SVG polyline within the clip block:
|
||||
```jsx
|
||||
// Inside clip rendering for audio tracks:
|
||||
{track >= 100 && waveform && (
|
||||
<svg className="clip-waveform" viewBox={`0 0 ${width} 48`} preserveAspectRatio="none">
|
||||
<polyline points={waveformToPoints(waveform, width, 48)} fill="none" stroke="var(--signal-good)" strokeWidth="1" />
|
||||
</svg>
|
||||
)}
|
||||
```
|
||||
|
||||
**Verify:** Audio clips on A1/A2 show waveform polylines. Waveform data is fetched from S3 (or gracefully falls back to empty).
|
||||
|
||||
---
|
||||
|
||||
### Task 2.5: Inspector Panel Wiring
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
Replace inspector placeholder values with real data from the selected clip:
|
||||
- **Transform:** Position (always 0,0 for v1 — no keyframes yet), scale (always 100%), rotation (always 0°)
|
||||
- **Color:** Exposure/contrast/saturation (all 0.0 — placeholder for future)
|
||||
- **Audio:** Level from asset metadata if available, or 0.0 dB
|
||||
|
||||
When no clip is selected, show a "Select a clip to inspect" message in the inspector panel.
|
||||
|
||||
**Verify:** Clicking a timeline clip shows its display_name and basic properties in the inspector.
|
||||
|
||||
---
|
||||
|
||||
### Task 2.6: Style Migration to Tailwind Primitives
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
Replace inline styles and legacy CSS class names with `/dist/app.css` Tailwind primitives from Wave 1:
|
||||
|
||||
| Element | Old style | New class |
|
||||
|---------|-----------|-----------|
|
||||
| Tool buttons | `tl-tool-btn` | `wd-btn wd-btn--ghost wd-btn--sm` |
|
||||
| Primary action | `btn primary sm` | `wd-btn wd-btn--primary wd-btn--sm` |
|
||||
| Secondary action | `btn ghost sm` | `wd-btn wd-btn--ghost wd-btn--sm` |
|
||||
| Save status | `tl-save-status` | `wd-text-tertiary text-xs` |
|
||||
| Media panel | `media-panel` | Custom component class |
|
||||
| Form fields | `field-input` | `wd-input` |
|
||||
| Labels | `form-label` | `wd-label` |
|
||||
| Dividers | `tl-sep` | `wd-topbar-divider` |
|
||||
| Sequence select | inline styles | `wd-select wd-select--sm` |
|
||||
|
||||
Add the Tailwind link if not already present:
|
||||
```html
|
||||
<link rel="stylesheet" href="/dist/app.css">
|
||||
```
|
||||
(May already be in `index.html` from the Wave 1 build — verify before adding.)
|
||||
|
||||
**Verify:** Editor looks consistent with other migrated pages. No visual regression in the editor layout.
|
||||
|
||||
---
|
||||
|
||||
### Task 2.7: Growing File / HLS Live Preview
|
||||
|
||||
See the detailed spec at `docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md` Section 2.
|
||||
|
||||
**Capture service changes:**
|
||||
1. Add `HLS_SESSION_DIR` env var (default: `/tmp/wd-hls`)
|
||||
2. In `capture-manager.js` `start()`: add HLS output pad to FFmpeg args
|
||||
3. Add route `GET /capture/live/:sessionId/index.m3u8` and `GET /capture/live/:sessionId/:file`
|
||||
4. In `stop()`: wait for FFmpeg exit, stitch segments to MP4, upload to S3, cleanup
|
||||
|
||||
**Frontend (capture screen):**
|
||||
MODIFY `services/web-ui/public/screens-ingest.jsx` — when recorder status shows `liveUrl`:
|
||||
```jsx
|
||||
const hlsRef = React.useRef(null);
|
||||
React.useEffect(() => {
|
||||
if (!recorder.liveUrl) return;
|
||||
if (typeof Hls !== 'undefined') {
|
||||
const hls = new Hls();
|
||||
hls.loadSource(recorder.liveUrl);
|
||||
hls.attachMedia(videoRef.current);
|
||||
hlsRef.current = hls;
|
||||
return () => hls.destroy();
|
||||
}
|
||||
}, [recorder.liveUrl]);
|
||||
```
|
||||
Load `hls.js` from CDN in `index.html`:
|
||||
```html
|
||||
<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
|
||||
```
|
||||
|
||||
**Verify:** During active recording, the capture/recorder screen shows a live HLS preview. After stop, the preview disappears and the full proxy appears in the library.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Tasks
|
||||
|
||||
### Task 3.1: Timeline Conform Integration
|
||||
|
||||
**Files:** MODIFY `services/mam-api/src/routes/sequences.js`, MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
**Backend:** Add FCP XML export endpoint following the EDL pattern:
|
||||
```js
|
||||
// POST /api/v1/sequences/:id/export/fcpxml
|
||||
router.post('/:id/export/fcpxml', async (req, res, next) => {
|
||||
// Generate FCP XML from sequence clips
|
||||
// Set Content-Type: application/xml
|
||||
});
|
||||
```
|
||||
|
||||
**Frontend:** In the editor topbar, add a "Conform" button that:
|
||||
1. Opens a slide-panel with conform options: resolution (1080p/4K), codec (H.264/ProRes/H.265), preset (Broadcast/Web/Archive)
|
||||
2. On submit: exports EDL/FCPXML, POSTs to the existing conform queue via `apiFetch('/sequences/' + id + '/export/conform', { method: 'POST', body: JSON.stringify({ format, resolution, codec }) })`
|
||||
3. Shows a job status indicator
|
||||
4. Jump to Jobs screen on completion
|
||||
|
||||
**Verify:** Select preset → click Conform → job appears in Jobs screen with type "Conform" → completes → conformed file is available.
|
||||
|
||||
---
|
||||
|
||||
### Task 3.2: Hi-Res Auto-Relink from Editor
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
Add "Relink to hi-res" button in the editor toolbar:
|
||||
1. Gather all unique assets referenced by timeline clips
|
||||
2. POST `/api/v1/assets/relink` with the asset IDs
|
||||
3. Server creates trim jobs → BullMQ trims hi-res segments from S3 originals
|
||||
4. Poll for completion, then update clip `streamUrl`s to point to hi-res segments
|
||||
5. Show progress bar per asset
|
||||
|
||||
**Verify:** Click Relink → hi-res trims appear → timeline clips now play hi-res segments instead of proxies.
|
||||
|
||||
---
|
||||
|
||||
### Task 3.3: Timecoded Comments
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-editor.jsx`
|
||||
|
||||
Add a comments panel to the editor (right side, collapsible):
|
||||
|
||||
```jsx
|
||||
<div className="editor-comments">
|
||||
<div className="editor-comments-header">Comments</div>
|
||||
<div className="editor-comments-list">
|
||||
{comments.map(c => (
|
||||
<div key={c.id} className="editor-comment" onClick={() => Timeline.setPlayhead(TC.secondsToFrames(c.timecode_seconds))}>
|
||||
<span className="editor-comment-tc">{TC.framesToTC(TC.secondsToFrames(c.timecode_seconds))}</span>
|
||||
<span className="editor-comment-body">{c.body}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
<div className="editor-comments-input">
|
||||
<input type="text" placeholder="Comment at current timecode..." />
|
||||
<button onClick={postComment}>Post</button>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
`postComment` calls the existing comments API:
|
||||
```js
|
||||
apiFetch('/assets/' + clip.asset_id + '/comments', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ timecode_seconds: TC.framesToSeconds(playheadFrames), body }),
|
||||
});
|
||||
```
|
||||
|
||||
**Verify:** Navigate to a frame → post comment → comment appears in list → click comment jumps to that frame.
|
||||
|
||||
---
|
||||
|
||||
### Task 3.4: Player Rebuild (P1)
|
||||
|
||||
See `docs/superpowers/specs/2026-05-18-editor-growing-file-features-design.md` Section 3 P1.
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-asset.jsx`
|
||||
|
||||
Changes:
|
||||
- Replace browser default video controls with custom transport bar (scrub slider + timecode display at 59.94 DF + frame-step buttons + JKL)
|
||||
- Inline rename: click `display_name` → editable input → auto-save on blur
|
||||
- "Open in Editor" button → `navigate('editor')` + pass asset context
|
||||
- Migrate from legacy `var(--color-bg-tertiary)` CSS to current Wave 1 tokens (`var(--bg-panel)`, etc.)
|
||||
|
||||
---
|
||||
|
||||
### Task 3.5: Subclips (P2)
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-asset.jsx`, MODIFY `services/mam-api/src/routes/assets.js`
|
||||
|
||||
**Backend:**
|
||||
New columns on `assets`: `parent_asset_id UUID REFERENCES assets`, `subclip_in_ms BIGINT`, `subclip_out_ms BIGINT`
|
||||
Migration 014. POST `/api/v1/assets` accepts `{ parent_asset_id, subclip_in_ms, subclip_out_ms, display_name, project_id }`.
|
||||
|
||||
**Frontend:**
|
||||
In the player, when in/out markers are set, show "Create Subclip" button:
|
||||
```jsx
|
||||
{srcIn !== null && srcOut !== null && (
|
||||
<button onClick={createSubclip} className="wd-btn wd-btn--primary wd-btn--sm">
|
||||
✂ Create Subclip
|
||||
</button>
|
||||
)}
|
||||
```
|
||||
`createSubclip` → `apiFetch('/assets', { method: 'POST', body: JSON.stringify({ parent_asset_id: asset.id, subclip_in_ms, subclip_out_ms, display_name, project_id }) })`
|
||||
|
||||
Library shows subclips with a ✂ badge; player pre-seeks to `subclip_in_ms`.
|
||||
|
||||
---
|
||||
|
||||
### Task 3.6: Multi-select & Bulk Ops (P3)
|
||||
|
||||
**Files:** MODIFY `services/web-ui/public/screens-library.jsx`
|
||||
|
||||
- Add checkbox to each asset card (visible on hover)
|
||||
- Shift-click range selection
|
||||
- Floating action bar when >= 1 selected: "Move to bin" (slide-panel with bin tree), "Add tags" (tag input), "Delete" (confirm modal)
|
||||
- Bulk operations call `PATCH /api/v1/assets/bulk` with `{ ids: [...], action: 'move_bin' | 'add_tags' | 'delete', value: ... }`
|
||||
|
||||
---
|
||||
|
||||
### Task 3.7: Smart Bins (P6)
|
||||
|
||||
**Files:** MODIFY `services/mam-api/src/routes/bins.js`, migration 015
|
||||
|
||||
**Backend:**
|
||||
```sql
|
||||
ALTER TABLE bins ADD COLUMN is_smart BOOLEAN DEFAULT false;
|
||||
ALTER TABLE bins ADD COLUMN smart_query JSONB;
|
||||
```
|
||||
GET `/api/v1/bins/:id/assets` for smart bins executes a dynamic query based on `smart_query`'s filters (tags, media_type, created_after, etc.).
|
||||
|
||||
**Frontend:**
|
||||
Smart bins show with ✦ icon in the bin tree. When selected, assets are fetched from `GET /api/v1/bins/:id/assets` instead of filtered client-side.
|
||||
|
||||
---
|
||||
|
||||
### Task 3.8: Metadata Templates (P7)
|
||||
|
||||
**Files:** Migration 016, MODIFY `services/mam-api/src/routes/projects.js`, MODIFY `services/mam-api/src/routes/assets.js`, MODIFY `services/web-ui/public/screens-admin.jsx` or `screens-projects.jsx`
|
||||
|
||||
```sql
|
||||
CREATE TABLE project_metadata_fields (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
project_id UUID NOT NULL REFERENCES projects ON DELETE CASCADE,
|
||||
field_key TEXT NOT NULL,
|
||||
label TEXT NOT NULL,
|
||||
field_type TEXT NOT NULL DEFAULT 'text',
|
||||
options JSONB,
|
||||
required BOOLEAN DEFAULT false,
|
||||
sort_order INTEGER DEFAULT 0
|
||||
);
|
||||
ALTER TABLE assets ADD COLUMN metadata JSONB DEFAULT '{}';
|
||||
```
|
||||
|
||||
**Frontend:** In project settings, add a "Metadata fields" tab where users define per-project metadata schemas. In asset detail, the metadata form renders based on the project's template (text inputs, selects, checkboxes, etc.).
|
||||
|
||||
---
|
||||
|
||||
## Verification Plan
|
||||
|
||||
After each Phase is complete:
|
||||
|
||||
```bash
|
||||
# 1. Rebuild web-ui
|
||||
docker compose up -d --build web-ui
|
||||
|
||||
# 2. Verify HTTP
|
||||
curl -sk -o /dev/null -w '%{http_code}' http://localhost:47434/ # → 200
|
||||
|
||||
# 3. Smoke test the editor route
|
||||
curl -sk http://localhost:47434/ | grep -c 'screens-editor.jsx' # → 1
|
||||
|
||||
# 4. API smoke (after Phase 1)
|
||||
curl -sk -b /tmp/wd.cookies \
|
||||
"http://localhost:47432/api/v1/sequences?project_id=$(curl -sk -b /tmp/wd.cookies http://localhost:47432/api/v1/projects | jq -r '.[0].id')"
|
||||
# → 200, array (possibly empty)
|
||||
|
||||
# 5. Full Phase 3 acceptance
|
||||
bash deploy/api-smoke.sh # → all 27 routes pass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
### Phase 1
|
||||
- [ ] Sequence API helpers added to `data.jsx` and exported on `window.ZAMPP_API`
|
||||
- [ ] Timecode utility available via `window.TC` or `useTimecode()`
|
||||
- [ ] Timeline React component wraps timeline.js with proper lifecycle
|
||||
- [ ] Source monitor loads proxy stream, supports in/out marking, Insert/Overwrite
|
||||
- [ ] Media panel shows project assets, filters by bin
|
||||
- [ ] Sequence picker lists sequences, supports create/open/save
|
||||
- [ ] Program monitor plays V1 clips with clip-by-clip virtual playback
|
||||
- [ ] Auto-save fires 2s after any timeline change
|
||||
- [ ] Undo/redo stack (Ctrl+Z / Ctrl+Shift+Z)
|
||||
- [ ] EDL export downloads valid CMX3600 file
|
||||
- [ ] Keyboard shortcuts: I/O, V/C/H, Space/JKL, Delete, arrows
|
||||
- [ ] "In Development" overlay removed
|
||||
- [ ] Editor nav item has no DEV badge
|
||||
|
||||
### Phase 2
|
||||
- [ ] Ripple delete shifts downstream clips
|
||||
- [ ] Snap-to-playhead on drag
|
||||
- [ ] Track locking (V2, A1, A2 lockable)
|
||||
- [ ] Clip overlap prevention
|
||||
- [ ] Zoom slider (20–500 px/s)
|
||||
- [ ] JKL transport with speed levels
|
||||
- [ ] Frame stepping (←/→, shift+←/→)
|
||||
- [ ] Waveform display on audio track clips
|
||||
- [ ] Inspector panel shows real selected clip data
|
||||
- [ ] Editor styles migrated to `/dist/app.css` Tailwind primitives
|
||||
- [ ] HLS live preview works during capture
|
||||
- [ ] Capture service: HLS segment writing, stitching, cleanup
|
||||
|
||||
### Phase 3
|
||||
- [ ] FCP XML export endpoint
|
||||
- [ ] Timeline conform via BullMQ queue
|
||||
- [ ] Hi-Res Auto-Relink from editor
|
||||
- [ ] Timecoded comments in editor
|
||||
- [ ] Player rebuilt with custom transport + 59.94 TC
|
||||
- [ ] Subclip creation from player markers
|
||||
- [ ] Multi-select with shift-click/checkbox
|
||||
- [ ] Bulk operation action bar (move, tag, delete)
|
||||
- [ ] Smart bins with dynamic queries
|
||||
- [ ] Metadata template CRUD per project
|
||||
- [ ] Metadata JSONB form in asset detail
|
||||
Loading…
Reference in a new issue