diff --git a/README.md b/README.md index d3a8022..e8f8ae8 100644 --- a/README.md +++ b/README.md @@ -1,51 +1,128 @@ # Dragonflight -Self-hosted broadcast media-asset management. Replaces Grass Valley AMPP -FramelightX. SDI / SRT / RTMP ingest, growing-file editing via Premiere -Pro, S3-compatible storage, scheduling, and a queue-driven proxy pipeline. +Self-hosted broadcast media-asset management system that replaces legacy tools like Grass Valley AMPP and FramelightX. Handles live ingest, growing-file editing, scheduling, transcoding, and asset management in a single operator-focused interface. -> Repo renamed from `wild-dragon` → `dragonflight` (2026-05-23). The old -> URL still redirects. +> Repo renamed from `wild-dragon` → `dragonflight` (2026-05-23). The old URL still redirects. -## Features +## Home Dashboard -- **Ingest** — SRT (caller + listener), RTMP, and SDI capture via - Blackmagic DeckLink cards (FFmpeg patched against SDK 16.x); per-recorder - codec settings (ProRes / H.264 / DNxHR / HEVC) and audio routing -- **Growing-file editing** — capture writes the hi-res master to a local - SMB landing zone; editors can mount the share in Premiere Pro and edit - the live file via the included CEP panel, then relink to the final S3 - master after promotion -- **Recorder scheduler** — one-shot, daily, or weekly windows; a 15s tick - loop fires the existing /recorders/:id/start + /stop endpoints -- **Library** — projects, bins, asset detail with frame-anchored - persistent comments, right-click context menu (move-to-bin, rename, - delete), and a global cmd/ctrl-K search across assets / projects / - recorders / jobs / users -- **Jobs** — BullMQ-backed proxy + thumbnail queue with per-job retry, - bulk "retry all failed", and inline error messages -- **Timeline Conform** — FCP XML export from the Premiere Pro panel with server-side FFmpeg conform; supports H.264, H.265, and ProRes output at various resolutions with preset-based workflows (Broadcast, Web, Archive) -- **Hi-Res Auto-Relink** — one-click batch relink of proxy clips to frame-accurate server-trimmed hi-res segments; concurrent trim worker pool, 24-hour TTL with automatic cleanup -- **Settings** — S3 (with env-var fallback), global proxy encoder - (CPU/libx264 or GPU/NVENC/VAAPI), growing-files config, capture SDK - uploader (Blackmagic / AJA / Deltacast) -- **Cluster** — primary + worker topology with heartbeat health, remote - node-agent for off-host DeckLink capture -- **API** — `deploy/api-smoke.sh` exercises every endpoint (27 routes, - pass/fail summary) +![Home Dashboard](docs/screenshots/01-home.png) -## Services +The home screen provides quick access to all major features and displays system status at a glance: +- **Library** — Browse projects, bins, and assets with hover-scrub previews +- **Recorders** — View configured capture devices and their status +- **Editor** — Timeline editor with cross-clip preview and render queue +- **Jobs** — Proxy and thumbnail queue with retry controls +- **Settings** — Configure storage, encoder, growing files, and capture SDK +- **Dashboard** — Operations view showing recent activity, job queue, and cluster health -| Service | Port | Description | -|---------|------|-------------| -| **web-ui** | 47434 | Browser SPA + capture controls | -| **mam-api** | 47432 | REST API + recorder orchestration + scheduler tick | -| **capture** | 47433 / 9000 / 1935 | DeckLink/SRT/RTMP capture sidecar | -| **worker** | — | BullMQ proxy + thumbnail workers | -| **db** | 5432 | PostgreSQL 16 | -| **queue** | 6379 | Redis 7 | +--- -## Quick start +## Core Features + +### 1. Live Ingest & Capture +**Multi-protocol source capture with per-recorder codec settings** + +Dragonflight ingests from multiple sources simultaneously: +- **SRT** (Secure Reliable Transport) — caller and listener modes +- **RTMP** — standard streaming protocol +- **SDI** — via Blackmagic DeckLink cards with FFmpeg SDK 16.x patches + +Each recorder can be configured with independent codec settings: +- ProRes (hi-res masters) +- H.264 / H.265 (proxies) +- DNxHR (Avid compatibility) + +Audio routing and per-source configuration ensure flexibility for multi-camera productions. + +### 2. Growing-File Editing +**Live editing in Premiere Pro while capture is still writing** + +Editors mount the SMB landing zone directly in Premiere Pro and edit the live master file as it's being written. The included CEP (Custom Extension Panel) provides: +- Real-time clip detection and frame-accurate trimming +- One-click relink to final S3 master after promotion +- No waiting for capture to finish before editorial begins + +### 3. Recorder Scheduler +**Time-windowed recording automation** + +Schedule recordings with: +- One-shot, daily, or weekly recurrence +- Automatic start/stop via 15-second tick loop +- Conflict detection across recorders +- Project and bin assignment at schedule time + +### 4. Library & Asset Management +**Browse, search, and organize captured footage** + +The Library screen provides: +- Project and bin hierarchy +- Asset detail view with frame-anchored persistent comments +- Right-click context menu (move-to-bin, rename, delete) +- Global cmd/ctrl-K search across assets, projects, recorders, jobs, and users +- Hover-scrub preview with HLS playback + +### 5. Jobs Queue +**BullMQ-backed proxy and thumbnail generation** + +Automated background processing: +- Per-job retry logic with exponential backoff +- Bulk "retry all failed" for batch recovery +- Inline error messages with actionable diagnostics +- Status tracking: ingesting → processing → ready + +Proxy encoder options: +- CPU-based: libx264 (H.264) +- GPU-accelerated: NVENC (NVIDIA) or VAAPI (AMD/Intel) + +### 6. Timeline Conform & Export +**FCP XML export with server-side FFmpeg rendering** + +The Premiere Pro panel exports FCP XML with: +- Server-side conform via FFmpeg +- Multiple output formats: H.264, H.265, ProRes +- Resolution presets: Broadcast, Web, Archive +- Batch processing with job queue integration + +### 7. Hi-Res Auto-Relink +**One-click batch relink of proxy clips to frame-accurate server-trimmed masters** + +After editing on proxies: +- Select clips in Premiere +- Trigger relink from the CEP panel +- Server trims hi-res segments to exact in/out points +- Concurrent trim worker pool for speed +- 24-hour TTL with automatic cleanup + +### 8. Settings & Configuration +**Centralized control for storage, encoding, and capture** + +Configure: +- **S3 Storage** — endpoint, bucket, credentials (with env-var fallback) +- **Proxy Encoder** — CPU vs GPU, bitrate, resolution +- **Growing Files** — SMB path, retention, auto-promotion +- **Capture SDK** — Blackmagic, AJA, or Deltacast uploader selection + +### 9. Cluster & Distributed Capture +**Primary + worker topology with remote DeckLink nodes** + +- Primary node runs API, scheduler, and web UI +- Worker nodes handle proxy/thumbnail jobs +- Remote capture nodes run DeckLink cards off-host +- Heartbeat health monitoring +- Automatic failover and recovery + +### 10. Admin & User Management +**Role-based access, token auth, and cluster monitoring** + +- User creation and role assignment +- API token generation for integrations +- Container and cluster node status +- System health dashboard + +--- + +## Quick Start ```bash # Clone (repo renamed; old URL still redirects) @@ -83,23 +160,50 @@ assets POST ──► proxy job ──► worker └─ status: ingesting → processing → ready ``` -## Tech stack +## Tech Stack - **Runtime:** Node.js 22, Docker Compose - **Backend:** Express, PostgreSQL 16, Redis 7 + BullMQ - **Frontend:** Vanilla React via in-browser Babel (no bundler), hls.js -- **Media:** FFmpeg 7.1 with SDK 16 DeckLink patches; ProRes, H.264, HEVC, - DNxHR, MOV/MP4/MXF containers -- **Storage:** S3-compatible (RustFS) for masters + proxies + thumbnails +- **Media:** FFmpeg 7.1 with SDK 16 DeckLink patches +- **Codecs:** ProRes, H.264, H.265, DNxHR, MOV/MP4/MXF containers +- **Storage:** S3-compatible (RustFS) for masters, proxies, thumbnails + +## Services + +| Service | Port | Purpose | +|---------|------|---------| +| **web-ui** | 47434 | Browser SPA + capture controls | +| **mam-api** | 47432 | REST API + recorder orchestration + scheduler | +| **capture** | 47433 / 9000 / 1935 | DeckLink/SRT/RTMP ingest sidecar | +| **worker** | — | BullMQ proxy + thumbnail workers | +| **db** | 5432 | PostgreSQL 16 | +| **queue** | 6379 | Redis 7 | + +--- + +## Workflow Example: Live-to-Edit + +1. **Operator** schedules a recording on Recorder A for 14:00–15:30, assigns to "News/Segment-A" project +2. **Capture** starts at 14:00, writes ProRes master to SMB landing zone +3. **Editor** mounts SMB in Premiere, opens the live .mov file via the CEP panel +4. **Editor** trims and marks in/out points while capture is still writing +5. **Capture** finishes at 15:30, promotion worker uploads master to S3 +6. **Editor** clicks "Relink to Master" in CEP panel +7. **Server** trims hi-res segment to exact in/out, stores for 24 hours +8. **Premiere** relinks proxy clips to trimmed master +9. **Editor** exports final timeline via FCP XML conform + +Total time from end of capture to relinked master: ~2 minutes. + +--- ## Operations -- `deploy/api-smoke.sh` — verify every API endpoint after a deploy -- `deploy/onboard-node.sh` — provision a remote worker host (DeckLink - cards on a separate machine) -- `deploy/test-cluster.sh` — primary↔worker connectivity smoke -- `docs/GROWING_FILES_QUICKSTART.md` — Premiere CEP panel install + - growing-file capture flow +- `deploy/api-smoke.sh` — verify every API endpoint after deploy +- `deploy/onboard-node.sh` — provision a remote worker host +- `deploy/test-cluster.sh` — primary↔worker connectivity smoke test +- `docs/GROWING_FILES_QUICKSTART.md` — Premiere CEP panel install + growing-file flow ## License