diff --git a/.env.example b/.env.example index b220868..493f47f 100644 --- a/.env.example +++ b/.env.example @@ -25,6 +25,13 @@ MAM_API_URL=http://mam-api:3000 # Auth — default to ON in production. Setting to 'false' is a dev-only escape # hatch that disables all auth checks and attaches a synthetic 'dev' user to # every request. Never run with AUTH_ENABLED=false on a network you don't control. +# +# RBAC v2 note: with AUTH_ENABLED=true, per-project access is enforced. Service +# API tokens (capture sidecar, Premiere panel, integrations) must belong to a +# user with the access they need — an 'admin' user (full access), or a user with +# the right project grants. A non-admin service token with no grants will get +# 403 on asset registration (ingest) and streaming. In dev mode the synthetic +# user is admin, so this only matters once auth is on. AUTH_ENABLED=true # CORS allowlist — comma-separated origins that may carry credentials to the API. @@ -36,3 +43,30 @@ ALLOWED_ORIGINS= # so secure-cookie + X-Forwarded-Proto behave correctly. ALSO required for accurate # per-IP login rate-limiting (otherwise req.ip is always the nginx IP). TRUST_PROXY=false + +# Google OAuth (OIDC) sign-in — OPTIONAL. Leave the client id/secret blank to +# disable; the "Sign in with Google" button and the /auth/google routes only +# activate when all three of CLIENT_ID, CLIENT_SECRET, and REDIRECT_URL are set. +# Create an OAuth 2.0 Client (type: Web application) in Google Cloud Console and +# add OAUTH_REDIRECT_URL to its authorized redirect URIs. +GOOGLE_CLIENT_ID= +GOOGLE_CLIENT_SECRET= +# Must exactly match a redirect URI on the OAuth client, e.g. +# https://dragonflight.live/api/v1/auth/google/callback +OAUTH_REDIRECT_URL= +# Restrict sign-in to one Google Workspace domain (recommended). First login from +# an allowed-domain account auto-provisions a NEW 'viewer' account (matched only +# by Google's stable subject id, never by email — so a Google login can never +# seize a pre-existing local account). An admin then grants project access. +# Leave blank to allow any verified Google account to self-provision (NOT advised). +GOOGLE_ALLOWED_DOMAIN= +# Note: if a Google-linked account also has TOTP enabled, sign-in still requires +# the authenticator code (Google is treated as the first factor). Accounts without +# TOTP complete sign-in in one Google step. + +# Playout / Master Control (MCR) +# Image tag the mam-api spawns when a channel starts. Build with: +# docker compose --profile build-only build playout +PLAYOUT_IMAGE=wild-dragon-playout:latest +# Base AMCP port — each channel binds to BASE + channel_id (in CasparCG terms). +PLAYOUT_AMCP_BASE_PORT=5250 diff --git a/WORK_LOG_PLAYOUT.md b/WORK_LOG_PLAYOUT.md new file mode 100644 index 0000000..7da9aa3 --- /dev/null +++ b/WORK_LOG_PLAYOUT.md @@ -0,0 +1,101 @@ +# Playout / Master Control — Implementation Work Log + +**Branch:** `feat/playout-mcr` (off `main`) +**Started:** 2026-05-30 +**Status:** Code complete, awaiting runtime validation + +Tracks the build of the playout (MCR) subsystem against the design at +`docs/superpowers/specs/2026-05-30-playout-mcr-design.md`. + +--- + +## Commit sequence + +| # | Commit | Scope | +|---|--------|-------| +| 1 | `docs(playout)` | Design spec, §7 questions answered | +| 2 | `feat(mam-api): migration 029` | Six tables, failover columns, audio_normalized flag | +| 3 | `feat(worker): playout-stage` | S3 → /media + EBU R128 loudnorm + index.js wiring | +| 4 | `feat(playout): sidecar` | CasparCG image + AMCP shim, HLS preview consumer, fps-aware frame math | +| 5 | `feat(mam-api): /playout control plane + auto-failover` | Routes + scheduler health tick + restartChannel helper | +| 6 | `feat(web-ui): MCR page` | screens-playout, styles, app/shell/index.html wiring | +| 7 | `build(playout): compose wiring + .env knobs` | /media volume, queue addition, build-only service | +| 8 | `docs(playout): work log` | This file | + +## Resolved §7 decisions (2026-05-30) + +- **Audio loudness:** pre-normalize at stage time. ffmpeg `loudnorm` two-pass + (I=-23 LUFS, TP=-1 dBTP, LRA=11), linear mode preserves dynamics. Output + AAC 192k @ 48 kHz, video stream copied. Per-item `audio_normalized` flag + so re-stages of the same asset skip the pass. +- **Frame rate:** `1080p5994` default (was `1080i5994`). Per-channel + override allowed via `video_format`. `fpsFor(videoFormat)` helper in + the sidecar drives SEEK / LENGTH / transition-frames math. +- **Preview latency:** HLS v1. CasparCG runs a second FFMPEG consumer + alongside the primary output, writing `/media/live//index.m3u8` + (~600 kbps, 2s segments, 6-window list). Web UI plays via the existing + HLS plumbing. +- **Failover:** auto-restart on healthy node for NDI/SRT/RTMP. Alert-only + for DeckLink (device-index pinning makes blind re-placement risky). + Scheduler tick (PG advisory lock, same lock as recorder schedules) polls + sidecar `/status`; ~3 missed checks → `restartChannel(id)` picks the most + recently-seen-online other node, bumps `restart_count`, calls `/start`. + +## Architecture notes + +**Sidecar model.** One CasparCG container per channel. Spawned by mam-api +via local Docker socket (primary node) or remote node-agent +`/sidecar/start`. Tracked in `playout_sidecars` plus `playout_channels.container_id`. +Killed on `/stop` or by `restartChannel` during failover. + +**Media flow.** +``` +S3 master/proxy → playout-stage worker → /media/playout/. + (loudnormed, AAC@-23 LUFS) + ↓ + CasparCG channel #1 + ↓ + primary consumer HLS consumer + (DeckLink/NDI/ ↓ + SRT/RTMP) /media/live//*.m3u8 +``` + +**Port contention.** `assertDeckLinkFree()` blocks starting a SDI channel +when a recorder or another channel on the same node+device_index is active. + +**Failover scope.** NDI/SRT/RTMP have no hardware tie, so any healthy +cluster_node is eligible. DeckLink channels surface an alert in the UI +(`status='error'` + `error_message`) and require operator intervention. + +## Testing checklist + +- [ ] Apply migration 029 on dev DB +- [ ] Build playout image: `docker compose --profile build-only build playout` +- [ ] Build web-ui (`screens-playout` joins the esbuild list automatically) +- [ ] Create channel via POST /api/v1/playout/channels (SRT first, no HW) +- [ ] Stage 2-3 assets to a playlist, verify loudnorm metadata in stderr +- [ ] Start channel → sidecar container appears in `docker ps` +- [ ] AMCP smoke: `telnet 5250`, `VERSION`, `INFO` +- [ ] Play playlist; verify HLS at /media/live//index.m3u8 +- [ ] Skip / pause / resume / stop +- [ ] As-run log: GET /api/v1/playout/channels/:id/asrun +- [ ] Kill sidecar container → scheduler should restart on another node + within ~3 ticks (~45s), restart_count increments +- [ ] DeckLink channel kill: status flips to 'error', NO restart attempt +- [ ] Try starting a decklink channel on a device_index already held by a + recorder → 409 +- [ ] MCR UI smoke: nav entry visible, page renders, drag-drop adds items, + transport buttons hit the API + +## Known gaps (deferred) + +- No WebRTC preview (HLS-only v1 — 4-6s lag, fine for confidence monitor). +- No graphics/CG overlay layer in Phase A (templates land in Phase B). +- No Phase B scheduler / 24/7 wall-clock channel (schema is in place, + scheduler tick is not). +- No multi-channel grid view (one channel at a time per page). +- No timecode / remaining-duration overlay (would need CasparCG INFO poll). +- No audio level meters on the UI. +- `restartChannel` updates DB state and triggers `/start`; if the new node + also fails repeatedly, there's no exponential backoff yet — bounded only + by the manual stop button. diff --git a/docker-compose.worker.yml b/docker-compose.worker.yml index 629516e..110ccd0 100644 --- a/docker-compose.worker.yml +++ b/docker-compose.worker.yml @@ -103,6 +103,7 @@ services: # worker-l4: HEAVY tier (proxy/conform/trim) on the L4 (NVENC). Talks to # zampp1's Redis/Postgres/S3 over the LAN (.200). No promotion scanner here. worker-l4: + profiles: [gpu] build: context: ./services/worker dockerfile: Dockerfile.gpu diff --git a/docker-compose.yml b/docker-compose.yml index 5733471..3e66986 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -40,6 +40,7 @@ services: - /var/run/docker.sock:/var/run/docker.sock - /mnt/NVME/MAM/wild-dragon-live:/live - /mnt/NVME/MAM/wild-dragon-growing:/growing + - /mnt/NVME/MAM/wild-dragon-media:/media - /mnt/NVME/MAM/sdk:/sdk - /dev/shm:/dev/shm - /run/dbus:/run/dbus @@ -61,6 +62,8 @@ services: NODE_IP: ${NODE_IP} NODE_HOSTNAME: ${NODE_HOSTNAME:-} CAPTURE_TOKEN: ${CAPTURE_TOKEN} + PLAYOUT_IMAGE: ${PLAYOUT_IMAGE:-wild-dragon-playout:latest} + PLAYOUT_AMCP_BASE_PORT: ${PLAYOUT_AMCP_BASE_PORT:-5250} deploy: resources: reservations: @@ -116,14 +119,20 @@ services: S3_SECRET_KEY: ${S3_SECRET_KEY} S3_REGION: ${S3_REGION:-us-east-1} GROWING_PATH: /growing - WORKER_QUEUES: proxy,conform,trim + # Includes `import` (YouTube importer): the import queue had no consumer + # after the capability-routing split, so import jobs sat unprocessed and + # assets stayed `ingesting` forever. import is concurrency-1 + network- + # bound, so one consumer (this heavy/primary worker) is sufficient. + WORKER_QUEUES: proxy,conform,trim,import,playout-stage RUN_PROMOTION: "true" PROXY_CONCURRENCY: "2" + PLAYOUT_MEDIA_DIR: /media NVIDIA_VISIBLE_DEVICES: GPU-79afca3e-2ab2-a6ea-1c44-706c1f0a26d6 WORKER_LABEL: "zampp1 / Tesla P4" NVIDIA_DRIVER_CAPABILITIES: video,compute,utility volumes: - /mnt/NVME/MAM/wild-dragon-growing:/growing + - /mnt/NVME/MAM/wild-dragon-media:/media networks: - wild-dragon @@ -172,12 +181,22 @@ services: - "${PORT_WEB_UI:-7434}:80" volumes: - /mnt/NVME/MAM/wild-dragon-live:/live + - /mnt/NVME/MAM/wild-dragon-media:/media:ro - /dev/shm:/dev/shm - /run/dbus:/run/dbus - /run/systemd:/run/systemd networks: - wild-dragon + # Build-only: the CasparCG sidecar image. mam-api spawns these on-demand per + # channel (one container per playout channel), so this service is never up'd — + # it exists so `docker compose build playout` produces the image the API tags + # via PLAYOUT_IMAGE. Profile excludes it from default `up`. + playout: + profiles: ["build-only"] + build: ./services/playout + image: wild-dragon-playout:latest + volumes: postgres_data: redis_data: diff --git a/docs/superpowers/specs/2026-05-30-playout-mcr-design.md b/docs/superpowers/specs/2026-05-30-playout-mcr-design.md new file mode 100644 index 0000000..c1bd8a3 --- /dev/null +++ b/docs/superpowers/specs/2026-05-30-playout-mcr-design.md @@ -0,0 +1,235 @@ +# Wild Dragon MAM — Playout / Master Control (MCR) + +**Date:** 2026-05-30 (revised 2026-05-30 — §7 closed) +**Status:** APPROVED — implementation in progress (code drafted but uncommitted; see WORK_LOG_PLAYOUT.md) +**Author:** Zac + Claude + +--- + +## Resolved Decisions + +| Question | Decision | +|----------|----------| +| Playout engine | **CasparCG Server** (orchestrated via AMCP), not ffmpeg-native | +| Channel count | **Multi-channel from start** — N independent channels, placed across cluster nodes by capability (mirrors recorders) | +| Scheduling model | **Phased** — Phase A: on-demand playlist player; Phase B: 24/7 continuous channel | +| Output targets | SDI (DeckLink), NDI, SRT, RTMP — all via CasparCG consumers | +| Media source | Assets live in **S3**; must be staged to a CasparCG-local media volume before play (see §4) | +| CasparCG packaging | **Build our own image** (like `capture/build-with-decklink.sh`) — GL context via GPU passthrough or Xvfb; NDI + DeckLink SDKs fetched at build time (not redistributable) | +| Master codec playability | Zac confirms current masters **play fine in CasparCG** — no transcode-for-playout step; staging is a plain S3→/media copy. Validate on hardware but do not gate on it | +| Management UI | **Single Dragonflight `playout.html` GUI** drives everything via AMCP; operator never touches CasparCG directly | + +--- + +## Overview + +Playout adds **master-control-room** capability to Dragonflight: take library assets, arrange them on a timeline / playlist, and play them out continuously to a broadcast output — SDI via DeckLink, or stream via SRT / RTMP / NDI. Drag-and-drop scheduling, a program/preview monitor, and as-run logging. + +This is the **mirror image** of the existing capture path. Capture is `input → ffmpeg encode → S3`. Playout is `S3 asset → engine → output`. We reuse three things wholesale: + +1. **Cluster node + capability model** — nodes already advertise DeckLink/Deltacast/GPU in `cluster_nodes.capabilities`; channels are placed on nodes that have a free output port, exactly as recorders claim input ports. +2. **Sidecar orchestration** — mam-api spawns containers via the local Docker socket or the remote `node-agent /sidecar/start`. A CasparCG channel is just a different sidecar image. +3. **Scheduler tick + PG advisory lock** — `src/scheduler.js` already runs a single-leader tick over a schedule table. Phase B's wall-clock channel reuses this pattern. + +### Why CasparCG over ffmpeg-native + +The capture stack proves we can drive ffmpeg + DeckLink. But playout's hard part is **gapless, frame-accurate, clean transitions between clips** — every clip boundary in an ffmpeg-per-clip model is a black flash unless we engineer a concat-feeder. CasparCG solves this natively: a channel is a persistent output with a playlist, hard/mix/wipe transitions, layered graphics/logo (CG), and DeckLink/NDI/SRT/RTMP consumers built in. We orchestrate it over **AMCP** (its TCP control protocol) instead of reinventing a feeder. Trade: a new dependency + container image, and media must be on a CasparCG-visible disk (§4). + +--- + +## 1. Data Model + +New migration `029-playout.sql`. Five tables. + +### 1.1 `playout_channels` +A logical output. One channel → one engine instance → one output target. + +``` +id uuid pk +name text -- "Channel 1", "Pop-up SDI" +node_id uuid -> cluster_nodes(id) -- where the engine runs (null = primary) +output_type text -- 'decklink' | 'ndi' | 'srt' | 'rtmp' +output_config jsonb -- { device_index } | { ndi_name } | { url, latency } | { url, key } +video_format text -- '1080i5994' | '1080p5994' | '720p5994' ... +status text -- 'stopped' | 'starting' | 'running' | 'error' +container_id text -- running CasparCG sidecar +project_id uuid -> projects(id) -- RBAC scoping (nullable = admin-only) +created_at / updated_at +``` + +`output_type` + `output_config` map straight to a CasparCG consumer: +- `decklink` → `ADD DECKLINK ...` +- `ndi` → `ADD NDI ...` +- `srt`/`rtmp` → `ADD FFMPEG -f mpegts ...` (CasparCG 2.3+ ffmpeg consumer) + +### 1.2 `playout_playlists` +An ordered list of items bound to a channel. Phase A's primary object. + +``` +id, channel_id -> playout_channels(id) +name, loop boolean, created_at / updated_at +``` + +### 1.3 `playout_items` +One entry on a playlist OR one entry on the 24/7 timeline. + +``` +id +playlist_id uuid -> playout_playlists(id) -- Phase A +asset_id uuid -> assets(id) +sort_order int -- position in playlist (Phase A) +scheduled_at timestamptz -- wall-clock start (Phase B, null in A) +in_point numeric -- seconds, trim head (reuse subclip in/out from editor) +out_point numeric -- seconds, trim tail +transition text -- 'cut' | 'mix' | 'wipe' +transition_ms int +graphics jsonb -- optional CG/template overlay (Phase B+) +media_status text -- 'pending' | 'staging' | 'ready' | 'error' (see §4) +media_path text -- resolved path inside the CasparCG media volume +``` + +### 1.4 `playout_schedule` (Phase B) +Day-ahead, wall-clock-bound timeline rows. Same shape as `playout_items` but `scheduled_at` is authoritative and the scheduler tick (§5) drives transitions. Phase A can ignore this table. + +### 1.5 `playout_as_run` +Append-only log: what actually played, when, for how long. Compliance / billing. + +``` +id, channel_id, asset_id, item_id +started_at, ended_at, duration_s, result -- 'played' | 'skipped' | 'error' +``` + +--- + +## 2. Services & Components + +### 2.1 New sidecar: `services/playout/` (CasparCG wrapper) +A thin container: **CasparCG Server** + a small Node control shim exposing HTTP, the same way `capture` wraps ffmpeg. + +- Base image: official/community `casparcg/server` (Linux build with DeckLink + NDI + FFmpeg producers/consumers). +- Node shim (`src/index.js`): opens an AMCP TCP socket to local CasparCG, exposes: + - `POST /channel/start` → `ADD ` for the channel's output target + - `POST /play` → `PLAY - [transition]` + - `POST /loadbg` + `/play` → preview/cue then take (preview monitor) + - `POST /stop`, `GET /status` → `INFO ` (current clip, position, fps) + - playlist load → translate `playout_items` rows into a sequence of AMCP `LOADBG`/`PLAY` calls, advancing on `OnTransition` / end-of-clip events. +- Mirrors capture's status-polling contract so the UI monitor reuses existing plumbing. + +### 2.2 mam-api: `src/routes/playout.js` +CRUD + control, RBAC-scoped via the existing `assertProjectAccess` helper (channels carry `project_id`). + +``` +GET /playout/channels list (project-filtered) +POST /playout/channels create (edit on project) +POST /playout/channels/:id/start|stop spawn/kill CasparCG sidecar +GET /playout/channels/:id/status proxy engine INFO +POST /playout/channels/:id/play|pause|skip transport control +GET/POST/PUT/DELETE /playout/playlists... playlist + item CRUD, reorder +POST /playout/items/:id/stage kick S3→media-volume staging (§4) +GET /playout/channels/:id/asrun as-run log +``` + +Channel start/stop reuses `resolveNodeTarget()` + the Docker-socket / `node-agent /sidecar/start` split already in `recorders.js`. **Refactor opportunity:** lift that sidecar-spawn logic out of `recorders.js` into `src/orchestration/sidecar.js` so both recorders and playout share it (keep this small — only what both need). + +### 2.3 web-ui: `playout.html` + `public/playout.jsx` +New MCR page. Layout: + +``` +┌─ PREVIEW ───────────┬─ PROGRAM (on air) ──────┐ +│ [cued clip] │ [live output] ● ON AIR │ +│ TC / duration │ TC / remaining │ +│ [CUE] [TAKE] │ [PLAY][PAUSE][SKIP][STOP]│ +├─ MEDIA BIN ─────────┴──────────────────────────┤ +│ (draggable asset list, reuse asset browser) │ +├─ PLAYLIST / TIMELINE ──────────────────────────┤ +│ ▸ clip A ──▸ clip B ──▸ clip C (drag-drop) │ Phase A: ordered list +│ └ 24h time grid w/ now-bar │ Phase B: time-of-day grid +└────────────────────────────────────────────────┘ +``` + +- Drag-drop: reuse whatever the NLE editor timeline uses (check `editor.jsx`); assets drag from the bin into the playlist/grid. +- API via existing `ZAMPP_API.fetch` wrapper. +- Program monitor: HLS preview of the output — CasparCG can emit a second low-bitrate FFmpeg consumer to HLS, reusing the `/live/` HLS plumbing capture already uses. + +--- + +## 3. Channel placement & ports + +A DeckLink port is exclusive — same constraint capture already handles. A node's DeckLink port can be an **input (recorder)** or an **output (playout channel)**, never both at once. So: + +- Extend the capability/port-claim check: when starting a channel on `output_type=decklink`, verify the target node has that device index free (no active recorder, no active channel). +- NDI / SRT / RTMP outputs have no hardware contention → can stack many per node (GPU/CPU-bound only). +- Surface a unified "device map" (extend the existing cluster DeckLink-status endpoint) showing each port's role: idle / recording / playing-out. + +--- + +## 4. Media staging (the S3 ⇄ CasparCG gap) + +**The crux.** Assets live in S3 (`original_s3_key` / `proxy_s3_key`). CasparCG plays from a **local media folder**. Options: + +- **A — Pre-stage to a shared media volume (recommended).** Before a clip can go on air, download/symlink it from S3 to a CasparCG-visible volume (`/media`), set `playout_items.media_status='ready'` + `media_path`. A new BullMQ `playout-stage` job (reuses the worker pattern) does the pull. UI shows per-item readiness; TAKE is blocked until `ready`. Mirrors the growing-file SMB share already mounted for capture. +- **B — Stream from S3 via presigned URL.** CasparCG FFmpeg producer plays an HTTPS presigned URL directly. Zero staging, but seeking/trim and gapless transitions over network are fragile for broadcast. Acceptable as a fallback for SRT/RTMP, risky for SDI. + +**Decision:** Phase A uses **A** (stage proxies for preview, masters for air) with **B** available as a per-channel "low-latency / no-stage" toggle. Zac confirms the current masters play fine in CasparCG, so staging is a **plain S3→/media copy — no transcode-for-playout step**. (Validate on hardware during implementation, but the model does not assume a transcode stage.) + +--- + +## 5. Scheduling + +### Phase A — playlist player +No wall clock. Operator builds a `playout_playlists` row, drags items in, hits PLAY. The playout sidecar walks `playout_items` by `sort_order`, cueing the next clip during the current one (`LOADBG`) and taking it at end-of-clip with the configured transition. `loop` repeats. As-run logged per item. + +### Phase B — 24/7 continuous channel +Wall-clock timeline in `playout_schedule`. Reuse `src/scheduler.js`: +- Add a second tick (or extend the existing one) under the **same PG advisory lock pattern** — exactly-one-leader, so a multi-replica deploy doesn't double-fire. +- Tick responsibilities: stage upcoming items (look-ahead window), verify the on-air item matches the schedule, **fill gaps** (loop a filler/slate asset when the timeline has a hole — a channel must never go black), roll the day forward. +- As-run becomes the compliance record. + +--- + +## 6. Phasing / Milestones + +**Phase A — Playlist playout MVP** +1. Migration `029-playout.sql` (channels, playlists, items, as-run). +2. `services/playout/` sidecar: CasparCG image + AMCP control shim, single output target (start with **SRT or NDI** — no hardware needed for dev; DeckLink behind hardware check). +3. mam-api `routes/playout.js` — channel + playlist CRUD, start/stop, transport, RBAC. +4. `playout-stage` BullMQ job (S3 → /media). +5. web-ui `playout.html` — bin + drag-drop ordered playlist + program/preview monitors + transport. +6. DeckLink output on real hardware; port-contention check vs recorders. + +**Phase B — 24/7 continuous channel** +7. `playout_schedule` + time-of-day grid UI. +8. Scheduler tick (advisory-locked) — look-ahead staging, gap-fill/slate, day-roll. +9. As-run reporting view. +10. Graphics/CG overlay (logo bug, lower-thirds) via CasparCG templates. + +--- + +## 7. Open Questions (for review) + +**Resolved (2026-05-30):** +- ~~CasparCG packaging~~ → **build our own image.** Fetch DeckLink + NDI SDKs at build time (not redistributable — same as capture's DeckLink build). GL context for the mixer comes from GPU passthrough on a real node, or **Xvfb** (virtual framebuffer) where there's no display — community images run `--privileged` + X11 socket. Pin the NDI SDK version to what the server expects (`.so` version mismatch is the common docker failure). +- ~~Master codec playability~~ → Zac confirms masters **play fine**; no transcode-for-playout. Staging = plain S3→/media copy. +- ~~Management GUI~~ → **single Dragonflight `playout.html`** drives everything via AMCP; operator never touches CasparCG. +- ~~Audio loudness~~ → **pre-normalize at stage time** (Zac, 2026-05-30). `playout-stage` job runs ffmpeg `loudnorm` (EBU R128, target −23 LUFS, true-peak −1 dBTP) once, on the S3→/media copy. Output is the cached version CasparCG plays. Staging is no longer a pure copy — staging cost ≈ realtime CPU per clip on first stage; results are reusable across channels. Override (`media_status='ready'` + raw copy) available for clips already mastered to spec. +- ~~Frame rate~~ → **`1080p5994`** default for new channels (Zac, 2026-05-30). Progressive 1080 @ 59.94 fps. Per-channel override allowed (`video_format` column). Streaming-friendly (SRT/RTMP/NDI) and current SDI gear accepts it; matches capture's 59.94 cadence. +- ~~Preview latency~~ → **HLS v1** (Zac, 2026-05-30). Reuse capture's `/live/` HLS plumbing. CasparCG emits a second low-bitrate FFmpeg consumer to HLS. ~4–6s lag, fine for confidence monitor. Operator desk gets a real downstream monitor off the SDI/NDI output anyway. Revisit WebRTC if MCR operators complain. +- ~~Failover~~ → **auto-restart on healthy node** (Zac, 2026-05-30). Scheduler tick (§5) monitors `playout_sidecars` health (AMCP ping + container alive); on N missed checks marks the channel `error`, re-places it on another capability-matching node with a free output port, resumes the playlist from the next item after the as-run-logged on-air item. Gap = black/slate for ~5–30 s during respawn (operator sees a flag in the UI). **DeckLink channels are not auto-failed-over in v1** — device-index pinning makes the destination port non-trivial; v1 alerts and lets the operator move the channel. NDI/SRT/RTMP channels (no hardware contention) failover automatically. Tracked via `restart_count` + `last_restart_at` on `playout_channels`. + +**Still open:** +- (none — all §7 questions resolved 2026-05-30) + +--- + +## 8. Reused building blocks (already in the repo) + +| Need | Existing piece | +|------|----------------| +| Spawn engine container local/remote | `recorders.js` Docker-socket + `node-agent /sidecar/start` | +| Node capability / port model | `cluster_nodes.capabilities`, cluster DeckLink-status endpoint | +| Single-leader scheduled transitions | `src/scheduler.js` + PG advisory lock | +| Background media jobs | BullMQ worker (`services/worker`) | +| RBAC scoping | `src/auth/authz.js` `assertProjectAccess` (channel/project_id) | +| HLS preview plumbing | capture's `/live/` HLS output | +| Subclip in/out points | NLE editor in/out marking | +| API wrapper / SPA shell | `ZAMPP_API.fetch`, esbuild JSX pages | diff --git a/docs/superpowers/specs/2026-05-31-storage-settings-growing-smb-design.md b/docs/superpowers/specs/2026-05-31-storage-settings-growing-smb-design.md new file mode 100644 index 0000000..863aa9c --- /dev/null +++ b/docs/superpowers/specs/2026-05-31-storage-settings-growing-smb-design.md @@ -0,0 +1,148 @@ +# Storage Settings Warning + Growing-Files SMB Auth + Per-Recorder Growing Mode +**Date:** 2026-05-31 +**Branch:** `feat/playout-mcr` (Forgejo: WildDragonLLC/dragonflight) +**Status:** Approved, ready for implementation plan + +--- + +## Scope + +Three related refinements to the Settings page and growing-files capture pipeline: + +1. **Storage warning header** — a prominent set-once warning at the top of the Storage settings section. +2. **Growing-files SMB credentials + system CIFS mount** — store an SMB username/password and have the capture stack mount the growing share itself (Approach A). +3. **Per-recorder growing mode** — remove the global "capture writes to local SMB share first" toggle; make growing-files mode a per-recorder setting. + +All changes live on the `growing-files` / storage-settings path. No playout changes (handled separately). + +--- + +## Background (current state) + +- **Settings storage:** single key/value `settings(key TEXT PRIMARY KEY, value TEXT, updated_at)` table (migration 006). Secrets like `s3_secret_key` are stored but `GET /settings/s3` returns only `s3_secret_key_exists` (never the value). +- **Growing settings UI:** `GrowingSettingsCard` in `services/web-ui/public/screens-admin.jsx` (rendered by `StorageSection` alongside `MountHealthStrip` and `S3SettingsCard`). Current keys: `growing_enabled`, `growing_path`, `growing_smb_url`, `growing_promote_after_seconds`. +- **Settings API:** `services/mam-api/src/routes/settings.js` — `GET/PUT /settings/growing` over `GROWING_KEYS`. +- **Global enable today:** the `growing_enabled` setting (checkbox labelled "Capture writes to the local SMB share first; Premier can edit while it's still growing"). `recorders.js` reads this global key at recorder-start and passes `GROWING_ENABLED=true/false` to the capture container. +- **Capture write path:** `services/capture/src/capture-manager.js` reads `process.env.GROWING_ENABLED` and `GROWING_PATH` (default `/growing`). When on, it writes the master to `/growing/{projectId}/{clipName}.{ext}` instead of streaming to S3; the promotion worker uploads to S3 after stop. +- **Current mount model:** `/growing` is a pre-mounted host bind-mount; the app never authenticates to SMB. +- **Per-recorder column already exists:** migration 014 added `recorders.growing_enabled BOOLEAN DEFAULT NULL` ("NULL = use global"), but recorder-start logic ignores it and the new-recorder modal does not expose it. + +--- + +## Part 1 — Storage warning header + +Add a danger-styled banner at the **top of `StorageSection()`**, above `MountHealthStrip`. + +- Visual: full-width banner, danger token styling (`--danger` border + subtle danger background), alert icon, bold uppercase text. +- Exact copy: + > **⚠ WARNING — THESE SETTINGS ARE MEANT TO BE SET ONCE AT INITIAL DEPLOYMENT. CHANGING STORAGE PATHS MID-CYCLE CAN CAUSE DATABASE CORRUPTION AND FILE LOSS. PLEASE USE WITH CAUTION.** +- Pure presentational; no backend, no dismiss state (always visible). + +**Files:** `services/web-ui/public/screens-admin.jsx` (and a small style rule in the appropriate CSS file if needed). + +--- + +## Part 2 — Growing-files SMB credentials + system CIFS mount (Approach A) + +The growing share is **shared infrastructure**, so the SMB connection config is global. + +### New settings keys +| Key | Purpose | Notes | +|-----|---------|-------| +| `growing_smb_mount` | CIFS source for the system mount, e.g. `//10.0.0.25/mam-growing` | Distinct from `growing_smb_url` | +| `growing_smb_username` | SMB user | Returned in GET (not secret) | +| `growing_smb_password` | SMB password | **Write-only** — never returned | +| `growing_smb_vers` *(optional)* | CIFS protocol version, default `3.0` | Avoids mount negotiation failures | + +`growing_smb_url` (the `smb://…` string) is retained unchanged as the **editor-facing** display value (Premiere connect string). + +### Settings API (`settings.js`) +- Extend `GROWING_KEYS` with the new keys (except the password is handled specially). +- `GET /settings/growing`: return `growing_smb_mount`, `growing_smb_username`, `growing_smb_vers`, and `growing_smb_password_exists: boolean` — **never** the password value. (Mirror the existing `s3_secret_key_exists` pattern.) +- `PUT /settings/growing`: upsert each provided key. For `growing_smb_password`, only write it when a non-empty value is provided (an empty/omitted field leaves the stored password unchanged). Provide a way to clear it (explicit empty sentinel or a "clear" affordance) — see Resolved decisions below. + +### Settings UI (`GrowingSettingsCard`) +Add three fields to the card: +- **SMB mount (CIFS):** text input bound to `growing_smb_mount`, placeholder `//10.0.0.25/mam-growing`. +- **SMB username:** text input bound to `growing_smb_username`. +- **SMB password:** masked password input. Shows a "saved" indicator when `growing_smb_password_exists` is true; typing a new value replaces it; leaving it blank keeps the existing one. `autoComplete="new-password"`. + +### Capture image (`services/capture/Dockerfile`) +Add `cifs-utils` to the installed packages so `mount -t cifs` is available inside the capture container. + +### Capture-manager (`capture-manager.js`) +On capture start, when growing mode is active **and** `GROWING_SMB_MOUNT` is set: +1. Write a root-only credentials file (e.g. `/run/smb-creds`, mode `0600`) containing: + ``` + username= + password= + ``` + (Credentials go in the file, **not** the mount command line, to avoid `ps`/log exposure.) +2. `mkdir -p $GROWING_PATH` then `mount -t cifs $GROWING_SMB_MOUNT $GROWING_PATH -o credentials=/run/smb-creds,uid=0,gid=0,file_mode=0664,dir_mode=0775,vers=$GROWING_SMB_VERS`. +3. If the mount succeeds → proceed writing the master to `$GROWING_PATH/...` (existing behaviour). +4. If the mount **fails** → log the error and fall back to S3 streaming (`growingPath = null`), so a recording is never lost. +5. On capture stop/cleanup, unmount `$GROWING_PATH` (best-effort; ignore "not mounted"). + +Mount isolation: each recorder runs in its **own** capture container, so each container mounts CIFS at its own private `/growing` — no cross-recorder mount conflicts, no ref-counting needed. + +### Recorder start (`recorders.js`) +- Pass new env to the spawned capture container: `GROWING_SMB_MOUNT`, `GROWING_SMB_USERNAME`, `GROWING_SMB_PASSWORD`, `GROWING_SMB_VERS` (read from the `settings` table at start). +- The dynamically-spawned capture container must get `/growing` as an **empty mountpoint** (not a host bind-mount) so the in-container CIFS mount lands cleanly. Confirm/adjust the container spec accordingly. The container is already privileged (required for `mount`). + +### Security notes +- The password is stored plaintext in the `settings` table, identical to the existing `s3_secret_key` handling — acceptable within this app's current secret model. +- The password reaches the capture container as an env var (visible via `docker inspect`), same as S3 keys already are. The credentials **file** (not the command line) is used for the actual mount. + +--- + +## Part 3 — Per-recorder growing mode (remove the global toggle) + +### Remove global enable +- Delete the global "Capture writes to the local SMB share first…" checkbox (`growing_enabled` key) from `GrowingSettingsCard`. The card no longer carries a global on/off — it is **infrastructure-only**: container mount path, SMB URL (editor), SMB mount + credentials, promote threshold. +- The `growing_enabled` settings *key* is retired from the UI. (It may remain in the table harmlessly; recorder-start no longer reads it.) + +### Per-recorder semantics +- Reuse the existing `recorders.growing_enabled BOOLEAN` column. New semantics (no global to defer to): `TRUE` = this recorder uses growing-files mode; `NULL`/`FALSE` = off. +- `recorders.js` recorder-start: read the **recorder's own** `growing_enabled` (defaulting `NULL`→off) and set `GROWING_ENABLED` for the capture container from that, instead of the global setting. +- Add `growing_enabled` to `RECORDER_FIELDS` so create/update accept it. + +### UI +- **New-recorder modal** (`modal-new-recorder.jsx`): add a "Growing-files mode" toggle that sets `growing_enabled` on the created recorder (default off). +- **Recorder edit** (wherever recorders are edited): same toggle. +- Helper text on the toggle notes that growing-files requires the SMB share to be configured in Settings → Storage. + +### Fallback +If a recorder has `growing_enabled = true` but `growing_smb_mount` is not configured globally, capture logs a warning and falls back to S3 streaming (same fallback path as a failed mount). Recording is never blocked. + +--- + +## Files changed + +| File | Change | +|------|--------| +| `services/web-ui/public/screens-admin.jsx` | Storage warning banner; SMB mount/username/password fields in `GrowingSettingsCard`; remove global growing-enable checkbox | +| `services/web-ui/public/modal-new-recorder.jsx` | Per-recorder "Growing-files mode" toggle | +| `services/mam-api/src/routes/settings.js` | New growing SMB keys; write-only password (`growing_smb_password_exists`) | +| `services/mam-api/src/routes/recorders.js` | Read per-recorder `growing_enabled`; pass SMB env to capture; `RECORDER_FIELDS` += `growing_enabled`; empty `/growing` mountpoint | +| `services/capture/Dockerfile` | Add `cifs-utils` | +| `services/capture/src/capture-manager.js` | CIFS mount-on-start (creds file), unmount-on-stop, fallback to S3 on failure | +| CSS (storage warning / fields) | Minor styles if needed | + +No DB migration required (the `recorders.growing_enabled` column already exists; new settings are key/value rows). + +--- + +## Resolved decisions + +- **Clearing the SMB password:** `PUT /settings/growing` treats a field value of the literal sentinel `""` with an explicit `growing_smb_password_clear: true` flag as "remove the stored password"; a blank field with no clear flag leaves it unchanged. (Keeps the common "don't retype the password on every save" UX while still allowing removal.) +- **CIFS version:** default `growing_smb_vers = 3.0`; overridable via settings to support older NAS targets. +- **Recorders already recording when the toggle changes:** the per-recorder `growing_enabled` is read at **start** only; changing it mid-recording has no effect on the active session (consistent with how all recorder encode settings already behave). + +--- + +## Out of scope (deferred) + +- Encrypting secrets at rest (the app's existing model stores `s3_secret_key` in plaintext; SMB password follows the same model). +- A global "growing-files master kill switch" (removed by design — control is now per-recorder). +- Exposing `growing_retention_days` in the UI (seeded in DB, still unsurfaced; unrelated to this work). +- Playout HLS preview fix (handled by a separate parallel effort). diff --git a/services/capture/Dockerfile b/services/capture/Dockerfile index 1f94e4e..455dc2c 100644 --- a/services/capture/Dockerfile +++ b/services/capture/Dockerfile @@ -67,10 +67,14 @@ RUN /usr/local/bin/ffmpeg -hide_banner -encoders 2>&1 | grep -E 'nvenc' \ # ── Stage 2: Runtime image ─────────────────────────────────────────────────── FROM node:20-bookworm -# Runtime deps for compiled ffmpeg libs +# Runtime deps for compiled ffmpeg libs. +# cifs-utils provides mount.cifs so growing-files capture can mount the SMB +# landing-zone share inside the (privileged) container at start (Approach A). +# util-linux supplies mount/umount/mountpoint. RUN apt-get update && apt-get install -y --no-install-recommends \ libx264-164 libx265-199 libvpx7 libopus0 libmp3lame0 \ libsrt1.5-openssl libzmq5 libstdc++6 libc++1 libc++abi1 \ + cifs-utils util-linux \ && rm -rf /var/lib/apt/lists/* # Copy compiled ffmpeg/ffprobe diff --git a/services/capture/src/capture-manager.js b/services/capture/src/capture-manager.js index 5f1315f..50108d8 100644 --- a/services/capture/src/capture-manager.js +++ b/services/capture/src/capture-manager.js @@ -1,5 +1,5 @@ -import { spawn } from 'child_process'; -import { mkdirSync } from 'node:fs'; +import { spawn, execFileSync } from 'child_process'; +import { mkdirSync, writeFileSync } from 'node:fs'; import { dirname } from 'node:path'; import { v4 as uuidv4 } from 'uuid'; import { createUploadStream } from './s3/client.js'; @@ -9,11 +9,76 @@ const S3_BUCKET = process.env.S3_BUCKET || 'wild-dragon'; // Growing-files mode: writes the master to a local SMB-backed share that the // editor can mount, instead of streaming to S3 in real time. The promotion // worker uploads the finalized file to S3 after the recording stops. -// Toggled per-process by `GROWING_ENABLED=true` on the capture container +// Toggled per-recorder via `GROWING_ENABLED=true` on the capture container // (see routes/recorders.js where the env is composed). const GROWING_ENABLED = process.env.GROWING_ENABLED === 'true'; const GROWING_PATH = process.env.GROWING_PATH || '/growing'; +// Approach A: when a CIFS source is supplied, this (privileged) container mounts +// the SMB landing-zone share at GROWING_PATH itself, using credentials supplied +// by the API from global Settings. Empty GROWING_SMB_MOUNT → no system mount +// (the host-bound /growing volume is used instead, or S3 streaming if growing +// is off). +const GROWING_SMB_MOUNT = process.env.GROWING_SMB_MOUNT || ''; +const GROWING_SMB_USERNAME = process.env.GROWING_SMB_USERNAME || ''; +const GROWING_SMB_PASSWORD = process.env.GROWING_SMB_PASSWORD || ''; +const GROWING_SMB_VERS = process.env.GROWING_SMB_VERS || '3.0'; +const SMB_CREDS_FILE = '/run/smb-creds'; + +// True when GROWING_PATH is already a mountpoint (e.g. a prior session left it +// mounted, or a host bind-mount is present). +function isMounted(path) { + try { execFileSync('mountpoint', ['-q', path]); return true; } + catch { return false; } +} + +// Mount the CIFS growing share at GROWING_PATH. Credentials go in a root-only +// file (NOT the command line) so they never appear in `ps`/process listings. +// Returns true on success (or if already mounted), false on failure — callers +// fall back to S3 streaming so a recording is never lost. +function mountGrowingShare() { + if (!GROWING_SMB_MOUNT) return false; + try { + if (isMounted(GROWING_PATH)) { + console.log('[capture] growing share already mounted at', GROWING_PATH); + return true; + } + try { mkdirSync(GROWING_PATH, { recursive: true }); } catch (_) {} + writeFileSync( + SMB_CREDS_FILE, + `username=${GROWING_SMB_USERNAME}\npassword=${GROWING_SMB_PASSWORD}\n`, + { mode: 0o600 } + ); + const opts = [ + `credentials=${SMB_CREDS_FILE}`, + 'uid=0', 'gid=0', 'file_mode=0664', 'dir_mode=0775', + `vers=${GROWING_SMB_VERS}`, + ].join(','); + execFileSync('mount', ['-t', 'cifs', GROWING_SMB_MOUNT, GROWING_PATH, '-o', opts], + { stdio: ['ignore', 'ignore', 'pipe'] }); + console.log('[capture] mounted CIFS growing share', GROWING_SMB_MOUNT, '->', GROWING_PATH); + return true; + } catch (err) { + const stderr = err.stderr ? err.stderr.toString().trim() : err.message; + console.error('[capture] CIFS mount failed (falling back to S3 streaming):', stderr); + return false; + } +} + +// Best-effort unmount on session stop. Ignores "not mounted". +function unmountGrowingShare() { + if (!GROWING_SMB_MOUNT) return; + try { + if (isMounted(GROWING_PATH)) { + execFileSync('umount', [GROWING_PATH], { stdio: ['ignore', 'ignore', 'pipe'] }); + console.log('[capture] unmounted growing share at', GROWING_PATH); + } + } catch (err) { + const stderr = err.stderr ? err.stderr.toString().trim() : err.message; + console.warn('[capture] growing share unmount failed (ignored):', stderr); + } +} + // ── Codec catalogue ────────────────────────────────────────────────────── // Each entry maps the UI value to a base ffmpeg flag set. Bitrate / framerate // / pix_fmt are layered on top from the per-recorder configuration. @@ -283,7 +348,15 @@ class CaptureManager { // Growing-files: write master to the local SMB share instead of streaming // to S3. Path is relative to the container's GROWING_PATH mount. - const growingPath = GROWING_ENABLED + // + // Approach A: if a CIFS source is configured, mount it now. A mount failure + // is non-fatal — we fall back to S3 streaming so the recording is never + // lost. + let growingActive = GROWING_ENABLED; + if (growingActive && GROWING_SMB_MOUNT) { + if (!mountGrowingShare()) growingActive = false; // fall back to S3 + } + const growingPath = growingActive ? `${GROWING_PATH}/${projectId}/${clipName}.${hiresExt}` : null; if (growingPath) { @@ -455,6 +528,11 @@ class CaptureManager { if (processes.proxy) processes.proxy.kill('SIGINT'); if (processes.hls) { try { processes.hls.kill('SIGINT'); } catch (_) {} } + // Release the CIFS mount (best-effort) once the ffmpeg writers are done with + // it. The promotion worker reads the staged file from the host/S3 side, not + // through this container's mount, so unmounting here is safe. + unmountGrowingShare(); + try { const uploadPromises = [currentSession.uploads.hires]; if (currentSession.uploads.proxy) uploadPromises.push(currentSession.uploads.proxy); diff --git a/services/mam-api/package.json b/services/mam-api/package.json index 04101d1..47a260e 100644 --- a/services/mam-api/package.json +++ b/services/mam-api/package.json @@ -22,7 +22,9 @@ "bullmq": "^5.5.0", "multer": "^1.4.5-lts.1", "uuid": "^9.0.1", - "dotenv": "^16.4.5" + "dotenv": "^16.4.5", + "qrcode": "^1.5.4", + "google-auth-library": "^9.14.0" }, "engines": { "node": ">=22.0.0" diff --git a/services/mam-api/src/auth/authz.js b/services/mam-api/src/auth/authz.js new file mode 100644 index 0000000..22a37a6 --- /dev/null +++ b/services/mam-api/src/auth/authz.js @@ -0,0 +1,90 @@ +// Per-project authorization — the single source of truth for "can this user +// touch this project?". v1 auth answers "are you logged in?"; this answers +// "which projects, and at what level?". +// +// Model (locked with Zac): +// - role 'admin' → global bypass; every project at 'edit'. +// - role 'editor'/'viewer' → scoped to projects granted to them directly +// (project_access subject_type='user') or via a +// group they belong to (subject_type='group'). +// - grant level 'view' → read-only; 'edit' → read-write. +// +// A user's effective level on a project is the MAX of every matching grant +// (direct + each group). 'edit' outranks 'view'. +// +// All functions take an optional `db` (defaults to the shared pool) so tests +// can inject an isolated test pool. + +import defaultPool from '../db/pool.js'; + +const LEVEL_RANK = { view: 1, edit: 2 }; + +export function isAdmin(user) { + return user?.role === 'admin'; +} + +// Returns the higher of two levels (either may be null/undefined). +function maxLevel(a, b) { + const ra = LEVEL_RANK[a] || 0; + const rb = LEVEL_RANK[b] || 0; + if (ra === 0 && rb === 0) return null; + return ra >= rb ? a : b; +} + +// Resolve every project the user can see, with their effective level. +// admin → { all: true, ids: null, levelByProject: null } +// else → { all: false, ids: Set, levelByProject: Map } +export async function accessibleProjectIds(user, db = defaultPool) { + if (isAdmin(user)) return { all: true, ids: null, levelByProject: null }; + + const levelByProject = new Map(); + if (!user?.id) return { all: false, ids: new Set(), levelByProject }; + + const { rows } = await db.query( + `SELECT pa.project_id, pa.level + FROM project_access pa + WHERE (pa.subject_type = 'user' AND pa.subject_id = $1) + OR (pa.subject_type = 'group' AND pa.subject_id IN ( + SELECT group_id FROM user_groups WHERE user_id = $1 + ))`, + [user.id] + ); + + for (const r of rows) { + levelByProject.set(r.project_id, maxLevel(levelByProject.get(r.project_id), r.level)); + } + return { all: false, ids: new Set(levelByProject.keys()), levelByProject }; +} + +// Effective level on a single project: 'edit' | 'view' | null. +export async function projectLevel(user, projectId, db = defaultPool) { + if (isAdmin(user)) return 'edit'; + if (!user?.id || !projectId) return null; + + const { rows } = await db.query( + `SELECT pa.level + FROM project_access pa + WHERE pa.project_id = $1 + AND ( (pa.subject_type = 'user' AND pa.subject_id = $2) + OR (pa.subject_type = 'group' AND pa.subject_id IN ( + SELECT group_id FROM user_groups WHERE user_id = $2 + )) )`, + [projectId, user.id] + ); + + let level = null; + for (const r of rows) level = maxLevel(level, r.level); + return level; +} + +// Throw a 403-shaped error (caught by errorHandler) unless the user has at +// least `need` access on the project. `need` ∈ 'view' | 'edit'. +export async function assertProjectAccess(user, projectId, need = 'view', db = defaultPool) { + if (isAdmin(user)) return; + const have = await projectLevel(user, projectId, db); + if (!have || (LEVEL_RANK[have] || 0) < (LEVEL_RANK[need] || 0)) { + const err = new Error('forbidden'); + err.status = 403; + throw err; + } +} diff --git a/services/mam-api/src/auth/google-oauth.js b/services/mam-api/src/auth/google-oauth.js new file mode 100644 index 0000000..1e53f27 --- /dev/null +++ b/services/mam-api/src/auth/google-oauth.js @@ -0,0 +1,90 @@ +// Google OAuth (OIDC) sign-in helpers. +// +// Entirely config-gated: if GOOGLE_CLIENT_ID / GOOGLE_CLIENT_SECRET / +// OAUTH_REDIRECT_URL aren't set, isConfigured() is false and the routes 404, so +// a deployment without Google SSO behaves exactly as before. google-auth-library +// is imported lazily so the dependency is only required when the feature is on. +// +// Flow: /auth/google redirects to Google's consent screen with a signed `state`; +// /auth/google/callback exchanges the code, verifies the ID token, enforces the +// allowed Workspace domain, and auto-provisions a viewer account on first login. + +const SCOPES = ['openid', 'email', 'profile']; + +export function isConfigured() { + return !!(process.env.GOOGLE_CLIENT_ID + && process.env.GOOGLE_CLIENT_SECRET + && process.env.OAUTH_REDIRECT_URL); +} + +export function allowedDomain() { + return (process.env.GOOGLE_ALLOWED_DOMAIN || '').trim().toLowerCase() || null; +} + +// Lazily build an OAuth2 client (throws a clear error if the dep is missing). +async function makeClient() { + let OAuth2Client; + try { + ({ OAuth2Client } = await import('google-auth-library')); + } catch { + const err = new Error('google-auth-library is not installed'); + err.status = 500; + throw err; + } + return new OAuth2Client({ + clientId: process.env.GOOGLE_CLIENT_ID, + clientSecret: process.env.GOOGLE_CLIENT_SECRET, + redirectUri: process.env.OAUTH_REDIRECT_URL, + }); +} + +// URL of Google's consent screen. `state` is an opaque anti-CSRF token we also +// stash in the session and re-check on callback. +export async function buildAuthUrl(state) { + const client = await makeClient(); + return client.generateAuthUrl({ + access_type: 'online', + scope: SCOPES, + state, + prompt: 'select_account', + // If a Workspace domain is configured, hint Google to scope the picker to it. + ...(allowedDomain() ? { hd: allowedDomain() } : {}), + }); +} + +// Exchange the authorization code and verify the returned ID token. Returns the +// verified { sub, email, name, hd } payload. Throws { status } on any failure. +export async function exchangeAndVerify(code) { + const client = await makeClient(); + const { tokens } = await client.getToken(code); + if (!tokens.id_token) { + const err = new Error('no id_token from Google'); err.status = 401; throw err; + } + const ticket = await client.verifyIdToken({ + idToken: tokens.id_token, + audience: process.env.GOOGLE_CLIENT_ID, + }); + const p = ticket.getPayload(); + if (!p || !p.sub) { + const err = new Error('invalid id_token'); err.status = 401; throw err; + } + // Require an explicitly verified email — a missing/undefined claim is NOT + // treated as verified, since the email drives account linking/provisioning. + if (!p.email || p.email_verified !== true) { + const err = new Error('email not verified'); err.status = 403; throw err; + } + const domain = allowedDomain(); + if (domain) { + // ONLY trust Google's `hd` (hosted-domain) claim — it's present iff the + // account is a member of a Google Workspace domain that Google itself + // has verified. The email-suffix fallback we used to allow let any + // non-Workspace account with a spoof-friendly email through; if a + // GOOGLE_ALLOWED_DOMAIN is set, the operator means "only this Workspace," + // and consumer accounts (no hd) must be rejected. + const hd = (p.hd || '').toLowerCase(); + if (hd !== domain) { + const err = new Error('domain not allowed'); err.status = 403; throw err; + } + } + return { sub: p.sub, email: p.email, name: p.name || p.email, hd: p.hd || null }; +} diff --git a/services/mam-api/src/auth/mfa-tickets.js b/services/mam-api/src/auth/mfa-tickets.js new file mode 100644 index 0000000..9c079e2 --- /dev/null +++ b/services/mam-api/src/auth/mfa-tickets.js @@ -0,0 +1,58 @@ +// Short-lived MFA tickets bridging the two login steps. +// +// When a user with TOTP enabled passes password auth, we don't create a session +// yet — we hand back an opaque ticket. The second request (code or recovery +// code) redeems the ticket to finish login. Tickets are single-use and expire +// fast so a stolen ticket is near-useless. +// +// Tickets are bound to the issuing request's IP and User-Agent (hashed). A +// stolen ticket replayed from a different origin redeems to null. This is +// defense in depth against ticket exfiltration via a logged proxy, browser +// extension, or shoulder-surf; it does not stop an attacker who is on the same +// IP and UA. +// +// In-memory + single-instance, matching the existing login rate-limiter +// (auth/rate-limit.js). Documented limitation: in a multi-instance deployment +// the second step must hit the same node. Acceptable for Dragonflight's +// one-mam-api-per-node shape; revisit if that changes. +import { randomBytes, createHash } from 'node:crypto'; + +const TTL_MS = 5 * 60 * 1000; // 5 minutes to enter a code +const tickets = new Map(); // id -> { userId, ipHash, uaHash, expiresAt } + +function sweep() { + const now = Date.now(); + for (const [id, t] of tickets) if (t.expiresAt <= now) tickets.delete(id); +} + +function hashBinding(value) { + return createHash('sha256').update(String(value || '')).digest('hex'); +} + +export function issueTicket(userId, { ip, userAgent } = {}) { + sweep(); + const id = randomBytes(32).toString('hex'); + tickets.set(id, { + userId, + ipHash: hashBinding(ip), + uaHash: hashBinding(userAgent), + expiresAt: Date.now() + TTL_MS, + }); + return id; +} + +// Redeem (and consume) a ticket. Returns the userId, or null if missing, +// expired, or the binding doesn't match the redeeming request. +export function redeemTicket(id, { ip, userAgent } = {}) { + if (!id) return null; + const t = tickets.get(id); + if (!t) return null; + tickets.delete(id); // single-use — burn even on binding mismatch so a + // wrong-binding probe can't be retried. + if (t.expiresAt <= Date.now()) return null; + // If a caller doesn't supply bindings (e.g. tests), accept — the issue side + // controls whether bindings get recorded. + if (ip !== undefined && t.ipHash !== hashBinding(ip)) return null; + if (userAgent !== undefined && t.uaHash !== hashBinding(userAgent)) return null; + return t.userId; +} diff --git a/services/mam-api/src/auth/totp.js b/services/mam-api/src/auth/totp.js new file mode 100644 index 0000000..1b390b7 --- /dev/null +++ b/services/mam-api/src/auth/totp.js @@ -0,0 +1,118 @@ +// TOTP (RFC 6238) implemented on node:crypto — no runtime dependency. +// +// Why hand-rolled: the algorithm is small and stable, and avoiding a dep keeps +// the auth core auditable. Verified against the RFC 6238 Appendix B test vectors +// in test/auth/totp.test.js. +// +// Defaults match every mainstream authenticator app (Google Authenticator, +// Authy, 1Password): SHA-1, 6 digits, 30-second step. + +import { createHmac, randomBytes, timingSafeEqual } from 'node:crypto'; + +const DIGITS = 6; +const STEP_SECONDS = 30; +const RFC4648_B32 = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ234567'; + +// ── base32 (RFC 4648, no padding) ────────────────────────────────────────── +export function base32Encode(buf) { + let bits = 0, value = 0, out = ''; + for (const byte of buf) { + value = (value << 8) | byte; + bits += 8; + while (bits >= 5) { + out += RFC4648_B32[(value >>> (bits - 5)) & 31]; + bits -= 5; + } + } + if (bits > 0) out += RFC4648_B32[(value << (5 - bits)) & 31]; + return out; +} + +export function base32Decode(str) { + const clean = str.replace(/=+$/,'').toUpperCase().replace(/\s+/g, ''); + let bits = 0, value = 0; + const out = []; + for (const ch of clean) { + const idx = RFC4648_B32.indexOf(ch); + if (idx === -1) continue; // skip stray chars + value = (value << 5) | idx; + bits += 5; + if (bits >= 8) { + out.push((value >>> (bits - 8)) & 0xff); + bits -= 8; + } + } + return Buffer.from(out); +} + +// Generate a new base32 secret (20 random bytes = 160 bits, the RFC-recommended +// SHA-1 key length). +export function generateSecret() { + return base32Encode(randomBytes(20)); +} + +// HOTP for a specific counter (RFC 4226). +function hotp(secretBuf, counter) { + const buf = Buffer.alloc(8); + // 64-bit big-endian counter. + buf.writeUInt32BE(Math.floor(counter / 0x100000000), 0); + buf.writeUInt32BE(counter >>> 0, 4); + const hmac = createHmac('sha1', secretBuf).update(buf).digest(); + const offset = hmac[hmac.length - 1] & 0x0f; + const code = ((hmac[offset] & 0x7f) << 24) + | ((hmac[offset + 1] & 0xff) << 16) + | ((hmac[offset + 2] & 0xff) << 8) + | (hmac[offset + 3] & 0xff); + return String(code % (10 ** DIGITS)).padStart(DIGITS, '0'); +} + +// The TOTP code for a given time (defaults to now). +export function generateToken(base32Secret, atMs = Date.now()) { + const counter = Math.floor(atMs / 1000 / STEP_SECONDS); + return hotp(base32Decode(base32Secret), counter); +} + +// Verify a user-supplied code, allowing ±`window` steps of clock drift +// (default ±1 = 90s total tolerance). Constant-time compare per candidate. +// +// Returns the matched counter on success (so callers can persist it for +// replay protection — RFC 6238 §5.2), or null on failure. Boolean truthiness +// still works for the common case (`if (verifyToken(...))`). +export function verifyToken(base32Secret, token, atMs = Date.now(), window = 1) { + if (!base32Secret || !token) return null; + const cleaned = String(token).replace(/\s+/g, ''); + if (!/^\d{6}$/.test(cleaned)) return null; + const secretBuf = base32Decode(base32Secret); + const counter = Math.floor(atMs / 1000 / STEP_SECONDS); + const want = Buffer.from(cleaned); + for (let w = -window; w <= window; w++) { + const candidate = Buffer.from(hotp(secretBuf, counter + w)); + if (candidate.length === want.length && timingSafeEqual(candidate, want)) return counter + w; + } + return null; +} + +// The otpauth:// URI an authenticator app scans. label/issuer show in the app. +export function otpauthURI(base32Secret, accountName, issuer = 'Dragonflight') { + const label = encodeURIComponent(`${issuer}:${accountName}`); + const params = new URLSearchParams({ + secret: base32Secret, + issuer, + algorithm: 'SHA1', + digits: String(DIGITS), + period: String(STEP_SECONDS), + }); + return `otpauth://totp/${label}?${params.toString()}`; +} + +// Generate N human-friendly one-time recovery codes (raw form). Caller hashes +// them before storage and shows the raw set to the user exactly once. +export function generateRecoveryCodes(n = 10) { + const codes = []; + for (let i = 0; i < n; i++) { + // 10 hex chars in two dash-separated groups, e.g. "a1b2c-3d4e5". + const hex = randomBytes(5).toString('hex'); + codes.push(hex.slice(0, 5) + '-' + hex.slice(5)); + } + return codes; +} diff --git a/services/mam-api/src/db/migrations/026-project-access.sql b/services/mam-api/src/db/migrations/026-project-access.sql new file mode 100644 index 0000000..3e49d4b --- /dev/null +++ b/services/mam-api/src/db/migrations/026-project-access.sql @@ -0,0 +1,30 @@ +-- Migration 026 — per-project access grants (RBAC v2). +-- +-- v1 auth is flat: any logged-in user can do everything. This adds per-project +-- scoping. A grant targets either a user or a group (polymorphic subject) and +-- carries a level: 'view' (read-only) or 'edit' (read-write). Admins bypass all +-- of this in code (authz.js) and need no rows here. +-- +-- subject_id is intentionally NOT a foreign key — it points at either users.id +-- or groups.id depending on subject_type. Rows are cleaned up when the project +-- is deleted (FK cascade). A deleted user/group leaves an orphan row that +-- resolves to nobody (harmless); a later sweep can prune them if desired. + +DO $$ BEGIN + IF NOT EXISTS (SELECT 1 FROM pg_type WHERE typname = 'access_level') THEN + CREATE TYPE access_level AS ENUM ('view', 'edit'); + END IF; +END $$; + +CREATE TABLE IF NOT EXISTS project_access ( + project_id UUID NOT NULL REFERENCES projects ON DELETE CASCADE, + subject_type TEXT NOT NULL CHECK (subject_type IN ('user', 'group')), + subject_id UUID NOT NULL, + level access_level NOT NULL DEFAULT 'view', + granted_by UUID REFERENCES users ON DELETE SET NULL, + granted_at TIMESTAMPTZ DEFAULT NOW(), + PRIMARY KEY (project_id, subject_type, subject_id) +); + +CREATE INDEX IF NOT EXISTS idx_project_access_subject + ON project_access (subject_type, subject_id); diff --git a/services/mam-api/src/db/migrations/027-totp-2fa.sql b/services/mam-api/src/db/migrations/027-totp-2fa.sql new file mode 100644 index 0000000..a4a2dbf --- /dev/null +++ b/services/mam-api/src/db/migrations/027-totp-2fa.sql @@ -0,0 +1,20 @@ +-- Migration 027 — TOTP two-factor auth. +-- +-- totp_secret holds the base32 shared secret once enrollment is confirmed +-- (NULL while disabled or mid-enrollment). totp_enabled flips true only after +-- the user verifies their first code, so a half-finished enrollment never locks +-- anyone out. Recovery codes are one-time bcrypt-hashed fallbacks; used_at marks +-- a code as spent. + +ALTER TABLE users ADD COLUMN IF NOT EXISTS totp_secret TEXT; +ALTER TABLE users ADD COLUMN IF NOT EXISTS totp_enabled BOOLEAN NOT NULL DEFAULT FALSE; + +CREATE TABLE IF NOT EXISTS user_recovery_codes ( + id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), + user_id UUID NOT NULL REFERENCES users ON DELETE CASCADE, + code_hash TEXT NOT NULL, + used_at TIMESTAMPTZ, + created_at TIMESTAMPTZ DEFAULT NOW() +); + +CREATE INDEX IF NOT EXISTS idx_user_recovery_codes_user ON user_recovery_codes(user_id); diff --git a/services/mam-api/src/db/migrations/028-google-oauth.sql b/services/mam-api/src/db/migrations/028-google-oauth.sql new file mode 100644 index 0000000..a7864df --- /dev/null +++ b/services/mam-api/src/db/migrations/028-google-oauth.sql @@ -0,0 +1,13 @@ +-- Migration 028 — Google OAuth (OIDC) sign-in. +-- +-- google_sub is Google's stable subject identifier — the join key for a linked +-- or auto-provisioned account (unique, but NULL for password-only users). +-- email is captured for display + domain checks. password_hash becomes nullable +-- so an OAuth-only account can exist without a local password; such an account +-- simply can't use the password login path until an admin sets one. + +ALTER TABLE users ADD COLUMN IF NOT EXISTS google_sub TEXT; +ALTER TABLE users ADD COLUMN IF NOT EXISTS email TEXT; +ALTER TABLE users ALTER COLUMN password_hash DROP NOT NULL; + +CREATE UNIQUE INDEX IF NOT EXISTS idx_users_google_sub ON users(google_sub) WHERE google_sub IS NOT NULL; diff --git a/services/mam-api/src/db/migrations/029-playout.sql b/services/mam-api/src/db/migrations/029-playout.sql new file mode 100644 index 0000000..541f455 --- /dev/null +++ b/services/mam-api/src/db/migrations/029-playout.sql @@ -0,0 +1,165 @@ +-- Migration 029 — Playout / Master Control (MCR). +-- +-- Adds a broadcast playout subsystem: take library assets, arrange them on a +-- playlist (Phase A) or a wall-clock timeline (Phase B), and play them out +-- continuously to SDI (DeckLink) / NDI / SRT / RTMP via a CasparCG sidecar. +-- +-- This is the mirror of the capture path (input -> ffmpeg -> S3). A channel is +-- placed on a cluster node by capability the same way recorders claim input +-- ports; the engine container is spawned via the same Docker-socket / +-- node-agent orchestration. See docs/superpowers/specs/2026-05-30-playout-mcr-design.md. +-- +-- Tables: +-- playout_channels — a logical output (one channel -> one CasparCG instance -> one target) +-- playout_playlists — an ordered list of items bound to a channel (Phase A) +-- playout_items — one clip on a playlist OR one row on the timeline +-- playout_sidecars — running CasparCG sidecar registry (one per channel; health-checked) +-- playout_schedule — wall-clock day-ahead rows (Phase B; unused in A) +-- playout_as_run — append-only log of what actually played (compliance) + +-- ── Channels ─────────────────────────────────────────────────────────────── +CREATE TABLE IF NOT EXISTS playout_channels ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + name TEXT NOT NULL, + node_id UUID REFERENCES cluster_nodes(id) ON DELETE SET NULL, + output_type TEXT NOT NULL DEFAULT 'srt', + -- output_config is consumer-shape-specific: + -- decklink: { "device_index": 1 } + -- ndi: { "ndi_name": "DRAGONFLIGHT CH1" } + -- srt: { "url": "srt://host:9000", "latency": 200 } + -- rtmp: { "url": "rtmp://host/live", "key": "streamkey" } + output_config JSONB NOT NULL DEFAULT '{}'::jsonb, + -- 1080p59.94 is the house standard (matches capture cadence, streaming-friendly, + -- accepted by current SDI gear). Per-channel override allowed. + video_format TEXT NOT NULL DEFAULT '1080p5994', + status TEXT NOT NULL DEFAULT 'stopped', + container_id TEXT, + -- For remote channels the node-agent reports the reachable host:port of the + -- sidecar HTTP shim; stored here so the API can proxy transport calls. + container_meta JSONB NOT NULL DEFAULT '{}'::jsonb, + error_message TEXT, + -- Failover bookkeeping. Scheduler tick health-checks the sidecar; on N missed + -- checks the channel is re-placed on a healthy node (auto for ndi/srt/rtmp, + -- alert-only for decklink — device-index pinning makes re-placement non-trivial). + restart_count INTEGER NOT NULL DEFAULT 0, + last_restart_at TIMESTAMPTZ, + last_heartbeat_at TIMESTAMPTZ, + -- RBAC scoping: a NULL project_id resolves to admin-only (authz.js), the same + -- convention recorders use for unassigned resources. + project_id UUID REFERENCES projects(id) ON DELETE SET NULL, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + CHECK (output_type IN ('decklink','ndi','srt','rtmp')), + CHECK (status IN ('stopped','starting','running','error')) +); + +CREATE INDEX IF NOT EXISTS idx_playout_channels_node ON playout_channels (node_id); +CREATE INDEX IF NOT EXISTS idx_playout_channels_project ON playout_channels (project_id); + +-- ── Playlists ────────────────────────────────────────────────────────────── +CREATE TABLE IF NOT EXISTS playout_playlists ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + channel_id UUID NOT NULL REFERENCES playout_channels(id) ON DELETE CASCADE, + name TEXT NOT NULL, + loop BOOLEAN NOT NULL DEFAULT FALSE, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW() +); + +CREATE INDEX IF NOT EXISTS idx_playout_playlists_channel ON playout_playlists (channel_id); + +-- ── Items ────────────────────────────────────────────────────────────────── +-- One entry on a playlist (Phase A, ordered by sort_order) OR one entry on the +-- timeline (Phase B, ordered by scheduled_at). in/out points reuse the editor's +-- subclip trim model (seconds). media_status tracks the S3 -> /media staging +-- (see playout-stage worker job); a clip cannot go on air until 'ready'. +CREATE TABLE IF NOT EXISTS playout_items ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + playlist_id UUID REFERENCES playout_playlists(id) ON DELETE CASCADE, + asset_id UUID NOT NULL REFERENCES assets(id) ON DELETE CASCADE, + sort_order INTEGER NOT NULL DEFAULT 0, + scheduled_at TIMESTAMPTZ, + in_point NUMERIC, + out_point NUMERIC, + transition TEXT NOT NULL DEFAULT 'cut', + transition_ms INTEGER NOT NULL DEFAULT 0, + graphics JSONB, + media_status TEXT NOT NULL DEFAULT 'pending', + media_path TEXT, + -- Set when playout-stage has run loudnorm (EBU R128, -23 LUFS / -1 dBTP) on + -- the staged file. Re-stages skip the loudnorm pass when true. + audio_normalized BOOLEAN NOT NULL DEFAULT FALSE, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + CHECK (transition IN ('cut','mix','wipe')), + CHECK (media_status IN ('pending','staging','ready','error')) +); + +CREATE INDEX IF NOT EXISTS idx_playout_items_playlist ON playout_items (playlist_id, sort_order); +CREATE INDEX IF NOT EXISTS idx_playout_items_asset ON playout_items (asset_id); + +-- ── Sidecars ─────────────────────────────────────────────────────────────── +-- Running CasparCG container registry, one row per running channel. The +-- scheduler tick (src/scheduler.js) pings each sidecar's /status endpoint and +-- updates last_heartbeat_at; missed checks trigger the failover path in +-- routes/playout.js. +CREATE TABLE IF NOT EXISTS playout_sidecars ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + channel_id UUID NOT NULL REFERENCES playout_channels(id) ON DELETE CASCADE, + node_id UUID REFERENCES cluster_nodes(id) ON DELETE SET NULL, + container_id TEXT NOT NULL, + sidecar_url TEXT, -- http://host:port for the shim + amcp_port INTEGER, -- in-container AMCP port (default 5250) + status TEXT NOT NULL DEFAULT 'running', + last_heartbeat_at TIMESTAMPTZ, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + CHECK (status IN ('starting','running','error','stopped')) +); + +CREATE UNIQUE INDEX IF NOT EXISTS idx_playout_sidecars_channel ON playout_sidecars (channel_id) + WHERE status IN ('starting','running'); +CREATE INDEX IF NOT EXISTS idx_playout_sidecars_status ON playout_sidecars (status); + +-- ── Schedule (Phase B) ───────────────────────────────────────────────────── +-- Wall-clock day-ahead timeline. The scheduler tick (src/scheduler.js, under +-- the existing PG advisory lock) drives transitions and gap-fill. Unused by the +-- Phase A playlist player but created now so the schema is stable. +CREATE TABLE IF NOT EXISTS playout_schedule ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + channel_id UUID NOT NULL REFERENCES playout_channels(id) ON DELETE CASCADE, + asset_id UUID REFERENCES assets(id) ON DELETE SET NULL, + scheduled_at TIMESTAMPTZ NOT NULL, + in_point NUMERIC, + out_point NUMERIC, + transition TEXT NOT NULL DEFAULT 'cut', + transition_ms INTEGER NOT NULL DEFAULT 0, + is_filler BOOLEAN NOT NULL DEFAULT FALSE, + status TEXT NOT NULL DEFAULT 'scheduled', + media_status TEXT NOT NULL DEFAULT 'pending', + media_path TEXT, + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + CHECK (transition IN ('cut','mix','wipe')), + CHECK (status IN ('scheduled','playing','played','skipped','error')), + CHECK (media_status IN ('pending','staging','ready','error')) +); + +CREATE INDEX IF NOT EXISTS idx_playout_schedule_channel_time ON playout_schedule (channel_id, scheduled_at); +CREATE INDEX IF NOT EXISTS idx_playout_schedule_status ON playout_schedule (status, scheduled_at); + +-- ── As-run log ───────────────────────────────────────────────────────────── +-- Append-only record of what actually went to air. Never updated after insert. +CREATE TABLE IF NOT EXISTS playout_as_run ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + channel_id UUID NOT NULL REFERENCES playout_channels(id) ON DELETE CASCADE, + asset_id UUID REFERENCES assets(id) ON DELETE SET NULL, + item_id UUID, + clip_name TEXT, + started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + ended_at TIMESTAMPTZ, + duration_s NUMERIC, + result TEXT NOT NULL DEFAULT 'played', + CHECK (result IN ('played','skipped','error')) +); + +CREATE INDEX IF NOT EXISTS idx_playout_as_run_channel ON playout_as_run (channel_id, started_at DESC); diff --git a/services/mam-api/src/db/migrations/030-totp-replay.sql b/services/mam-api/src/db/migrations/030-totp-replay.sql new file mode 100644 index 0000000..352ef96 --- /dev/null +++ b/services/mam-api/src/db/migrations/030-totp-replay.sql @@ -0,0 +1,9 @@ +-- Migration 030 — TOTP replay protection. +-- +-- RFC 6238 §5.2 hardening: track the last counter value we accepted for each +-- user and reject codes at counters ≤ the last one. Without this, the same +-- 6-digit code can be submitted N times within its 30s step. Low impact in +-- practice (the code is only valid for ~90s with ±1 drift) but standard. + +ALTER TABLE users + ADD COLUMN IF NOT EXISTS totp_last_counter BIGINT NOT NULL DEFAULT 0; diff --git a/services/mam-api/src/index.js b/services/mam-api/src/index.js index d7cf83f..8831034 100644 --- a/services/mam-api/src/index.js +++ b/services/mam-api/src/index.js @@ -8,7 +8,7 @@ import os from 'node:os'; import { exec } from 'node:child_process'; import pool from './db/pool.js'; import { errorHandler } from './middleware/errors.js'; -import { requireAuth, requireUiHeader } from './middleware/auth.js'; +import { requireAuth, requireUiHeader, requireAdmin } from './middleware/auth.js'; import { loadS3ConfigFromDb } from './s3/client.js'; import authRouter from './routes/auth.js'; @@ -22,6 +22,7 @@ import jobsRouter from './routes/jobs.js'; import captureRouter from './routes/capture.js'; import uploadRouter from './routes/upload.js'; import recordersRouter from './routes/recorders.js'; +import playoutRouter from './routes/playout.js'; import settingsRouter from './routes/settings.js'; import amppRouter from './routes/ampp.js'; import groupsRouter from './routes/groups.js'; @@ -104,7 +105,10 @@ app.get('/health', (_req, res) => res.json({ status: 'ok' })); // ── Auth gate ───────────────────────────────────────────────────────────────── // req.path is relative to the /api/v1 mount, so /auth/login NOT /api/v1/auth/login. -const UNAUTH_PATHS = new Set(['/auth/login', '/auth/setup', '/auth/setup-required']); +const UNAUTH_PATHS = new Set([ + '/auth/login', '/auth/login/totp', '/auth/setup', '/auth/setup-required', + '/auth/google', '/auth/google/callback', '/auth/google/enabled', +]); // node-agent now authenticates /cluster/heartbeat with a bound api_token // (migration 019 + bound_hostname on the token). requireAuth handles the // bearer lookup and sets req.tokenBoundHostname; the heartbeat handler in @@ -117,8 +121,10 @@ app.use('/api/v1', (req, res, next) => { // ── API Routes ──────────────────────────────────────────────────────────────── app.use('/api/v1/auth', authRouter); -app.use('/api/v1/auth/users', usersRouter); -app.use('/api/v1/users', usersRouter); // alias for the existing SPA Users page that calls /api/v1/users; keeps the same auth gate +// User and group administration is admin-only (RBAC v2). The auth gate above +// already established req.user; requireAdmin rejects non-admins with 403. +app.use('/api/v1/auth/users', requireAdmin, usersRouter); +app.use('/api/v1/users', requireAdmin, usersRouter); // alias for the SPA Users page app.use('/api/v1/auth/tokens', requireAuth, tokensRouter); app.use('/api/v1/assets', assetsRouter); app.use('/api/v1/projects', projectsRouter); @@ -127,9 +133,10 @@ app.use('/api/v1/jobs', jobsRouter); app.use('/api/v1/capture', captureRouter); app.use('/api/v1/upload', uploadRouter); app.use('/api/v1/recorders', recordersRouter); +app.use('/api/v1/playout', playoutRouter); app.use('/api/v1/settings', settingsRouter); app.use('/api/v1/ampp', amppRouter); -app.use('/api/v1/groups', groupsRouter); +app.use('/api/v1/groups', requireAdmin, groupsRouter); app.use('/api/v1/sequences', sequencesRouter); app.use('/api/v1/system', systemRouter); app.use('/api/v1/cluster', clusterRouter); diff --git a/services/mam-api/src/middleware/auth.js b/services/mam-api/src/middleware/auth.js index d835c42..21f1d39 100644 --- a/services/mam-api/src/middleware/auth.js +++ b/services/mam-api/src/middleware/auth.js @@ -1,10 +1,29 @@ +import crypto from 'crypto'; import pool from '../db/pool.js'; import { parseBearer, hashToken } from '../auth/tokens.js'; +// In-process service token for the scheduler's loopback self-calls +// (scheduler.js -> /recorders|/playout). The scheduler runs in THIS process, so +// a per-boot random constant needs no env/compose config and is never exposed: +// it only travels over the loopback fetch inside the same process. Multi-replica +// is safe because each replica's scheduler only ever calls 127.0.0.1 (itself), +// matching that replica's token. Requests bearing it are treated as the seeded +// admin (DEV_USER) so RBAC + FK-bearing routes work. +export const INTERNAL_TOKEN = crypto.randomBytes(32).toString('hex'); +const INTERNAL_HEADER = 'x-internal-token'; + +function isInternalCall(req) { + const got = req.headers[INTERNAL_HEADER]; + if (typeof got !== 'string' || got.length !== INTERNAL_TOKEN.length) return false; + return crypto.timingSafeEqual(Buffer.from(got), Buffer.from(INTERNAL_TOKEN)); +} + // Stable UUID matching migration 023's seeded dev user. /** UUID of the seeded dev-mode placeholder. NOT a sentinel for "any unauthenticated user". */ export const DEV_USER_ID = '00000000-0000-4000-8000-000000000000'; -export const DEV_USER = { id: DEV_USER_ID, username: 'dev', display_name: 'Dev (AUTH_ENABLED=false)' }; +// role 'admin' so dev mode (AUTH_ENABLED=false) keeps full access through the +// RBAC v2 gates — matches migration 023's seeded dev row. +export const DEV_USER = { id: DEV_USER_ID, username: 'dev', display_name: 'Dev (AUTH_ENABLED=false)', role: 'admin' }; const ABSOLUTE_MS = 8 * 3600 * 1000; const IDLE_MS = 1 * 3600 * 1000; @@ -18,11 +37,18 @@ async function destroyAnd401(req, res) { async function loadUser(id) { const { rows } = await pool.query( - `SELECT id, username, display_name, role FROM users WHERE id = $1`, [id]); + `SELECT id, username, display_name, role, totp_enabled FROM users WHERE id = $1`, [id]); return rows[0] || null; } export async function requireAuth(req, res, next) { + // Internal loopback self-call (scheduler). Acts as the seeded admin so RBAC + // and FK-bearing routes work, regardless of AUTH_ENABLED. + if (isInternalCall(req)) { + req.user = DEV_USER; + return next(); + } + // Dev mode — attach the seeded dev user so FK-bearing routes work. if (process.env.AUTH_ENABLED !== 'true') { req.user = DEV_USER; @@ -73,6 +99,14 @@ export async function requireAuth(req, res, next) { return res.status(401).json({ error: 'unauthorized' }); } +// Gate a route to admins only. requireAuth must run first (it sets req.user). +// 401 when unauthenticated, 403 when authenticated but not an admin. +export function requireAdmin(req, res, next) { + if (!req.user) return res.status(401).json({ error: 'unauthorized' }); + if (req.user.role !== 'admin') return res.status(403).json({ error: 'forbidden' }); + return next(); +} + // Belt-and-suspenders CSRF: SameSite=Lax already blocks most cross-site // cookie sends, but a custom header that no
can produce hardens // against the edge cases. Applied to mutating verbs only. @@ -88,6 +122,8 @@ const CSRF_EXEMPT_PATHS = new Set(['/cluster/heartbeat']); export function requireUiHeader(req, res, next) { if (!MUTATING.has(req.method)) return next(); + // Internal loopback self-call (scheduler) — not a browser, can't be drive-by'd. + if (isInternalCall(req)) return next(); // Bearer-authed requests (Premiere panel, scripts) are exempt — they're not // browsers and can't be drive-by'd from another origin. if (req.headers.authorization?.toLowerCase().startsWith('bearer ')) return next(); diff --git a/services/mam-api/src/routes/assets.js b/services/mam-api/src/routes/assets.js index d31387e..8e3463d 100644 --- a/services/mam-api/src/routes/assets.js +++ b/services/mam-api/src/routes/assets.js @@ -7,9 +7,36 @@ import pool from '../db/pool.js'; import { getSignedUrlForObject, deleteObject, s3Client, getS3Bucket } from '../s3/client.js'; import { GetObjectCommand, HeadObjectCommand } from '@aws-sdk/client-s3'; import { validateUuid } from '../middleware/errors.js'; +import { assertProjectAccess, accessibleProjectIds } from '../auth/authz.js'; +import { requireAdmin } from '../middleware/auth.js'; const router = express.Router(); -router.param('id', (req, res, next) => validateUuid('id')(req, res, next)); + +// Every /:id asset route is scoped to the asset's project. The param handler +// validates the UUID, resolves the owning project_id, and asserts at least +// 'view' access (the baseline for touching an asset at all). Mutating routes +// additionally assert 'edit' via req.assetProjectId. A missing asset is a clean +// 404 here rather than leaking existence to users without access. +router.param('id', async (req, res, next) => { + validateUuid('id')(req, res, () => {}); + if (res.headersSent) return; + try { + const { rows } = await pool.query('SELECT project_id FROM assets WHERE id = $1', [req.params.id]); + if (rows.length === 0) return res.status(404).json({ error: 'Asset not found' }); + req.assetProjectId = rows[0].project_id; + await assertProjectAccess(req.user, req.assetProjectId, 'view'); + next(); + } catch (err) { next(err); } +}); + +// Route-level guard for mutating /:id endpoints — escalates the param handler's +// 'view' baseline to 'edit'. Reuses req.assetProjectId (already resolved). +async function requireAssetEdit(req, res, next) { + try { + await assertProjectAccess(req.user, req.assetProjectId, 'edit'); + next(); + } catch (err) { next(err); } +} // BullMQ queue connection (mirrors worker/src/index.js) const parseRedisUrl = (url) => { @@ -66,6 +93,15 @@ router.get('/', async (req, res, next) => { const params = []; let paramCount = 1; + // Scope to projects the caller can access (admins are unfiltered). Without + // this, a granted user would see every asset across every project. + const access = await accessibleProjectIds(req.user); + if (!access.all) { + if (access.ids.size === 0) return res.json({ assets: [], total: 0 }); + query += ` AND a.project_id = ANY($${paramCount++}::uuid[])`; + params.push([...access.ids]); + } + // Exclude archived unless explicitly requested — independent of status filter if (include_archived !== 'true') { query += ` AND a.status <> 'archived'`; @@ -132,6 +168,9 @@ router.post('/', async (req, res, next) => { return res.status(400).json({ error: 'projectId and clipName are required' }); } + // Registering an asset writes into a project — require edit access there. + await assertProjectAccess(req.user, projectId, 'edit'); + const durationNum = duration !== undefined && duration !== null ? Number(duration) : null; if (durationNum !== null && !Number.isFinite(durationNum)) { return res.status(400).json({ error: 'duration must be a finite number (seconds)' }); @@ -220,8 +259,8 @@ router.post('/', async (req, res, next) => { } }); -// POST /cleanup-live -router.post('/cleanup-live', async (req, res, next) => { +// POST /cleanup-live — cross-project maintenance, admin only. +router.post('/cleanup-live', requireAdmin, async (req, res, next) => { try { const maxAgeHours = Math.max(1, parseInt(req.query.max_age_hours || '4', 10)); const result = await pool.query( @@ -234,8 +273,8 @@ router.post('/cleanup-live', async (req, res, next) => { } catch (err) { next(err); } }); -// POST /cleanup-live-orphans -router.post('/cleanup-live-orphans', async (_req, res, next) => { +// POST /cleanup-live-orphans — cross-project maintenance, admin only. +router.post('/cleanup-live-orphans', requireAdmin, async (_req, res, next) => { try { const liveRoot = process.env.LIVE_DIR || '/live'; let entries; @@ -277,10 +316,22 @@ router.get('/:id', async (req, res, next) => { }); // PATCH /:id -router.patch('/:id', async (req, res, next) => { +router.patch('/:id', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; const { display_name, tags, notes, bin_id } = req.body; + + // bin_id must reference a bin in the asset's OWN project — otherwise an + // editor in project A could stuff their asset into project B's bin tree. + // Null/empty clears the bin, which is always allowed. + if (bin_id) { + const bin = await pool.query('SELECT project_id FROM bins WHERE id = $1', [bin_id]); + if (bin.rows.length === 0) return res.status(400).json({ error: 'bin_id not found' }); + if (bin.rows[0].project_id !== req.assetProjectId) { + return res.status(400).json({ error: 'bin_id belongs to a different project' }); + } + } + const updates = [], params = []; let paramCount = 1; if (display_name !== undefined) { updates.push(`display_name = $${paramCount++}`); params.push(display_name); } @@ -299,13 +350,32 @@ router.patch('/:id', async (req, res, next) => { }); // POST /:id/copy -router.post('/:id/copy', async (req, res, next) => { +router.post('/:id/copy', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; const { binId, projectId } = req.body; const r = await pool.query('SELECT * FROM assets WHERE id = $1', [id]); if (r.rows.length === 0) return res.status(404).json({ error: 'Asset not found' }); const src = r.rows[0]; + + // Destination project defaults to source's. If the caller overrides it, + // assert edit on the target — without this, an editor in project A could + // clone any asset they can see into project B with no grant on B. + const destProjectId = projectId || src.project_id; + if (projectId && projectId !== src.project_id) { + await assertProjectAccess(req.user, destProjectId, 'edit'); + } + // Destination bin (if any) must belong to the destination project — same + // class of bug as the PATCH bin_id hole. + const destBinId = binId === undefined ? src.bin_id : (binId || null); + if (destBinId) { + const bin = await pool.query('SELECT project_id FROM bins WHERE id = $1', [destBinId]); + if (bin.rows.length === 0) return res.status(400).json({ error: 'binId not found' }); + if (bin.rows[0].project_id !== destProjectId) { + return res.status(400).json({ error: 'binId belongs to a different project than the destination' }); + } + } + const newId = uuidv4(); // Bug #60: null out proxy_s3_key and thumbnail_s3_key on the copy to avoid // sharing S3 objects with the source. Set status to 'processing' so the copy @@ -320,8 +390,8 @@ router.post('/:id/copy', async (req, res, next) => { $1,$2,$3,$4,$5,$6,$7,$8,NULL,NULL,$9,$10,$11,$12,$13,$14,$15,$16,NOW(),NOW() ) RETURNING *`, [ - newId, projectId || src.project_id, - binId === undefined ? src.bin_id : (binId || null), + newId, destProjectId, + destBinId, src.filename, src.display_name, 'processing', src.media_type, src.original_s3_key, src.codec, src.resolution, src.fps, src.duration_ms, src.start_tc, @@ -346,7 +416,7 @@ router.post('/:id/copy', async (req, res, next) => { }); // POST /:id/mark-empty -router.post('/:id/mark-empty', async (req, res, next) => { +router.post('/:id/mark-empty', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; // Bug #66: first check the asset exists and what status it is in @@ -384,7 +454,7 @@ router.post('/:id/mark-empty', async (req, res, next) => { // the existing live row -> 409 -> asset stuck 'live', no jobs. Finalising by id // flips it out of 'live', records duration + S3 keys, and kicks off the // proxy -> thumbnail -> filmstrip job chain. -router.post('/:id/finalize', async (req, res, next) => { +router.post('/:id/finalize', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; const { hiresKey, proxyKey, duration } = req.body; @@ -436,7 +506,7 @@ router.post('/:id/finalize', async (req, res, next) => { }); // POST /:id/generate-proxy -router.post('/:id/generate-proxy', async (req, res, next) => { +router.post('/:id/generate-proxy', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; const r = await pool.query('SELECT * FROM assets WHERE id = $1', [id]); @@ -452,8 +522,8 @@ router.post('/:id/generate-proxy', async (req, res, next) => { } catch (err) { next(err); } }); -// POST /backfill-proxies -router.post('/backfill-proxies', async (_req, res, next) => { +// POST /backfill-proxies — cross-project maintenance, admin only. +router.post('/backfill-proxies', requireAdmin, async (_req, res, next) => { try { const targets = await pool.query( `SELECT id, original_s3_key FROM assets @@ -477,7 +547,7 @@ router.post('/backfill-proxies', async (_req, res, next) => { // POST /:id/reprocess?type=proxy|thumbnail|filmstrip // Force-requeue a processing job regardless of current asset status. -router.post('/:id/reprocess', async (req, res, next) => { +router.post('/:id/reprocess', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; const type = req.query.type || 'proxy'; @@ -528,7 +598,7 @@ router.get('/:id/filmstrip', async (req, res, next) => { }); // POST /:id/retry -router.post('/:id/retry', async (req, res, next) => { +router.post('/:id/retry', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; const r = await pool.query('SELECT * FROM assets WHERE id = $1', [id]); @@ -547,7 +617,7 @@ router.post('/:id/retry', async (req, res, next) => { }); // DELETE /:id -router.delete('/:id', async (req, res, next) => { +router.delete('/:id', requireAssetEdit, async (req, res, next) => { try { const { id } = req.params; const { hard } = req.query; @@ -595,10 +665,19 @@ router.get('/:id/stream', async (req, res, next) => { if (r.rows.length === 0) return res.status(404).json({ error: 'Asset not found' }); const a = r.rows[0]; if (a.status === 'live') return res.json({ url: `/live/${a.id}/index.m3u8`, type: 'hls', live: true }); - // Prefer the HLS rendition for recorded assets — whole-file segment GETs - // avoid the RustFS ranged-GET stitching the MP4 /video path has to do. + // `url` is the directly-downloadable MP4 proxy; `hls_url` is the HLS + // rendition for in-browser playback (whole-file segment GETs avoid the + // RustFS ranged-GET stitching the MP4 path needs). The Premiere plugin + // downloads `url` to a file and imports it, so `url` must NOT be the + // .m3u8 playlist — Premiere can't import a playlist ("unsupported + // compression type"). The web player prefers `hls_url` when present. if (a.hls_s3_key) { - return res.json({ url: `/api/v1/assets/${id}/hls/playlist.m3u8`, type: 'hls', source: 'proxy' }); + return res.json({ + url: `/api/v1/assets/${id}/video`, + type: 'mp4', + source: a.proxy_s3_key ? 'proxy' : 'original', + hls_url: `/api/v1/assets/${id}/hls/playlist.m3u8`, + }); } const VIDEO_EXTS = ['.mp4', '.mov', '.mxf', '.ts', '.m4v', '.mkv', '.avi', '.webm']; const key = a.proxy_s3_key || @@ -655,11 +734,14 @@ router.get('/:id/live-path', async (req, res, next) => { if (a.rows.length === 0) return res.status(404).json({ error: 'Asset not found' }); const asset = a.rows[0]; if (asset.status !== 'live') return res.status(409).json({ error: 'Asset is not currently growing', status: asset.status }); - const s = await pool.query(`SELECT key, value FROM settings WHERE key IN ('growing_enabled','growing_smb_url')`); + // Growing-files mode is now per-recorder (recorders.growing_enabled), so we + // no longer gate on the removed global `growing_enabled` setting. A + // status='live' asset already proves a growing recorder is producing this + // file; we only need the editor-facing SMB URL to build the UNC path. + const s = await pool.query(`SELECT key, value FROM settings WHERE key = 'growing_smb_url'`); const cfg = {}; for (const { key, value } of s.rows) cfg[key] = value; - if (cfg.growing_enabled !== 'true') return res.status(409).json({ error: 'Growing-files mode is disabled' }); - if (!cfg.growing_smb_url) return res.status(409).json({ error: 'No SMB URL configured — set growing_smb_url in Settings' }); + if (!cfg.growing_smb_url) return res.status(409).json({ error: 'No SMB URL configured — set the editor SMB URL in Settings → Storage' }); const rec = await pool.query( `SELECT recording_container FROM recorders WHERE current_session_id = $1 ORDER BY updated_at DESC LIMIT 1`, [asset.id] @@ -899,6 +981,15 @@ router.post('/batch-trim', async (req, res, next) => { return res.status(400).json({ error: 'Each clip must have assetId, filename, sourceInFrames, sourceOutFrames, timelineInFrames, timelineOutFrames, and a non-negative integer trackIndex' }); } } + // Authorize every source asset's project (edit) before queuing any work. + const trimAssetIds = [...new Set(clips.map(c => c.assetId))]; + const owning = await pool.query('SELECT id, project_id FROM assets WHERE id = ANY($1::uuid[])', [trimAssetIds]); + const projById = new Map(owning.rows.map(r => [r.id, r.project_id])); + for (const aid of trimAssetIds) { + const pid = projById.get(aid); + if (!pid) return res.status(404).json({ error: 'Asset not found: ' + aid }); + await assertProjectAccess(req.user, pid, 'edit'); + } const jobId = uuidv4(); const expiresAt = new Date(Date.now() + 24 * 60 * 60 * 1000); await pool.query( diff --git a/services/mam-api/src/routes/auth.js b/services/mam-api/src/routes/auth.js index b2dc7b3..00e0456 100644 --- a/services/mam-api/src/routes/auth.js +++ b/services/mam-api/src/routes/auth.js @@ -3,6 +3,14 @@ import pool from '../db/pool.js'; import { DEV_USER_ID, requireAuth } from '../middleware/auth.js'; import { hashPassword, comparePassword } from '../auth/passwords.js'; import { ipBackoff } from '../auth/rate-limit.js'; +import { + generateSecret, verifyToken, otpauthURI, generateRecoveryCodes, +} from '../auth/totp.js'; +import { issueTicket, redeemTicket } from '../auth/mfa-tickets.js'; +import { + isConfigured as googleConfigured, buildAuthUrl, exchangeAndVerify, +} from '../auth/google-oauth.js'; +import { randomBytes } from 'node:crypto'; const DUMMY_PASSWORD_HASH = '$2b$12$gSeC58PregWedNFK/8Q61OephUo.JJ7EUs0LCTdnJV5AzCS5qQH7K'; @@ -76,7 +84,7 @@ router.post('/login', async (req, res, next) => { } const { rows } = await pool.query( - `SELECT id, username, display_name, password_hash FROM users WHERE username = $1 AND id <> $2`, + `SELECT id, username, display_name, password_hash, totp_enabled FROM users WHERE username = $1 AND id <> $2`, [username.trim(), DEV_USER_ID] ); if (rows.length === 0) { @@ -93,21 +101,123 @@ router.post('/login', async (req, res, next) => { return res.status(401).json({ error: 'invalid credentials' }); } - req.session.user_id = user.id; - req.session.first_seen_at = Date.now(); - req.session.last_seen_at = Date.now(); - // The critical line — wait for the row to land in `sessions` before responding. - // Without this, the SPA's next request races the store write, hits 401, and - // the prior bounce-to-login logic produced an infinite loop. - await new Promise((resolve, reject) => req.session.save(err => err ? reject(err) : resolve())); + // Second factor: if TOTP is enabled, don't create a session yet. Hand back + // a short-lived ticket the client redeems via /login/totp with a code. + // Crucially: do NOT clear the per-IP failure counter here. If we did, each + // /login retry would reset the backoff and let an attacker brute the 6-digit + // TOTP space (10^6) with no per-attempt delay. The counter is cleared + // inside establishSession() once MFA has actually passed. + if (user.totp_enabled) { + return res.json({ + mfa_required: true, + ticket: issueTicket(user.id, { ip, userAgent: req.get('user-agent') }), + }); + } - pool.query(`UPDATE users SET last_login_at = NOW() WHERE id = $1`, [user.id]) - .catch(err => console.error('[auth] last_login_at update failed:', err.message)); - - ipBackoff.recordSuccess(ip); - res.json({ user: { id: user.id, username: user.username, display_name: user.display_name } }); } catch (err) { next(err); } + await establishSession(req, user, ip); + res.json({ user: { id: user.id, username: user.username, display_name: user.display_name } }); + } catch (err) { next(err); } }); +// Write the session and wait for it to persist before responding. Extracted so +// both the password-only and the MFA-completion paths share one implementation. +// Clears the per-IP failure counter only here — after every required factor has +// actually been proven (password [+ TOTP if enabled, or OAuth + TOTP]). +async function establishSession(req, user, ip) { + req.session.user_id = user.id; + req.session.first_seen_at = Date.now(); + req.session.last_seen_at = Date.now(); + // The critical line — wait for the row to land in `sessions` before responding. + // Without this, the SPA's next request races the store write, hits 401, and + // the prior bounce-to-login logic produced an infinite loop. + await new Promise((resolve, reject) => req.session.save(err => err ? reject(err) : resolve())); + if (ip) ipBackoff.recordSuccess(ip); + pool.query(`UPDATE users SET last_login_at = NOW() WHERE id = $1`, [user.id]) + .catch(err => console.error('[auth] last_login_at update failed:', err.message)); +} + +// POST /api/v1/auth/login/totp { ticket?, code } — second login step. `code` is +// either a 6-digit TOTP or a one-time recovery code. The ticket comes from the +// request body (password-login path) or req.session.mfa_ticket (Google path). +router.post('/login/totp', async (req, res, next) => { + try { + const ip = req.ip || req.socket?.remoteAddress || 'unknown'; + // Rate-limit the second factor with the same per-IP backoff as /login so + // the 6-digit code space can't be hammered. + const delay = ipBackoff.delayMs(ip); + if (delay > 0) await new Promise(r => setTimeout(r, delay)); + + const { ticket: bodyTicket, code } = req.body || {}; + const ticket = bodyTicket || req.session?.mfa_ticket; + if (req.session?.mfa_ticket) delete req.session.mfa_ticket; + // Bound to the issuing request's IP + UA — replays from a different origin + // redeem to null. See mfa-tickets.js for the binding model. + const userId = redeemTicket(ticket, { ip, userAgent: req.get('user-agent') }); + if (!userId) { + ipBackoff.recordFailure(ip); + return res.status(401).json({ error: 'invalid or expired ticket' }); + } + if (!code) return res.status(400).json({ error: 'code required' }); + + const { rows } = await pool.query( + `SELECT id, username, display_name, totp_secret, totp_enabled, totp_last_counter + FROM users WHERE id = $1`, [userId]); + const user = rows[0]; + if (!user || !user.totp_enabled || !user.totp_secret) { + return res.status(401).json({ error: 'invalid credentials' }); + } + + // verifyToken returns the matched counter on success. Reject codes at + // counters ≤ totp_last_counter to prevent replay within the same step. + // The CAS-style UPDATE makes this race-free under concurrent submissions. + const matchedCounter = verifyToken(user.totp_secret, code); + let ok = false; + if (matchedCounter !== null) { + const lastCounter = BigInt(user.totp_last_counter || 0); + if (BigInt(matchedCounter) > lastCounter) { + const upd = await pool.query( + `UPDATE users SET totp_last_counter = $1 + WHERE id = $2 AND totp_last_counter < $1`, + [String(matchedCounter), user.id] + ); + ok = upd.rowCount === 1; + } + // matchedCounter ≤ last → silent replay; falls through to recovery-code + // path which also fails → 401. Same UX as a wrong code, no info leak. + } + if (!ok) ok = await consumeRecoveryCode(user.id, code); + if (!ok) { + ipBackoff.recordFailure(ip); + // The ticket was single-use; the client must restart from /login. + return res.status(401).json({ error: 'invalid code' }); + } + + // recordSuccess is called by establishSession once the session lands — + // that's the first moment we know every required factor has passed. + await establishSession(req, user, ip); + res.json({ user: { id: user.id, username: user.username, display_name: user.display_name } }); + } catch (err) { next(err); } +}); + +// Check a recovery code against the user's unused codes; mark it spent on match. +// The marking is atomic (UPDATE ... WHERE used_at IS NULL with a rowCount check) +// so two concurrent redemptions of the same code can't both succeed. +async function consumeRecoveryCode(userId, code) { + const cleaned = String(code).trim().toLowerCase(); + if (!/^[0-9a-f]{5}-[0-9a-f]{5}$/.test(cleaned)) return false; + const { rows } = await pool.query( + `SELECT id, code_hash FROM user_recovery_codes WHERE user_id = $1 AND used_at IS NULL`, [userId]); + for (const row of rows) { + if (await comparePassword(cleaned, row.code_hash)) { + const upd = await pool.query( + `UPDATE user_recovery_codes SET used_at = NOW() WHERE id = $1 AND used_at IS NULL`, [row.id]); + // Lost the race if another request already consumed it. + return upd.rowCount === 1; + } + } + return false; +} + // POST /api/v1/auth/logout — destroys the session and clears the cookie. router.post('/logout', (req, res) => { if (!req.session) return res.status(204).end(); @@ -125,6 +235,7 @@ router.get('/me', requireAuth, (req, res) => { username: req.user.username, display_name: req.user.display_name, role: req.user.role, + totp_enabled: !!req.user.totp_enabled, }); }); @@ -149,5 +260,202 @@ router.post('/password', requireAuth, async (req, res, next) => { } catch (err) { next(err); } }); +// ── TOTP enrollment (all require an active session) ───────────────────────── + +// POST /api/v1/auth/totp/setup — begin enrollment. Generates a fresh secret, +// stores it (but leaves totp_enabled=false), and returns the otpauth URI + the +// base32 secret for manual entry. Enrollment isn't active until /enable +// confirms a code, so a started-but-abandoned setup never locks the user out. +router.post('/totp/setup', requireAuth, async (req, res, next) => { + try { + const { rows } = await pool.query(`SELECT totp_enabled FROM users WHERE id = $1`, [req.user.id]); + if (rows[0]?.totp_enabled) return res.status(409).json({ error: 'TOTP already enabled' }); + + const secret = generateSecret(); + await pool.query(`UPDATE users SET totp_secret = $1 WHERE id = $2`, [secret, req.user.id]); + const uri = otpauthURI(secret, req.user.username || 'user'); + + // QR rendering is optional — the otpauth URI + manual secret are sufficient + // to enroll. Render a data-URL QR only if the optional `qrcode` dep is + // present, so a missing dependency degrades instead of 500-ing. + let qr = null; + try { + const QRCode = (await import('qrcode')).default; + qr = await QRCode.toDataURL(uri); + } catch { /* qrcode not installed — client falls back to manual entry */ } + + res.json({ secret, otpauth_uri: uri, qr }); + } catch (err) { next(err); } +}); + +// POST /api/v1/auth/totp/enable { code } — confirm enrollment with a code from +// the authenticator. On success, flips totp_enabled and returns one-time +// recovery codes (shown exactly once). +router.post('/totp/enable', requireAuth, async (req, res, next) => { + try { + const { code } = req.body || {}; + if (!code) return badRequest(res, 'code required'); + + const { rows } = await pool.query( + `SELECT totp_secret, totp_enabled FROM users WHERE id = $1`, [req.user.id]); + const row = rows[0]; + if (!row?.totp_secret) return badRequest(res, 'start setup first'); + if (row.totp_enabled) return res.status(409).json({ error: 'TOTP already enabled' }); + const enrollCounter = verifyToken(row.totp_secret, code); + if (enrollCounter === null) return badRequest(res, 'incorrect code'); + + const recovery = generateRecoveryCodes(10); + const hashes = await Promise.all(recovery.map(c => hashPassword(c))); + // Enable + seed totp_last_counter to the enrollment code's counter so the + // same code can't be reused on first login. Replace any stale recovery + // codes atomically. + const client = await pool.connect(); + try { + await client.query('BEGIN'); + await client.query( + `UPDATE users SET totp_enabled = TRUE, totp_last_counter = $2 WHERE id = $1`, + [req.user.id, String(enrollCounter)] + ); + await client.query(`DELETE FROM user_recovery_codes WHERE user_id = $1`, [req.user.id]); + for (const h of hashes) { + await client.query( + `INSERT INTO user_recovery_codes (user_id, code_hash) VALUES ($1, $2)`, [req.user.id, h]); + } + await client.query('COMMIT'); + } catch (e) { + await client.query('ROLLBACK').catch(() => {}); + throw e; + } finally { client.release(); } + + res.json({ enabled: true, recovery_codes: recovery }); + } catch (err) { next(err); } +}); + +// POST /api/v1/auth/totp/disable { password } — turn off 2FA. Requires the +// account password as a confirmation so a hijacked live session can't silently +// strip the second factor. +router.post('/totp/disable', requireAuth, async (req, res, next) => { + try { + const { password } = req.body || {}; + if (!password) return badRequest(res, 'password required'); + const { rows } = await pool.query(`SELECT password_hash FROM users WHERE id = $1`, [req.user.id]); + if (!rows.length) return res.status(401).json({ error: 'unauthorized' }); + if (!(await comparePassword(password, rows[0].password_hash))) { + return badRequest(res, 'incorrect password'); + } + await pool.query( + `UPDATE users SET totp_enabled = FALSE, totp_secret = NULL, totp_last_counter = 0 WHERE id = $1`, + [req.user.id] + ); + await pool.query(`DELETE FROM user_recovery_codes WHERE user_id = $1`, [req.user.id]); + res.status(204).end(); + } catch (err) { next(err); } +}); + +// ── Google OAuth (OIDC) sign-in ───────────────────────────────────────────── +// All Google routes are config-gated: without GOOGLE_CLIENT_ID/SECRET and +// OAUTH_REDIRECT_URL they 404, so a deployment without SSO is unaffected. + +// GET /api/v1/auth/google/enabled — cheap, no auth. Lets the login screen decide +// whether to render the "Sign in with Google" button. +router.get('/google/enabled', (_req, res) => { + res.json({ enabled: googleConfigured() }); +}); + +// GET /api/v1/auth/google — kick off the OAuth dance. Stores an anti-CSRF state +// in the session and redirects to Google's consent screen. +router.get('/google', async (req, res, next) => { + try { + if (!googleConfigured()) return res.status(404).json({ error: 'google sign-in not configured' }); + const state = randomBytes(16).toString('hex'); + req.session.oauth_state = state; + await new Promise((resolve, reject) => req.session.save(err => err ? reject(err) : resolve())); + res.redirect(await buildAuthUrl(state)); + } catch (err) { next(err); } +}); + +// GET /api/v1/auth/google/callback — Google redirects back here with ?code&state. +// Verifies the ID token, enforces the allowed domain, auto-provisions a viewer +// on first login, establishes the session, then redirects to the SPA. +router.get('/google/callback', async (req, res, next) => { + try { + if (!googleConfigured()) return res.status(404).json({ error: 'google sign-in not configured' }); + const { code, state } = req.query; + const expected = req.session.oauth_state; + delete req.session.oauth_state; + if (!code || !state || !expected || state !== expected) { + return res.status(400).json({ error: 'invalid oauth state' }); + } + + const profile = await exchangeAndVerify(code); + const user = await resolveGoogleUser(profile); + + // If this account has TOTP enabled, Google is only the FIRST factor — route + // through the same second-factor step as password login. The ticket lives in + // the session (not the URL) and the SPA prompts for the code. + if (user.totp_enabled) { + const ip = req.ip || req.socket?.remoteAddress || 'unknown'; + req.session.mfa_ticket = issueTicket(user.id, { + ip, + userAgent: req.get('user-agent'), + }); + await new Promise((resolve, reject) => req.session.save(err => err ? reject(err) : resolve())); + return res.redirect('/?mfa=1'); + } + + const ip = req.ip || req.socket?.remoteAddress || 'unknown'; + await establishSession(req, user, ip); + + // Redirect to the SPA root; AuthGate will re-check /auth/me and render the app. + res.redirect('/'); + } catch (err) { + // Surface a friendly message on the login screen rather than a raw 500. + if (err.status === 403) return res.redirect('/?auth_error=domain'); + if (err.status === 401) return res.redirect('/?auth_error=google'); + next(err); + } +}); + +// Map a verified Google profile to a Dragonflight user row. +// +// Resolution order: +// 1. Existing link by google_sub → that user. +// 2. Otherwise auto-provision a fresh 'viewer'. +// +// We deliberately do NOT auto-link to an existing account by matching email: +// that would let anyone who controls a Google address with the same email sign +// in as a pre-existing local (possibly admin) account, bypassing its password +// and TOTP. Linking an existing account to Google is an explicit, authenticated +// action (a future "connect Google" under Settings), not something a login does. +async function resolveGoogleUser(profile) { + const found = await pool.query( + `SELECT id, username, display_name, totp_enabled FROM users WHERE google_sub = $1`, [profile.sub]); + if (found.rows.length) return found.rows[0]; + + const base = (profile.email.split('@')[0] || 'user').replace(/[^a-z0-9._-]/gi, '').toLowerCase() || 'user'; + let username = base, n = 1; + while ((await pool.query(`SELECT 1 FROM users WHERE username = $1`, [username])).rows.length) { + username = base + (++n); + } + + try { + const ins = await pool.query( + `INSERT INTO users (username, password_hash, display_name, role, email, google_sub) + VALUES ($1, NULL, $2, 'viewer', $3, $4) + RETURNING id, username, display_name, totp_enabled`, + [username, profile.name, profile.email, profile.sub]); + return ins.rows[0]; + } catch (err) { + // Concurrent first-login race: the unique google_sub index rejected our + // INSERT because a sibling request just created the row. Re-resolve. + if (err.code === '23505') { + const retry = await pool.query( + `SELECT id, username, display_name, totp_enabled FROM users WHERE google_sub = $1`, [profile.sub]); + if (retry.rows.length) return retry.rows[0]; + } + throw err; + } +} + export default router; -export { realUserCount }; +export { realUserCount, resolveGoogleUser, consumeRecoveryCode }; diff --git a/services/mam-api/src/routes/bins.js b/services/mam-api/src/routes/bins.js index 1fc91db..646d1eb 100644 --- a/services/mam-api/src/routes/bins.js +++ b/services/mam-api/src/routes/bins.js @@ -1,25 +1,60 @@ import express from 'express'; import pool from '../db/pool.js'; import { validateUuid } from '../middleware/errors.js'; +import { assertProjectAccess, accessibleProjectIds } from '../auth/authz.js'; import { v4 as uuidv4 } from 'uuid'; const router = express.Router(); -router.param('id', (req, res, next) => validateUuid('id')(req, res, next)); -// GET / - List bins. Filter by project_id when supplied; otherwise return -// every bin across every project so the Library / asset-context-menu can -// present a global "move to bin" picker. +// Resolve the owning project for a /:id bin, assert 'view' baseline, stash the +// project_id for mutating routes to escalate to 'edit'. +router.param('id', async (req, res, next) => { + validateUuid('id')(req, res, () => {}); + if (res.headersSent) return; + try { + const { rows } = await pool.query('SELECT project_id FROM bins WHERE id = $1', [req.params.id]); + if (rows.length === 0) return res.status(404).json({ error: 'Bin not found' }); + req.binProjectId = rows[0].project_id; + await assertProjectAccess(req.user, req.binProjectId, 'view'); + next(); + } catch (err) { next(err); } +}); + +async function requireBinEdit(req, res, next) { + try { + await assertProjectAccess(req.user, req.binProjectId, 'edit'); + next(); + } catch (err) { next(err); } +} + +// GET / - List bins. When project_id is supplied, scope to it (after an access +// check); otherwise return bins across every project the caller can access. router.get('/', async (req, res, next) => { try { const { project_id } = req.query; - const params = []; - let where = ''; if (project_id) { - where = 'WHERE b.project_id = $1'; - params.push(project_id); + await assertProjectAccess(req.user, project_id, 'view'); + const result = await pool.query( + `SELECT b.*, p.name AS project_name, + (SELECT COUNT(*)::int FROM assets a WHERE a.bin_id = b.id) AS asset_count + FROM bins b + LEFT JOIN projects p ON p.id = b.project_id + WHERE b.project_id = $1 + ORDER BY b.created_at DESC`, + [project_id] + ); + return res.json(result.rows); } + const access = await accessibleProjectIds(req.user); + let where = ''; + const params = []; + if (!access.all) { + if (access.ids.size === 0) return res.json([]); + where = 'WHERE b.project_id = ANY($1::uuid[])'; + params.push([...access.ids]); + } const result = await pool.query( `SELECT b.*, p.name AS project_name, (SELECT COUNT(*)::int FROM assets a WHERE a.bin_id = b.id) AS asset_count @@ -29,14 +64,13 @@ router.get('/', async (req, res, next) => { ORDER BY b.created_at DESC`, params ); - res.json(result.rows); } catch (err) { next(err); } }); -// POST / - Create bin +// POST / - Create bin (requires edit on the target project). router.post('/', async (req, res, next) => { try { const { project_id, name, parent_id } = req.body; @@ -44,6 +78,7 @@ router.post('/', async (req, res, next) => { if (!project_id || !name) { return res.status(400).json({ error: 'project_id and name are required' }); } + await assertProjectAccess(req.user, project_id, 'edit'); const id = uuidv4(); @@ -61,7 +96,7 @@ router.post('/', async (req, res, next) => { }); // PATCH /:id - Update bin -router.patch('/:id', async (req, res, next) => { +router.patch('/:id', requireBinEdit, async (req, res, next) => { try { const { id } = req.params; const { name, parent_id } = req.body; @@ -107,7 +142,7 @@ router.patch('/:id', async (req, res, next) => { }); // DELETE /:id - Delete bin -router.delete('/:id', async (req, res, next) => { +router.delete('/:id', requireBinEdit, async (req, res, next) => { try { const { id } = req.params; @@ -126,8 +161,8 @@ router.delete('/:id', async (req, res, next) => { } }); -// POST /:id/assets - Add asset to bin -router.post('/:id/assets', async (req, res, next) => { +// POST /:id/assets - Add asset to bin (requires edit on the bin's project). +router.post('/:id/assets', requireBinEdit, async (req, res, next) => { try { const { id } = req.params; const { asset_id } = req.body; @@ -136,10 +171,13 @@ router.post('/:id/assets', async (req, res, next) => { return res.status(400).json({ error: 'asset_id is required' }); } - // Verify bin exists - const binCheck = await pool.query('SELECT id FROM bins WHERE id = $1', [id]); - if (binCheck.rows.length === 0) { - return res.status(404).json({ error: 'Bin not found' }); + // Asset must live in the bin's own project. Without this, an editor in + // project A (where the bin lives) could pull an asset from project B (no + // grant) into A's bin tree, exposing it in A's views. + const a = await pool.query('SELECT project_id FROM assets WHERE id = $1', [asset_id]); + if (a.rows.length === 0) return res.status(404).json({ error: 'Asset not found' }); + if (a.rows[0].project_id !== req.binProjectId) { + return res.status(400).json({ error: 'asset belongs to a different project than the bin' }); } // Update asset's bin_id @@ -158,8 +196,8 @@ router.post('/:id/assets', async (req, res, next) => { } }); -// DELETE /:id/assets/:assetId - Remove asset from bin -router.delete('/:id/assets/:assetId', async (req, res, next) => { +// DELETE /:id/assets/:assetId - Remove asset from bin (requires edit). +router.delete('/:id/assets/:assetId', requireBinEdit, async (req, res, next) => { try { const { id, assetId } = req.params; diff --git a/services/mam-api/src/routes/capture.js b/services/mam-api/src/routes/capture.js index 2f14630..f699d83 100644 --- a/services/mam-api/src/routes/capture.js +++ b/services/mam-api/src/routes/capture.js @@ -1,3 +1,7 @@ +// authz: intentionally any-logged-in (no per-project scoping). This is a thin +// proxy to shared capture hardware with no project_id of its own; the resulting +// asset is scoped when it's registered via the /assets route. Gated by the +// global requireAuth in index.js, like the rest of /api/v1. import express from 'express'; const router = express.Router(); diff --git a/services/mam-api/src/routes/cluster.js b/services/mam-api/src/routes/cluster.js index 174ca49..d13e64f 100644 --- a/services/mam-api/src/routes/cluster.js +++ b/services/mam-api/src/routes/cluster.js @@ -1,9 +1,23 @@ import express from 'express'; import http from 'http'; import pool from '../db/pool.js'; +import { requireAdmin } from '../middleware/auth.js'; const router = express.Router(); +// GET /onboard-info – admin-only. Supplies the Add Node wizard with the bits it +// needs to build a `curl … | bash` onboarding command: the primary API URL the +// remote node-agent should heartbeat to, the raw URL of onboard-node.sh, and +// the deploy branch. apiUrl is a best guess the UI lets the operator edit. +router.get('/onboard-info', requireAdmin, (req, res) => { + const branch = process.env.DEPLOY_BRANCH || 'main'; + const apiUrl = process.env.PUBLIC_API_URL + || `${req.protocol}://${req.hostname}:${process.env.API_PORT || 47432}`; + const scriptUrl = + `https://forge.wilddragon.net/zgaetano/wild-dragon/raw/branch/${branch}/deploy/onboard-node.sh`; + res.json({ apiUrl, scriptUrl, branch }); +}); + // If the agent reported Docker's default bridge IP (172.17.x) but the request // itself came from a real LAN address, prefer the request source IP instead. // We only check 172.17.x — the default docker0 bridge — not the full RFC1918 diff --git a/services/mam-api/src/routes/comments.js b/services/mam-api/src/routes/comments.js index 6701155..5a0cae1 100644 --- a/services/mam-api/src/routes/comments.js +++ b/services/mam-api/src/routes/comments.js @@ -5,9 +5,23 @@ import express from 'express'; import pool from '../db/pool.js'; +import { assertProjectAccess } from '../auth/authz.js'; const router = express.Router({ mergeParams: true }); +// Scope every comment route to the parent asset's project: resolve project_id +// via the asset, then require 'view' to read and 'edit' to write. A non-UUID or +// unknown asset is a clean 404 before any access decision leaks its existence. +const MUTATING = new Set(['POST', 'PUT', 'PATCH', 'DELETE']); +router.use(async (req, res, next) => { + try { + const { rows } = await pool.query('SELECT project_id FROM assets WHERE id = $1', [req.params.assetId]); + if (rows.length === 0) return res.status(404).json({ error: 'Asset not found' }); + await assertProjectAccess(req.user, rows[0].project_id, MUTATING.has(req.method) ? 'edit' : 'view'); + next(); + } catch (err) { next(err); } +}); + function rowToJson(r) { return { id: r.id, @@ -49,8 +63,9 @@ router.post('/', async (req, res, next) => { if (!body || !String(body).trim()) { return res.status(400).json({ error: 'body is required' }); } - // Best-effort author lookup — pull from the session if AUTH_ENABLED is on. - const userId = req.session?.userId || null; + // Author is the authenticated user (requireAuth sets req.user for both + // session and bearer auth, and the dev user when AUTH_ENABLED=false). + const userId = req.user?.id || null; const ins = await pool.query( `INSERT INTO asset_comments (asset_id, user_id, body, frame_ms) diff --git a/services/mam-api/src/routes/imports.js b/services/mam-api/src/routes/imports.js index 7540e46..eeab8c3 100644 --- a/services/mam-api/src/routes/imports.js +++ b/services/mam-api/src/routes/imports.js @@ -10,6 +10,7 @@ import express from 'express'; import { Queue } from 'bullmq'; import { v4 as uuidv4 } from 'uuid'; import pool from '../db/pool.js'; +import { assertProjectAccess } from '../auth/authz.js'; const router = express.Router(); @@ -60,6 +61,8 @@ router.post('/youtube', async (req, res, next) => { if (projCheck.rows.length === 0) { return res.status(404).json({ error: 'Project not found' }); } + // Importing writes an asset into the project — require edit access. + await assertProjectAccess(req.user, projectId, 'edit'); const assetId = uuidv4(); diff --git a/services/mam-api/src/routes/jobs.js b/services/mam-api/src/routes/jobs.js index 1079b19..5608cb1 100644 --- a/services/mam-api/src/routes/jobs.js +++ b/services/mam-api/src/routes/jobs.js @@ -1,6 +1,7 @@ import express from 'express'; import pool from '../db/pool.js'; import { Queue } from 'bullmq'; +import { assertProjectAccess } from '../auth/authz.js'; const router = express.Router(); // Note: jobs use BullMQ id format ":" (e.g. "conform:42"), @@ -21,20 +22,22 @@ const parseRedisUrl = (url) => { const redisConn = parseRedisUrl(process.env.REDIS_URL || 'redis://queue:6379'); -const proxyQueue = new Queue('proxy', { connection: redisConn }); -const thumbnailQueue = new Queue('thumbnail', { connection: redisConn }); -const filmstripQueue = new Queue('filmstrip', { connection: redisConn }); -const conformQueue = new Queue('conform', { connection: redisConn }); -const importQueue = new Queue('import', { connection: redisConn }); -const trimQueue = new Queue('trim', { connection: redisConn }); +const proxyQueue = new Queue('proxy', { connection: redisConn }); +const thumbnailQueue = new Queue('thumbnail', { connection: redisConn }); +const filmstripQueue = new Queue('filmstrip', { connection: redisConn }); +const conformQueue = new Queue('conform', { connection: redisConn }); +const importQueue = new Queue('import', { connection: redisConn }); +const trimQueue = new Queue('trim', { connection: redisConn }); +const playoutStageQueue = new Queue('playout-stage', { connection: redisConn }); const QUEUES = [ - { queue: proxyQueue, type: 'proxy' }, - { queue: thumbnailQueue, type: 'thumbnail' }, - { queue: filmstripQueue, type: 'filmstrip' }, - { queue: conformQueue, type: 'conform' }, - { queue: importQueue, type: 'import' }, - { queue: trimQueue, type: 'trim' }, + { queue: proxyQueue, type: 'proxy' }, + { queue: thumbnailQueue, type: 'thumbnail' }, + { queue: filmstripQueue, type: 'filmstrip' }, + { queue: conformQueue, type: 'conform' }, + { queue: importQueue, type: 'import' }, + { queue: trimQueue, type: 'trim' }, + { queue: playoutStageQueue, type: 'playout-stage' }, ]; // BullMQ state → API status mapping @@ -324,6 +327,10 @@ router.post('/conform', async (req, res, next) => { }); } + // Conform writes back into a project — require edit on that project. Without + // this, any logged-in user could enqueue conform jobs targeting any project. + await assertProjectAccess(req.user, project_id, 'edit'); + const bullJob = await conformQueue.add('conform-task', { edl, projectId: project_id, diff --git a/services/mam-api/src/routes/playout.js b/services/mam-api/src/routes/playout.js new file mode 100644 index 0000000..168075e --- /dev/null +++ b/services/mam-api/src/routes/playout.js @@ -0,0 +1,735 @@ +// Playout / Master Control routes. +// +// Control plane for the CasparCG-backed playout subsystem. Channels are placed +// on cluster nodes and their engine containers spawned via the same Docker-socket +// / node-agent path recorders use; the channel's transport (play / pause / skip) +// is proxied through to the sidecar's HTTP shim, which drives CasparCG over AMCP. +// +// RBAC: every channel carries a project_id (NULL = admin-only, the recorder +// convention). List routes filter by accessible projects; mutating routes assert +// 'edit'. See docs/superpowers/specs/2026-05-30-playout-mcr-design.md. + +import express from 'express'; +import http from 'http'; +import { readFile } from 'fs/promises'; +import { Queue } from 'bullmq'; +import pool from '../db/pool.js'; +import { validateUuid } from '../middleware/errors.js'; +import { + assertProjectAccess, accessibleProjectIds, isAdmin, +} from '../auth/authz.js'; + +const router = express.Router(); + +// ── BullMQ: media staging queue (S3 -> /media volume) ──────────────────────── +const parseRedisUrl = (url) => { + const parsed = new URL(url); + return { host: parsed.hostname, port: parseInt(parsed.port, 10) }; +}; +const stageQueue = new Queue('playout-stage', { + connection: parseRedisUrl(process.env.REDIS_URL || 'redis://queue:6379'), +}); + +// ── Sidecar orchestration (mirrors recorders.js) ───────────────────────────── +const PLAYOUT_SIDECAR_IMAGE = process.env.PLAYOUT_IMAGE || 'wild-dragon-playout:latest'; + +function dockerApi(method, path, body = null) { + return new Promise((resolve, reject) => { + const options = { + socketPath: '/var/run/docker.sock', + path: `/v1.43${path}`, + method, + headers: { 'Content-Type': 'application/json' }, + }; + const req = http.request(options, (res) => { + let data = ''; + res.on('data', (chunk) => (data += chunk)); + res.on('end', () => { + try { resolve({ status: res.statusCode, data: data ? JSON.parse(data) : {} }); } + catch { resolve({ status: res.statusCode, data }); } + }); + }); + req.on('error', reject); + req.setTimeout(10000, () => req.destroy(new Error('Docker API timeout after 10s'))); + if (body) req.write(JSON.stringify(body)); + req.end(); + }); +} + +async function resolveNodeTarget(nodeId) { + if (!nodeId) return { remote: false }; + const r = await pool.query( + 'SELECT hostname, ip_address, api_url FROM cluster_nodes WHERE id = $1', [nodeId] + ); + if (r.rows.length === 0) return { remote: false }; + const node = r.rows[0]; + const localHostname = process.env.NODE_HOSTNAME || ''; + if (!node.api_url || node.hostname === localHostname) return { remote: false }; + return { remote: true, apiUrl: node.api_url, ip: node.ip_address }; +} + +// The sidecar shim listens on this port inside the container. The mam-api talks +// to it by container alias on the shared docker network (local) or via the +// node-agent's returned host:port (remote). +const SIDECAR_HTTP_PORT = 3002; + +function channelAlias(id) { return `playout-${id}`; } + +// Resolve the base URL the API uses to reach a running channel's sidecar shim. +// Local: the docker-network alias. Remote: the node-agent reported the host the +// container is published on (stored in container_meta.sidecar_url). +function sidecarBaseUrl(channel) { + if (channel.container_meta && channel.container_meta.sidecar_url) { + return channel.container_meta.sidecar_url; + } + return `http://${channelAlias(channel.id)}:${SIDECAR_HTTP_PORT}`; +} + +async function callSidecar(channel, path, method = 'POST', body = null) { + const url = `${sidecarBaseUrl(channel)}${path}`; + const res = await fetch(url, { + method, + headers: { 'Content-Type': 'application/json' }, + body: body ? JSON.stringify(body) : undefined, + signal: AbortSignal.timeout(20000), + }); + if (!res.ok) { + const text = await res.text().catch(() => ''); + throw new Error(`sidecar ${method} ${path} -> HTTP ${res.status}: ${text.slice(0, 200)}`); + } + return res.json().catch(() => ({})); +} + +// ── Serialization ──────────────────────────────────────────────────────────── +function channelToJson(r) { + return { + id: r.id, + name: r.name, + node_id: r.node_id, + output_type: r.output_type, + output_config: r.output_config, + video_format: r.video_format, + status: r.status, + container_id: r.container_id, + error_message: r.error_message, + project_id: r.project_id, + restart_count: r.restart_count ?? 0, + last_restart_at: r.last_restart_at, + last_heartbeat_at: r.last_heartbeat_at, + created_at: r.created_at, + updated_at: r.updated_at, + }; +} + +const OUTPUT_TYPES = new Set(['decklink', 'ndi', 'srt', 'rtmp']); + +// ── Param resolver: scope every /:id route to the channel's project ────────── +router.param('id', async (req, res, next) => { + validateUuid('id')(req, res, () => {}); + if (res.headersSent) return; + try { + const { rows } = await pool.query( + 'SELECT * FROM playout_channels WHERE id = $1', [req.params.id] + ); + if (rows.length === 0) return res.status(404).json({ error: 'Channel not found' }); + req.channel = rows[0]; + await assertProjectAccess(req.user, req.channel.project_id, 'view'); + next(); + } catch (err) { next(err); } +}); + +async function requireChannelEdit(req, res, next) { + try { await assertProjectAccess(req.user, req.channel.project_id, 'edit'); next(); } + catch (err) { next(err); } +} + +// ── Channels ───────────────────────────────────────────────────────────────── + +// GET /playout/channels — list (filtered to accessible projects) +router.get('/channels', async (req, res, next) => { + try { + let rows; + if (isAdmin(req.user)) { + ({ rows } = await pool.query('SELECT * FROM playout_channels ORDER BY created_at DESC')); + } else { + const ids = await accessibleProjectIds(req.user); + if (ids.length === 0) return res.json([]); + ({ rows } = await pool.query( + 'SELECT * FROM playout_channels WHERE project_id = ANY($1) ORDER BY created_at DESC', [ids] + )); + } + res.json(rows.map(channelToJson)); + } catch (err) { next(err); } +}); + +// POST /playout/channels — create +router.post('/channels', async (req, res, next) => { + try { + const { name, node_id = null, output_type = 'srt', output_config = {}, + video_format = '1080p5994', project_id = null } = req.body || {}; + if (!name || typeof name !== 'string') { + return res.status(400).json({ error: 'name is required' }); + } + if (!OUTPUT_TYPES.has(output_type)) { + return res.status(400).json({ error: `output_type must be one of: ${[...OUTPUT_TYPES].join(', ')}` }); + } + // Creating a project-scoped channel requires edit on that project; a + // null-project (admin-only) channel requires admin. + if (project_id) await assertProjectAccess(req.user, project_id, 'edit'); + else if (!isAdmin(req.user)) return res.status(403).json({ error: 'admin required for unassigned channel' }); + + const { rows } = await pool.query( + `INSERT INTO playout_channels (name, node_id, output_type, output_config, video_format, project_id) + VALUES ($1,$2,$3,$4,$5,$6) RETURNING *`, + [name.trim(), node_id, output_type, JSON.stringify(output_config), video_format, project_id] + ); + res.status(201).json(channelToJson(rows[0])); + } catch (err) { next(err); } +}); + +// PATCH /playout/channels/:id — update config (only while stopped) +router.patch('/channels/:id', requireChannelEdit, async (req, res, next) => { + try { + if (req.channel.status === 'running') { + return res.status(409).json({ error: 'Cannot edit a running channel — stop it first' }); + } + const allowed = ['name', 'node_id', 'output_type', 'output_config', 'video_format', 'project_id']; + const sets = []; + const vals = []; + let i = 1; + for (const k of allowed) { + if (req.body[k] === undefined) continue; + if (k === 'output_type' && !OUTPUT_TYPES.has(req.body[k])) { + return res.status(400).json({ error: 'invalid output_type' }); + } + sets.push(`${k} = $${i++}`); + vals.push(k === 'output_config' ? JSON.stringify(req.body[k]) : req.body[k]); + } + if (sets.length === 0) return res.json(channelToJson(req.channel)); + vals.push(req.channel.id); + const { rows } = await pool.query( + `UPDATE playout_channels SET ${sets.join(', ')}, updated_at = NOW() WHERE id = $${i} RETURNING *`, vals + ); + res.json(channelToJson(rows[0])); + } catch (err) { next(err); } +}); + +// DELETE /playout/channels/:id +router.delete('/channels/:id', requireChannelEdit, async (req, res, next) => { + try { + if (req.channel.status === 'running') { + return res.status(409).json({ error: 'Stop the channel before deleting it' }); + } + await pool.query('DELETE FROM playout_channels WHERE id = $1', [req.channel.id]); + res.json({ deleted: true }); + } catch (err) { next(err); } +}); + +// ── Port-contention guard (DeckLink) ───────────────────────────────────────── +// A DeckLink device on a node is exclusive: an active recorder OR another active +// channel on the same node+index blocks a new SDI channel. NDI/SRT/RTMP have no +// hardware contention. +async function assertDeckLinkFree(channel) { + if (channel.output_type !== 'decklink') return; + const idx = (channel.output_config && channel.output_config.device_index) || 1; + // Another running channel on the same node + device index? + const chan = await pool.query( + `SELECT id FROM playout_channels + WHERE id <> $1 AND node_id IS NOT DISTINCT FROM $2 AND status = 'running' + AND output_type = 'decklink' AND (output_config->>'device_index')::int = $3`, + [channel.id, channel.node_id, idx] + ); + if (chan.rows.length > 0) { + throw Object.assign(new Error(`DeckLink device ${idx} already in use by another channel on this node`), { httpStatus: 409 }); + } + // An active recorder using the same device index on the same node? + const rec = await pool.query( + `SELECT id FROM recorders + WHERE node_id IS NOT DISTINCT FROM $1 AND device_index = $2 + AND status = 'recording' AND source_type = 'sdi'`, + [channel.node_id, idx] + ); + if (rec.rows.length > 0) { + throw Object.assign(new Error(`DeckLink device ${idx} is in use by a recorder on this node`), { httpStatus: 409 }); + } +} + +// Spawn the CasparCG sidecar for a channel and flip it to 'running'. Shared by +// the /start route and the scheduler failover path (restartChannel) so neither +// duplicates the docker/node-agent orchestration. Caller is responsible for the +// pre-flight guards (status check, DeckLink contention) appropriate to its path. +// +// On any spawn failure the channel is left status='error' with a message and an +// Error carrying { httpStatus } is thrown. On success returns the updated row. +async function spawnChannelSidecar(channel) { + await pool.query('UPDATE playout_channels SET status = $1, error_message = NULL WHERE id = $2', ['starting', channel.id]); + + const env = [ + `OUTPUT_TYPE=${channel.output_type}`, + `OUTPUT_CONFIG=${JSON.stringify(channel.output_config || {})}`, + `VIDEO_FORMAT=${channel.video_format}`, + `PORT=${SIDECAR_HTTP_PORT}`, + // Drives the HLS preview path (/media/live//index.m3u8) and + // the per-channel resource naming inside the sidecar. + `CHANNEL_ID=${channel.id}`, + ]; + + const { remote: isRemote, apiUrl: targetNodeApiUrl } = await resolveNodeTarget(channel.node_id); + const dockerNetwork = process.env.DOCKER_NETWORK || 'wild-dragon_wild-dragon'; + let containerId; + let containerMeta = {}; + + if (isRemote) { + const sidecarRes = await fetch(`${targetNodeApiUrl}/sidecar/start`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + image: PLAYOUT_SIDECAR_IMAGE, env, + capturePort: SIDECAR_HTTP_PORT, + sourceType: channel.output_type, + useGpu: false, + publishHttp: true, + }), + signal: AbortSignal.timeout(20000), + }); + if (!sidecarRes.ok) { + const details = await sidecarRes.json().catch(() => ({})); + console.error('[playout] remote sidecar start failed:', JSON.stringify(details)); + await pool.query('UPDATE playout_channels SET status = $1, error_message = $2 WHERE id = $3', + ['error', 'remote node failed to start sidecar', channel.id]); + throw Object.assign(new Error('Remote node failed to start sidecar'), { httpStatus: 502 }); + } + const data = await sidecarRes.json(); + containerId = data.containerId; + // node-agent returns the reachable host:port the shim is published on. + if (data.sidecarUrl || data.host) { + containerMeta.sidecar_url = data.sidecarUrl || `http://${data.host}:${SIDECAR_HTTP_PORT}`; + } + } else { + const alias = channelAlias(channel.id); + const hostBinds = ['/mnt/NVME/MAM/wild-dragon-media:/media']; + if (channel.output_type === 'decklink') hostBinds.push('/dev/blackmagic:/dev/blackmagic'); + + const containerConfig = { + Image: PLAYOUT_SIDECAR_IMAGE, + Env: env, + HostConfig: { + Privileged: true, + NetworkMode: dockerNetwork, + Binds: hostBinds, + }, + NetworkingConfig: { EndpointsConfig: { [dockerNetwork]: { Aliases: [alias] } } }, + Hostname: alias, + }; + const createRes = await dockerApi('POST', '/containers/create', containerConfig); + if (createRes.status !== 201) { + console.error('[playout] container create failed:', JSON.stringify(createRes.data)); + await pool.query('UPDATE playout_channels SET status = $1, error_message = $2 WHERE id = $3', + ['error', 'container create failed', channel.id]); + throw Object.assign(new Error('Failed to create container'), { httpStatus: 500 }); + } + containerId = createRes.data.Id; + const startRes = await dockerApi('POST', `/containers/${containerId}/start`); + if (startRes.status !== 204) { + console.error('[playout] container start failed:', JSON.stringify(startRes.data)); + await dockerApi('DELETE', `/containers/${containerId}?force=true`).catch(() => {}); + await pool.query('UPDATE playout_channels SET status = $1, error_message = $2 WHERE id = $3', + ['error', 'container start failed', channel.id]); + throw Object.assign(new Error('Failed to start container'), { httpStatus: 500 }); + } + } + + // Set last_heartbeat_at = NOW() so the scheduler health tick treats this + // channel as freshly alive. Without this, last_heartbeat_at starts as NULL + // (epoch=0), and the very first tick sees ageMs >> TIMEOUT_MS and triggers + // failover immediately — before the sidecar has had a chance to respond. + const { rows } = await pool.query( + `UPDATE playout_channels + SET status = 'running', container_id = $1, container_meta = $2, + last_heartbeat_at = NOW(), updated_at = NOW() + WHERE id = $3 RETURNING *`, + [containerId, JSON.stringify(containerMeta), channel.id] + ); + return rows[0]; +} + +// POST /playout/channels/:id/start — spawn the CasparCG sidecar + bring up output +router.post('/channels/:id/start', requireChannelEdit, async (req, res, next) => { + try { + const channel = req.channel; + if (channel.status === 'running' || channel.status === 'starting') { + return res.status(409).json({ error: `Channel already ${channel.status}` }); + } + await assertDeckLinkFree(channel); + const row = await spawnChannelSidecar(channel); + res.json(channelToJson(row)); + } catch (err) { + if (err.httpStatus) return res.status(err.httpStatus).json({ error: err.message }); + next(err); + } +}); + +// POST /playout/channels/:id/stop — tear down the sidecar +router.post('/channels/:id/stop', requireChannelEdit, async (req, res, next) => { + try { + const channel = req.channel; + if (channel.container_id) { + const { remote: isRemote, apiUrl } = await resolveNodeTarget(channel.node_id); + if (isRemote) { + await fetch(`${apiUrl}/sidecar/stop`, { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ containerId: channel.container_id }), + signal: AbortSignal.timeout(20000), + }).catch((e) => console.error('[playout] remote stop failed:', e.message)); + } else { + await dockerApi('POST', `/containers/${channel.container_id}/stop?t=10`).catch(() => {}); + await dockerApi('DELETE', `/containers/${channel.container_id}?force=true`).catch(() => {}); + } + } + const { rows } = await pool.query( + `UPDATE playout_channels SET status = 'stopped', container_id = NULL, updated_at = NOW() + WHERE id = $1 RETURNING *`, [channel.id] + ); + res.json(channelToJson(rows[0])); + } catch (err) { next(err); } +}); + +// GET /playout/channels/:id/status — live engine status (proxied to sidecar) +router.get('/channels/:id/status', async (req, res, next) => { + try { + if (req.channel.status !== 'running') { + return res.json({ running: false, status: req.channel.status }); + } + const out = await callSidecar(req.channel, '/status', 'GET'); + res.json({ running: true, status: req.channel.status, engine: out }); + } catch (err) { + res.json({ running: true, status: req.channel.status, engine: null, engine_error: err.message }); + } +}); + +// GET /playout/channels/:id/hls/index.m3u8 — the live preview playlist, served +// through the API (not the static /media/live path) so it bypasses the public +// reverse proxy's static cache. That proxy caches the .m3u8 by path with a +// multi-second TTL and ignores the origin's no-store, so hls.js's ~1s reloads +// always got a STALE playlist ("MISSED" forever → monitor stayed black). The +// /api/ path is not proxy-cached (the status poll updates fine), so this always +// returns the fresh live edge. Segment (.ts) lines are rewritten to absolute +// /media/live// URLs so they still load from the static path (immutable, +// caching them is fine). mam-api shares the same /media volume the sidecars +// write to. +const HLS_MEDIA_DIR = process.env.PLAYOUT_MEDIA_DIR || '/media'; +router.get('/channels/:id/hls/index.m3u8', async (req, res, next) => { + try { + const cid = req.channel.id; + let body; + try { + body = await readFile(`${HLS_MEDIA_DIR}/live/${cid}/index.m3u8`, 'utf8'); + } catch (e) { + return res.status(404).json({ error: 'No live preview for this channel yet' }); + } + // Rewrite bare segment names to absolute static URLs. + const rewritten = body + .split('\n') + .map((line) => (/^[^#].*\.ts\s*$/.test(line) ? `/media/live/${cid}/${line.trim()}` : line)) + .join('\n'); + res.setHeader('Content-Type', 'application/vnd.apple.mpegurl'); + res.setHeader('Cache-Control', 'no-store, no-cache, must-revalidate'); + res.setHeader('Pragma', 'no-cache'); + res.send(rewritten); + } catch (err) { next(err); } +}); + +// ── Transport ──────────────────────────────────────────────────────────────── +async function transport(req, res, action, body = null) { + if (req.channel.status !== 'running') { + return res.status(409).json({ error: 'Channel is not running' }); + } + try { res.json(await callSidecar(req.channel, action, 'POST', body)); } + catch (err) { res.status(502).json({ error: err.message }); } +} + +// POST /playout/channels/:id/play — resolve the channel's playlist, stage-check, +// and hand the engine the ordered list of ready clips. +router.post('/channels/:id/play', requireChannelEdit, async (req, res, next) => { + try { + if (req.channel.status !== 'running') { + return res.status(409).json({ error: 'Start the channel before playing' }); + } + const { playlist_id } = req.body || {}; + if (!playlist_id) return res.status(400).json({ error: 'playlist_id is required' }); + + const pl = await pool.query('SELECT * FROM playout_playlists WHERE id = $1 AND channel_id = $2', + [playlist_id, req.channel.id]); + if (pl.rows.length === 0) return res.status(404).json({ error: 'Playlist not found for this channel' }); + + const items = await pool.query( + `SELECT i.*, a.filename AS clip_name, a.duration_ms AS asset_duration_ms + FROM playout_items i JOIN assets a ON a.id = i.asset_id + WHERE i.playlist_id = $1 ORDER BY i.sort_order ASC`, [playlist_id]); + + const notReady = items.rows.filter((i) => i.media_status !== 'ready' || !i.media_path); + if (notReady.length > 0) { + return res.status(409).json({ + error: 'Some items are not staged yet', + pending: notReady.map((i) => i.id), + }); + } + + const payload = { + loop: pl.rows[0].loop, + items: items.rows.map((i) => ({ + id: i.id, asset_id: i.asset_id, media_path: i.media_path, + in_point: i.in_point ? Number(i.in_point) : null, + out_point: i.out_point ? Number(i.out_point) : null, + transition: i.transition, transition_ms: i.transition_ms, + clip_name: i.clip_name, + asset_duration_ms: i.asset_duration_ms != null ? Number(i.asset_duration_ms) : null, + })), + }; + // callSidecar throws on network/timeout errors. Return 502 (not 409) so + // the UI and operators know it's a gateway problem, not a state conflict. + let out; + try { + out = await callSidecar(req.channel, '/playlist/load', 'POST', payload); + } catch (err) { + return res.status(502).json({ error: 'Sidecar unreachable: ' + err.message }); + } + res.json(out); + } catch (err) { next(err); } +}); + +router.post('/channels/:id/pause', requireChannelEdit, (req, res) => transport(req, res, '/transport/pause')); +router.post('/channels/:id/resume', requireChannelEdit, (req, res) => transport(req, res, '/transport/resume')); +router.post('/channels/:id/skip', requireChannelEdit, (req, res) => transport(req, res, '/transport/skip')); +router.post('/channels/:id/stop-playback', requireChannelEdit, (req, res) => transport(req, res, '/channel/stop')); + +// GET /playout/channels/:id/asrun — as-run log +router.get('/channels/:id/asrun', async (req, res, next) => { + try { + const { rows } = await pool.query( + `SELECT * FROM playout_as_run WHERE channel_id = $1 ORDER BY started_at DESC LIMIT 500`, + [req.channel.id]); + res.json(rows); + } catch (err) { next(err); } +}); + +// ── Playlists ──────────────────────────────────────────────────────────────── +async function loadChannelForBody(req, res, next) { + // For playlist/item routes the channel is referenced indirectly; resolve it + // and assert edit. Used on create/mutate routes that carry channel_id. + const channelId = req.body.channel_id || req.query.channel_id; + if (!channelId) return res.status(400).json({ error: 'channel_id is required' }); + try { + const { rows } = await pool.query('SELECT * FROM playout_channels WHERE id = $1', [channelId]); + if (rows.length === 0) return res.status(404).json({ error: 'Channel not found' }); + req.channel = rows[0]; + await assertProjectAccess(req.user, req.channel.project_id, 'edit'); + next(); + } catch (err) { next(err); } +} + +// GET /playout/playlists?channel_id=... +router.get('/playlists', async (req, res, next) => { + try { + const channelId = req.query.channel_id; + if (!channelId) return res.status(400).json({ error: 'channel_id is required' }); + const ch = await pool.query('SELECT project_id FROM playout_channels WHERE id = $1', [channelId]); + if (ch.rows.length === 0) return res.status(404).json({ error: 'Channel not found' }); + await assertProjectAccess(req.user, ch.rows[0].project_id, 'view'); + const { rows } = await pool.query( + 'SELECT * FROM playout_playlists WHERE channel_id = $1 ORDER BY created_at ASC', [channelId]); + res.json(rows); + } catch (err) { next(err); } +}); + +// POST /playout/playlists +router.post('/playlists', loadChannelForBody, async (req, res, next) => { + try { + const { name, loop = false } = req.body || {}; + if (!name) return res.status(400).json({ error: 'name is required' }); + const { rows } = await pool.query( + 'INSERT INTO playout_playlists (channel_id, name, loop) VALUES ($1,$2,$3) RETURNING *', + [req.channel.id, name.trim(), !!loop]); + res.status(201).json(rows[0]); + } catch (err) { next(err); } +}); + +// GET /playout/playlists/:plid/items +router.get('/playlists/:plid/items', validateUuid('plid'), async (req, res, next) => { + try { + const pl = await pool.query( + `SELECT p.*, c.project_id FROM playout_playlists p + JOIN playout_channels c ON c.id = p.channel_id WHERE p.id = $1`, [req.params.plid]); + if (pl.rows.length === 0) return res.status(404).json({ error: 'Playlist not found' }); + await assertProjectAccess(req.user, pl.rows[0].project_id, 'view'); + const { rows } = await pool.query( + `SELECT i.*, a.filename AS clip_name, a.duration_ms AS asset_duration_ms + FROM playout_items i JOIN assets a ON a.id = i.asset_id + WHERE i.playlist_id = $1 ORDER BY i.sort_order ASC`, [req.params.plid]); + res.json(rows); + } catch (err) { next(err); } +}); + +// Helper: load a playlist + assert edit on its channel's project. +async function loadPlaylistEdit(plid, user) { + const pl = await pool.query( + `SELECT p.*, c.project_id FROM playout_playlists p + JOIN playout_channels c ON c.id = p.channel_id WHERE p.id = $1`, [plid]); + if (pl.rows.length === 0) { throw Object.assign(new Error('Playlist not found'), { httpStatus: 404 }); } + await assertProjectAccess(user, pl.rows[0].project_id, 'edit'); + return pl.rows[0]; +} + +// POST /playout/playlists/:plid/items — add an asset to a playlist +router.post('/playlists/:plid/items', validateUuid('plid'), async (req, res, next) => { + try { + await loadPlaylistEdit(req.params.plid, req.user); + const { asset_id, in_point = null, out_point = null, + transition = 'cut', transition_ms = 0 } = req.body || {}; + if (!asset_id) return res.status(400).json({ error: 'asset_id is required' }); + + // Append at the end of the playlist. + const ord = await pool.query( + 'SELECT COALESCE(MAX(sort_order), -1) + 1 AS next FROM playout_items WHERE playlist_id = $1', + [req.params.plid]); + const { rows } = await pool.query( + `INSERT INTO playout_items (playlist_id, asset_id, sort_order, in_point, out_point, transition, transition_ms) + VALUES ($1,$2,$3,$4,$5,$6,$7) RETURNING *`, + [req.params.plid, asset_id, ord.rows[0].next, in_point, out_point, transition, transition_ms]); + + // Kick staging immediately so the clip is air-ready by the time the operator + // hits play. + await stageQueue.add('stage', { itemId: rows[0].id, assetId: asset_id }).catch((e) => + console.error('[playout] failed to enqueue stage job:', e.message)); + + res.status(201).json(rows[0]); + } catch (err) { + if (err.httpStatus) return res.status(err.httpStatus).json({ error: err.message }); + next(err); + } +}); + +// PUT /playout/playlists/:plid/reorder — body { order: [itemId, itemId, ...] } +router.put('/playlists/:plid/reorder', validateUuid('plid'), async (req, res, next) => { + const client = await pool.connect(); + try { + await loadPlaylistEdit(req.params.plid, req.user); + const { order } = req.body || {}; + if (!Array.isArray(order)) return res.status(400).json({ error: 'order must be an array of item ids' }); + await client.query('BEGIN'); + for (let i = 0; i < order.length; i++) { + await client.query( + 'UPDATE playout_items SET sort_order = $1, updated_at = NOW() WHERE id = $2 AND playlist_id = $3', + [i, order[i], req.params.plid]); + } + await client.query('COMMIT'); + res.json({ reordered: order.length }); + } catch (err) { + await client.query('ROLLBACK').catch(() => {}); + if (err.httpStatus) return res.status(err.httpStatus).json({ error: err.message }); + next(err); + } finally { client.release(); } +}); + +// DELETE /playout/items/:itemId +router.delete('/items/:itemId', validateUuid('itemId'), async (req, res, next) => { + try { + const it = await pool.query( + `SELECT i.id, c.project_id FROM playout_items i + JOIN playout_playlists p ON p.id = i.playlist_id + JOIN playout_channels c ON c.id = p.channel_id WHERE i.id = $1`, [req.params.itemId]); + if (it.rows.length === 0) return res.status(404).json({ error: 'Item not found' }); + await assertProjectAccess(req.user, it.rows[0].project_id, 'edit'); + await pool.query('DELETE FROM playout_items WHERE id = $1', [req.params.itemId]); + res.json({ deleted: true }); + } catch (err) { next(err); } +}); + +// POST /playout/items/:itemId/stage — (re)kick staging for one item +router.post('/items/:itemId/stage', validateUuid('itemId'), async (req, res, next) => { + try { + const it = await pool.query( + `SELECT i.id, i.asset_id, c.project_id FROM playout_items i + JOIN playout_playlists p ON p.id = i.playlist_id + JOIN playout_channels c ON c.id = p.channel_id WHERE i.id = $1`, [req.params.itemId]); + if (it.rows.length === 0) return res.status(404).json({ error: 'Item not found' }); + await assertProjectAccess(req.user, it.rows[0].project_id, 'edit'); + await pool.query("UPDATE playout_items SET media_status = 'pending' WHERE id = $1", [req.params.itemId]); + await stageQueue.add('stage', { itemId: it.rows[0].id, assetId: it.rows[0].asset_id }); + res.json({ queued: true }); + } catch (err) { next(err); } +}); + +// ── Failover (called by scheduler tick) ────────────────────────────────────── +// Tear down a (presumed dead) sidecar and re-spawn it on another cluster node +// matching the original capability. DeckLink channels are excluded — the +// device-index pinning makes blind re-placement risky, so they alert only. +// +// Returns { restarted: true, new_node_id } on success, or { restarted: false, +// reason } when no eligible node exists or the channel is decklink. +export async function restartChannel(channelId) { + const { rows } = await pool.query('SELECT * FROM playout_channels WHERE id = $1', [channelId]); + if (rows.length === 0) return { restarted: false, reason: 'channel not found' }; + const channel = rows[0]; + + if (channel.output_type === 'decklink') { + return { restarted: false, reason: 'decklink channels are alert-only' }; + } + + // Best-effort teardown of the old container — it may already be dead. + if (channel.container_id) { + const { remote, apiUrl } = await resolveNodeTarget(channel.node_id); + if (remote && apiUrl) { + await fetch(`${apiUrl}/sidecar/stop`, { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ containerId: channel.container_id }), + signal: AbortSignal.timeout(10000), + }).catch(() => {}); + } else { + await dockerApi('DELETE', `/containers/${channel.container_id}?force=true`).catch(() => {}); + } + } + + // Pick a different healthy node. For NDI/SRT/RTMP every online node is + // eligible (no hardware contention). Prefer the original if it's still + // online — the failure may have been transient. + const nodes = await pool.query( + `SELECT id, hostname, api_url, last_seen_at FROM cluster_nodes + WHERE id <> $1 AND last_seen_at > NOW() - INTERVAL '60 seconds' + ORDER BY last_seen_at DESC LIMIT 1`, + [channel.node_id] + ); + if (nodes.rows.length === 0) { + await pool.query( + "UPDATE playout_channels SET status = 'error', error_message = $1 WHERE id = $2", + ['no healthy node available for failover', channel.id] + ); + return { restarted: false, reason: 'no eligible node' }; + } + const newNodeId = nodes.rows[0].id; + + // Move the channel to the new node + bump the restart counters; the operator + // UI surfaces these to flag restarts. container_meta is cleared so the new + // spawn re-derives the sidecar URL. + const { rows: moved } = await pool.query( + `UPDATE playout_channels + SET node_id = $1, status = 'stopped', container_id = NULL, container_meta = '{}'::jsonb, + restart_count = restart_count + 1, last_restart_at = NOW(), + error_message = NULL, updated_at = NOW() + WHERE id = $2 RETURNING *`, + [newNodeId, channel.id] + ); + + // Spawn the sidecar directly via the shared helper. We do NOT route through + // the HTTP /start endpoint: its guard rejects status 'starting'/'running' and + // would deadlock the failover. spawnChannelSidecar flips the channel to + // running (or leaves it 'error' and throws on spawn failure). + try { + await spawnChannelSidecar(moved[0]); + return { restarted: true, new_node_id: newNodeId }; + } catch (err) { + return { restarted: false, reason: `respawn failed: ${err.message}` }; + } +} + +export default router; diff --git a/services/mam-api/src/routes/projects.js b/services/mam-api/src/routes/projects.js index 3524ab0..2e0529e 100644 --- a/services/mam-api/src/routes/projects.js +++ b/services/mam-api/src/routes/projects.js @@ -1,6 +1,8 @@ import express from 'express'; import pool from '../db/pool.js'; import { validateUuid } from '../middleware/errors.js'; +import { requireAdmin } from '../middleware/auth.js'; +import { accessibleProjectIds, assertProjectAccess } from '../auth/authz.js'; import { v4 as uuidv4 } from 'uuid'; const router = express.Router(); @@ -16,18 +18,29 @@ const slugify = (str) => { .replace(/-+/g, '-'); }; -// GET / - List all projects +// GET / - List projects the caller can access (admins see all). router.get('/', async (req, res, next) => { try { - const result = await pool.query('SELECT * FROM projects ORDER BY created_at DESC'); + const access = await accessibleProjectIds(req.user); + if (access.all) { + const result = await pool.query('SELECT * FROM projects ORDER BY created_at DESC'); + return res.json(result.rows); + } + if (access.ids.size === 0) return res.json([]); + const ids = [...access.ids]; + const result = await pool.query( + `SELECT * FROM projects WHERE id = ANY($1::uuid[]) ORDER BY created_at DESC`, + [ids] + ); res.json(result.rows); } catch (err) { next(err); } }); -// POST / - Create project -router.post('/', async (req, res, next) => { +// POST / - Create project (admin only; new projects have no grants, so a +// scoped user could never reach one they just made). +router.post('/', requireAdmin, async (req, res, next) => { try { const { name, description } = req.body; @@ -51,10 +64,11 @@ router.post('/', async (req, res, next) => { } }); -// GET /:id - Single project with asset count +// GET /:id - Single project with asset count (requires view access). router.get('/:id', async (req, res, next) => { try { const { id } = req.params; + await assertProjectAccess(req.user, id, 'view'); const result = await pool.query( `SELECT p.*, @@ -76,10 +90,11 @@ router.get('/:id', async (req, res, next) => { } }); -// PATCH /:id - Update project +// PATCH /:id - Update project (requires edit access). router.patch('/:id', async (req, res, next) => { try { const { id } = req.params; + await assertProjectAccess(req.user, id, 'edit'); const { name, description } = req.body; const updates = []; @@ -122,8 +137,9 @@ router.patch('/:id', async (req, res, next) => { } }); -// DELETE /:id - Delete project and cascade -router.delete('/:id', async (req, res, next) => { +// DELETE /:id - Delete project and cascade (admin only — destructive, wipes +// every asset/bin/recorder under it). +router.delete('/:id', requireAdmin, async (req, res, next) => { try { const { id } = req.params; @@ -143,4 +159,78 @@ router.delete('/:id', async (req, res, next) => { } }); +// ── Per-project access grants (admin only) ────────────────────────────────── +// GET /:id/access — list grants with resolved user/group display names. +router.get('/:id/access', requireAdmin, async (req, res, next) => { + try { + const { rows } = await pool.query( + `SELECT pa.subject_type, pa.subject_id, pa.level, pa.granted_at, + CASE pa.subject_type + WHEN 'user' THEN u.display_name + WHEN 'group' THEN g.name + END AS subject_name, + CASE pa.subject_type + WHEN 'user' THEN u.username + ELSE NULL + END AS username + FROM project_access pa + LEFT JOIN users u ON pa.subject_type = 'user' AND u.id = pa.subject_id + LEFT JOIN groups g ON pa.subject_type = 'group' AND g.id = pa.subject_id + WHERE pa.project_id = $1 + ORDER BY pa.subject_type, subject_name`, + [req.params.id] + ); + res.json(rows); + } catch (err) { next(err); } +}); + +// POST /:id/access { subject_type, subject_id, level } — grant or update. +router.post('/:id/access', requireAdmin, async (req, res, next) => { + try { + const { subject_type, subject_id, level } = req.body || {}; + if (!['user', 'group'].includes(subject_type)) { + return res.status(400).json({ error: "subject_type must be 'user' or 'group'" }); + } + if (!subject_id) return res.status(400).json({ error: 'subject_id required' }); + const lvl = level || 'view'; + if (!['view', 'edit'].includes(lvl)) { + return res.status(400).json({ error: "level must be 'view' or 'edit'" }); + } + + // Validate the subject actually exists so we don't create dead grants. + const tbl = subject_type === 'user' ? 'users' : 'groups'; + const exists = await pool.query(`SELECT 1 FROM ${tbl} WHERE id = $1`, [subject_id]); + if (exists.rows.length === 0) { + return res.status(404).json({ error: subject_type + ' not found' }); + } + + const { rows } = await pool.query( + `INSERT INTO project_access (project_id, subject_type, subject_id, level, granted_by) + VALUES ($1, $2, $3, $4, $5) + ON CONFLICT (project_id, subject_type, subject_id) + DO UPDATE SET level = EXCLUDED.level, granted_by = EXCLUDED.granted_by, granted_at = NOW() + RETURNING project_id, subject_type, subject_id, level, granted_at`, + [req.params.id, subject_type, subject_id, lvl, req.user?.id || null] + ); + res.status(201).json(rows[0]); + } catch (err) { next(err); } +}); + +// DELETE /:id/access/:subjectType/:subjectId — revoke a grant. +router.delete('/:id/access/:subjectType/:subjectId', requireAdmin, async (req, res, next) => { + try { + const { id, subjectType, subjectId } = req.params; + if (!['user', 'group'].includes(subjectType)) { + return res.status(400).json({ error: "subjectType must be 'user' or 'group'" }); + } + const { rowCount } = await pool.query( + `DELETE FROM project_access + WHERE project_id = $1 AND subject_type = $2 AND subject_id = $3`, + [id, subjectType, subjectId] + ); + if (rowCount === 0) return res.status(404).json({ error: 'grant not found' }); + res.status(204).end(); + } catch (err) { next(err); } +}); + export default router; diff --git a/services/mam-api/src/routes/recorders.js b/services/mam-api/src/routes/recorders.js index cf6bff3..19e5868 100644 --- a/services/mam-api/src/routes/recorders.js +++ b/services/mam-api/src/routes/recorders.js @@ -1,15 +1,53 @@ import express from 'express'; import http from 'http'; import fs from 'fs'; +import { createReadStream, existsSync } from 'fs'; +import { stat } from 'fs/promises'; import net from 'net'; import dgram from 'dgram'; import pool from '../db/pool.js'; -import { getS3Bucket } from '../s3/client.js'; +import { s3Client, getS3Bucket } from '../s3/client.js'; +import { Upload } from '@aws-sdk/lib-storage'; import { validateUuid } from '../middleware/errors.js'; +import { assertProjectAccess, accessibleProjectIds } from '../auth/authz.js'; import { v4 as uuidv4 } from 'uuid'; +import { Queue } from 'bullmq'; const router = express.Router(); -router.param('id', (req, res, next) => validateUuid('id')(req, res, next)); + +// BullMQ proxy queue — used by the growing-file stop handler to queue proxy +// jobs when the capture container's finalize call races with the S3 upload. +const parseRedisUrl = (url) => { + const parsed = new URL(url); + return { host: parsed.hostname, port: parseInt(parsed.port, 10) || 6379 }; +}; +const proxyQueue = new Queue('proxy', { + connection: parseRedisUrl(process.env.REDIS_URL || 'redis://queue:6379'), +}); + +// Every /:id recorder route is scoped to the recorder's project. The param +// handler validates the UUID, resolves the owning project_id, and asserts the +// 'view' baseline; mutating routes escalate to 'edit' via requireRecorderEdit. +// A recorder with a NULL project_id resolves to admin-only (assertProjectAccess +// throws 403 for non-admins on a null project). +router.param('id', async (req, res, next) => { + validateUuid('id')(req, res, () => {}); + if (res.headersSent) return; + try { + const { rows } = await pool.query('SELECT project_id FROM recorders WHERE id = $1', [req.params.id]); + if (rows.length === 0) return res.status(404).json({ error: 'Recorder not found' }); + req.recorderProjectId = rows[0].project_id; + await assertProjectAccess(req.user, req.recorderProjectId, 'view'); + next(); + } catch (err) { next(err); } +}); + +async function requireRecorderEdit(req, res, next) { + try { + await assertProjectAccess(req.user, req.recorderProjectId, 'edit'); + next(); + } catch (err) { next(err); } +} // Base port for on-demand SDI sidecar containers on remote worker nodes. // Device index 0 → 7438, index 1 → 7439, etc. @@ -130,6 +168,7 @@ const RECORDER_FIELDS = [ 'proxy_audio_codec', 'proxy_audio_bitrate', 'proxy_audio_channels', 'proxy_container', 'project_id', 'node_id', 'device_index', + 'growing_enabled', ]; function pickRecorderFields(body) { @@ -149,6 +188,17 @@ function pickRecorderFields(body) { // parallel with a per-call timeout from `dockerApi`. router.get('/', async (req, res, next) => { try { + // Scope to recorders in projects the caller can access (admins unfiltered). + // Recorders with a NULL project are admin-only and never appear for scoped + // users (accessibleProjectIds never yields a null id). + const access = await accessibleProjectIds(req.user); + let scopeClause = ''; + const params = []; + if (!access.all) { + if (access.ids.size === 0) return res.json([]); + scopeClause = 'WHERE r.project_id = ANY($1::uuid[])'; + params.push([...access.ids]); + } const result = await pool.query(` SELECT r.*, la.live_asset_id FROM recorders r @@ -162,8 +212,9 @@ router.get('/', async (req, res, next) => { ORDER BY a.created_at DESC LIMIT 1 ) la ON TRUE + ${scopeClause} ORDER BY r.created_at DESC - `); + `, params); const rows = result.rows; // Only inspect containers for recorders that actually claim to be recording. @@ -194,10 +245,15 @@ router.post('/', async (req, res, next) => { .json({ error: 'Name and source_type are required' }); } + // Creating a recorder writes into a project — require edit there. A recorder + // with no project_id is admin-only (assertProjectAccess denies non-admins on + // a null project). + await assertProjectAccess(req.user, fields.project_id ?? null, 'edit'); + // Defaults — written on insert so the DB row is always self-contained. const defaults = { source_config: {}, - recording_codec: 'prores_hq', + recording_codec: 'hevc_nvenc', recording_resolution: 'native', recording_audio_codec: 'pcm_s24le', recording_audio_channels: 2, @@ -256,7 +312,7 @@ router.get('/:id', async (req, res, next) => { // PATCH /:id - Edit recorder settings // Blocked while recorder is actively recording to prevent config drift. -router.patch('/:id', async (req, res, next) => { +router.patch('/:id', requireRecorderEdit, async (req, res, next) => { try { const { id } = req.params; @@ -295,7 +351,7 @@ router.patch('/:id', async (req, res, next) => { }); // POST /:id/start - Start recording -router.post('/:id/start', async (req, res, next) => { +router.post('/:id/start', requireRecorderEdit, async (req, res, next) => { try { const { id } = req.params; @@ -322,14 +378,25 @@ router.post('/:id/start', async (req, res, next) => { const externalMamApiUrl = `http://${process.env.NODE_IP || '172.18.91.200'}:${process.env.PORT_MAM_API || 47432}`; const dockerNetwork = process.env.DOCKER_NETWORK || 'wild-dragon_wild-dragon'; - // Growing-files mode is a global setting (settings table). When on, the - // capture container writes the master to its /growing/ mount instead of - // streaming it to S3 — Premiere can mount the SMB share and edit it live. - const growingRow = await pool.query( - `SELECT value FROM settings WHERE key = 'growing_enabled'` - ); - const growingEnabled = - growingRow.rows[0]?.value === 'true' || growingRow.rows[0]?.value === true; + // Growing-files mode is a PER-RECORDER setting (recorders.growing_enabled). + // When on, the capture container writes the master to its /growing/ mount + // instead of streaming it to S3 — editors can mount the SMB share and cut it + // live. The SMB share itself (mount source + credentials) is shared + // infrastructure configured globally in Settings → Storage. + const growingEnabled = recorder.growing_enabled === true; + + // Shared growing-files SMB infrastructure (global settings). Used to mount + // the CIFS share inside the capture container (services/capture mounts it + // with these credentials when GROWING_SMB_MOUNT is set). + const growingInfra = {}; + { + const r = await pool.query( + `SELECT key, value FROM settings WHERE key = ANY($1)`, + [['growing_smb_mount', 'growing_smb_username', 'growing_smb_password', 'growing_smb_vers']] + ); + for (const { key, value } of r.rows) growingInfra[key] = value; + } + const smbMount = growingEnabled ? (growingInfra.growing_smb_mount || '') : ''; // Operator-supplied clip name wins over the auto-timestamped fallback. // The Recorders UI passes this on the start request when the user types @@ -345,6 +412,14 @@ router.post('/:id/start', async (req, res, next) => { ? req.body.projectId : recorder.project_id; + // requireRecorderEdit only covered the recorder's own project. If this take + // is being routed into a DIFFERENT project, the caller must have edit there + // too — otherwise edit on recorder A's project would let them write live + // assets into any project B. + if (takeProjectId !== recorder.project_id) { + await assertProjectAccess(req.user, takeProjectId, 'edit'); + } + // live-asset: create the asset row right now (status='live') so the // library shows the recording while it is happening. const assetIdLive = uuidv4(); @@ -406,6 +481,13 @@ router.post('/:id/start', async (req, res, next) => { `MAM_API_TOKEN=${process.env.CAPTURE_TOKEN || ''}`, `GROWING_ENABLED=${growingEnabled ? 'true' : 'false'}`, `GROWING_PATH=/growing`, + // SMB mount details for the in-container CIFS mount (Approach A). Empty + // GROWING_SMB_MOUNT → capture falls back to the host-bound /growing volume + // (or to S3 streaming if growing isn't enabled). + `GROWING_SMB_MOUNT=${smbMount}`, + `GROWING_SMB_USERNAME=${growingInfra.growing_smb_username || ''}`, + `GROWING_SMB_PASSWORD=${growingInfra.growing_smb_password || ''}`, + `GROWING_SMB_VERS=${growingInfra.growing_smb_vers || '3.0'}`, ]; // Deltacast: pass port count so the capture container can enumerate @@ -481,7 +563,15 @@ router.post('/:id/start', async (req, res, next) => { for (const d of dcEntries) hostBinds.push(`/dev/${d}:/dev/${d}`); } catch (_) { /* no /dev/deltacast* nodes on this host */ } } - if (growingEnabled) hostBinds.push('/mnt/NVME/MAM/wild-dragon-growing:/growing'); + // /growing handling: + // - SMB mount configured → DON'T host-bind; the capture container mounts + // the CIFS share at /growing itself (Approach A). A bind-mount here + // would shadow the in-container mount. + // - growing on but no SMB mount → legacy host bind-mount fallback. + // - growing off → no /growing mount at all. + if (growingEnabled && !smbMount) { + hostBinds.push('/mnt/NVME/MAM/wild-dragon-growing:/growing'); + } const localEnv = [...env]; if (useGpu) { @@ -551,7 +641,7 @@ router.post('/:id/start', async (req, res, next) => { }); // POST /:id/stop - Stop recording -router.post('/:id/stop', async (req, res, next) => { +router.post('/:id/stop', requireRecorderEdit, async (req, res, next) => { try { const { id } = req.params; @@ -616,6 +706,28 @@ router.post('/:id/stop', async (req, res, next) => { } } + // ── Growing-files S3 promotion ──────────────────────────────────────────── + // When growing_enabled=true the capture container writes the master file to + // /growing/{projectId}/{clipName}.{ext} (a host bind-mount that the mam-api + // container also has at /growing). The capture container's graceful-shutdown + // handler (triggered by the Docker stop above) calls POST /assets/:id/finalize + // with the expected S3 key, which queues the proxy job — but the file was + // never uploaded to S3, so the proxy worker fails with "unable to open file". + // + // Fix: after the container has exited (ffmpeg is done flushing), upload the + // growing file to the canonical S3 key from here. This is synchronous and + // completes before the HTTP response reaches the client, so the already-queued + // proxy job will find a valid S3 object when the worker dequeues it. + // + // Only applies to LOCAL recorders — remote recorders write to a different + // node's /growing mount which this process cannot access. + if (!isRemote && recorder.growing_enabled === true && recorder.current_session_id) { + await promoteGrowingFileToS3(recorder).catch(err => { + // Non-fatal — log and continue so the stop always succeeds. + console.error('[recorders/stop] growing-file promotion failed (non-fatal):', err.message); + }); + } + const updateResult = await pool.query( `UPDATE recorders SET container_id = NULL, status = $1, updated_at = NOW() @@ -630,6 +742,109 @@ router.post('/:id/stop', async (req, res, next) => { } }); +/** + * Upload a completed growing-file master from /growing to S3 so the proxy + * worker can find it at the expected original_s3_key. + * + * The capture container writes to: + * /growing/{projectId}/{clipName}.{ext} + * + * The canonical S3 key (set on the asset row at recording start) is: + * projects/{projectId}/masters/{clipName}.{ext} + * + * We look up the live/processing asset to derive both paths, do a multipart + * upload, update the asset's original_s3_key and file_size to match what we + * actually uploaded, then ensure a proxy job exists for it. + */ +async function promoteGrowingFileToS3(recorder) { + const clipName = recorder.current_session_id; + const container = recorder.recording_container || 'mov'; + + // Find the asset that was pre-created at recording start. It could be in + // 'live' (finalize hasn't fired yet) or 'processing' (finalize already ran + // from the container's SIGTERM handler). We need both its id and its + // project_id to reconstruct the growing path. + const assetRes = await pool.query( + `SELECT id, project_id, status, original_s3_key + FROM assets + WHERE display_name = $1 + AND status IN ('live', 'processing', 'error') + ORDER BY created_at DESC + LIMIT 1`, + [clipName] + ); + + if (assetRes.rows.length === 0) { + console.warn(`[recorders/stop] no asset found for clip "${clipName}" — skipping growing-file promotion`); + return; + } + + const asset = assetRes.rows[0]; + const projectId = asset.project_id; + const growingDir = process.env.GROWING_DIR || '/growing'; + const localPath = `${growingDir}/${projectId}/${clipName}.${container}`; + const s3Key = `projects/${projectId}/masters/${clipName}.${container}`; + + if (!existsSync(localPath)) { + console.warn(`[recorders/stop] growing file not found at ${localPath} — nothing to promote (empty recording?)`); + return; + } + + const fileStat = await stat(localPath); + if (fileStat.size === 0) { + console.warn(`[recorders/stop] growing file at ${localPath} is empty — skipping promotion`); + return; + } + + console.log(`[recorders/stop] promoting growing file ${localPath} (${fileStat.size} bytes) → s3://${getS3Bucket()}/${s3Key}`); + + const upload = new Upload({ + client: s3Client, + params: { + Bucket: getS3Bucket(), + Key: s3Key, + Body: createReadStream(localPath), + }, + queueSize: 4, + partSize: 8 * 1024 * 1024, + }); + await upload.done(); + + console.log(`[recorders/stop] S3 upload complete for ${s3Key}`); + + // Ensure the asset row reflects the correct S3 key and file size. The + // capture container's finalize call may have already set original_s3_key to + // this same value (it was pre-set at start), but update file_size which + // finalize doesn't touch. + await pool.query( + `UPDATE assets + SET original_s3_key = $1, + file_size = $2, + updated_at = NOW() + WHERE id = $3`, + [s3Key, fileStat.size, asset.id] + ); + + // If the asset is still 'live' (capture container's finalize hasn't fired or + // failed), flip it to 'processing' and queue the proxy job ourselves so the + // clip doesn't get stuck in the library as "Recording…". + if (asset.status === 'live') { + console.log(`[recorders/stop] finalize not yet called — queueing proxy and flipping asset ${asset.id} to processing`); + await pool.query( + `UPDATE assets SET status = 'processing', updated_at = NOW() WHERE id = $1`, + [asset.id] + ); + await proxyQueue.add('generate', { + assetId: asset.id, + inputKey: s3Key, + outputKey: `proxies/${asset.id}.mp4`, + }); + } + // If status is already 'processing', the capture container's finalize already + // ran and queued the proxy job. The S3 upload we just did ensures the worker + // will find a valid object when it dequeues that job — nothing else to do. +} + // GET /:id/status - Get live status router.get('/:id/status', async (req, res, next) => { try { @@ -722,7 +937,7 @@ router.get('/:id/status', async (req, res, next) => { }); // DELETE /:id - Delete recorder -router.delete('/:id', async (req, res, next) => { +router.delete('/:id', requireRecorderEdit, async (req, res, next) => { try { const { id } = req.params; diff --git a/services/mam-api/src/routes/sequences.js b/services/mam-api/src/routes/sequences.js index b7dd20c..76041a5 100644 --- a/services/mam-api/src/routes/sequences.js +++ b/services/mam-api/src/routes/sequences.js @@ -3,6 +3,7 @@ import express from 'express'; import pool from '../db/pool.js'; import { getSignedUrlForObject } from '../s3/client.js'; import { validateUuid } from '../middleware/errors.js'; +import { assertProjectAccess } from '../auth/authz.js'; import { Queue } from 'bullmq'; const parseRedisUrl = (url) => { @@ -19,7 +20,27 @@ const conformQueue = new Queue('conform', { }); const router = express.Router(); -router.param('id', (req, res, next) => validateUuid('id')(req, res, next)); + +// Scope every /:id sequence route to its project: validate the UUID, resolve +// project_id, assert the 'view' baseline; mutators escalate via requireSequenceEdit. +router.param('id', async (req, res, next) => { + validateUuid('id')(req, res, () => {}); + if (res.headersSent) return; + try { + const { rows } = await pool.query('SELECT project_id FROM sequences WHERE id = $1', [req.params.id]); + if (rows.length === 0) return res.status(404).json({ error: 'Sequence not found' }); + req.sequenceProjectId = rows[0].project_id; + await assertProjectAccess(req.user, req.sequenceProjectId, 'view'); + next(); + } catch (err) { next(err); } +}); + +async function requireSequenceEdit(req, res, next) { + try { + await assertProjectAccess(req.user, req.sequenceProjectId, 'edit'); + next(); + } catch (err) { next(err); } +} // ── Row mapper ──────────────────────────────────────────────────────────────── // node-postgres returns NUMERIC columns as strings. Coerce frame_rate to a @@ -124,6 +145,7 @@ router.get('/', async (req, res, next) => { try { const { project_id } = req.query; if (!project_id) return res.status(400).json({ error: 'project_id is required' }); + await assertProjectAccess(req.user, project_id, 'view'); const r = await pool.query( `SELECT * FROM sequences WHERE project_id = $1 ORDER BY updated_at DESC`, [project_id] @@ -143,6 +165,7 @@ router.post('/', async (req, res, next) => { height = 1080, } = req.body; if (!project_id) return res.status(400).json({ error: 'project_id is required' }); + await assertProjectAccess(req.user, project_id, 'edit'); const r = await pool.query( `INSERT INTO sequences (project_id, name, frame_rate, width, height) VALUES ($1, $2, $3, $4, $5) RETURNING *`, @@ -188,7 +211,7 @@ router.get('/:id', async (req, res, next) => { }); // ── PUT /:id – update sequence metadata ────────────────────────────────────── -router.put('/:id', async (req, res, next) => { +router.put('/:id', requireSequenceEdit, async (req, res, next) => { try { const { name, frame_rate, width, height } = req.body; const updates = []; @@ -211,7 +234,7 @@ router.put('/:id', async (req, res, next) => { }); // ── DELETE /:id ─────────────────────────────────────────────────────────────── -router.delete('/:id', async (req, res, next) => { +router.delete('/:id', requireSequenceEdit, async (req, res, next) => { try { const r = await pool.query(`DELETE FROM sequences WHERE id = $1`, [req.params.id]); if (!r.rowCount) return res.status(404).json({ error: 'Sequence not found' }); @@ -220,25 +243,41 @@ router.delete('/:id', async (req, res, next) => { }); // ── PUT /:id/clips – full replace of clip array (single transaction) ────────── -router.put('/:id/clips', async (req, res, next) => { - // Verify sequence exists first (before acquiring transaction client) - const seqCheck = await pool.query(`SELECT id FROM sequences WHERE id = $1`, [req.params.id]); - if (!seqCheck.rows.length) return res.status(404).json({ error: 'Sequence not found' }); - +router.put('/:id/clips', requireSequenceEdit, async (req, res, next) => { const clips = Array.isArray(req.body) ? req.body : []; - for (const c of clips) { - if (!c.asset_id) return res.status(400).json({ error: 'Each clip must have asset_id' }); - if (!Number.isFinite(Number(c.timeline_in_frames)) || !Number.isFinite(Number(c.timeline_out_frames)) || - !Number.isFinite(Number(c.source_in_frames)) || !Number.isFinite(Number(c.source_out_frames))) { - return res.status(400).json({ error: 'Clip frame fields must be finite numbers' }); - } - if (!Number.isInteger(Number(c.track)) || Number(c.track) < 0) { - return res.status(400).json({ error: 'Clip track must be a non-negative integer' }); - } - } - - const client = await pool.connect(); + let client; try { + // Verify sequence exists first (before acquiring transaction client). + const seqCheck = await pool.query(`SELECT id FROM sequences WHERE id = $1`, [req.params.id]); + if (!seqCheck.rows.length) return res.status(404).json({ error: 'Sequence not found' }); + + for (const c of clips) { + if (!c.asset_id) return res.status(400).json({ error: 'Each clip must have asset_id' }); + if (!Number.isFinite(Number(c.timeline_in_frames)) || !Number.isFinite(Number(c.timeline_out_frames)) || + !Number.isFinite(Number(c.source_in_frames)) || !Number.isFinite(Number(c.source_out_frames))) { + return res.status(400).json({ error: 'Clip frame fields must be finite numbers' }); + } + if (!Number.isInteger(Number(c.track)) || Number(c.track) < 0) { + return res.status(400).json({ error: 'Clip track must be a non-negative integer' }); + } + } + + // Every referenced asset must belong to THIS sequence's project. Without this, + // a user with edit on the sequence could splice in assets from a project they + // can't access — and GET /:id would then hand back those assets' names and + // signed proxy URLs (cross-project leak). + const assetIds = [...new Set(clips.map(c => c.asset_id))]; + if (assetIds.length) { + const owning = await pool.query( + `SELECT id FROM assets WHERE id = ANY($1::uuid[]) AND project_id = $2`, + [assetIds, req.sequenceProjectId] + ); + if (owning.rows.length !== assetIds.length) { + return res.status(400).json({ error: 'All clip assets must belong to the sequence\'s project' }); + } + } + + client = await pool.connect(); await client.query('BEGIN'); await client.query( `DELETE FROM sequence_clips WHERE sequence_id = $1`, @@ -265,10 +304,12 @@ router.put('/:id/clips', async (req, res, next) => { await client.query('COMMIT'); res.json({ ok: true, count: clips.length }); } catch (e) { - await client.query('ROLLBACK'); + // client is only set once we've connected; a failure in the pre-transaction + // queries (existence/validation/ownership) has no transaction to roll back. + if (client) await client.query('ROLLBACK').catch(() => {}); next(e); } finally { - client.release(); + if (client) client.release(); } }); @@ -300,7 +341,7 @@ router.post('/:id/export/edl', async (req, res, next) => { // ── POST /:id/conform – conform sequence via FCP XML ───────────────────────── // Accepts FCP XML content and encode settings from the Premiere plugin, // queues a conform job in BullMQ, and returns the job ID for polling. -router.post('/:id/conform', async (req, res, next) => { +router.post('/:id/conform', requireSequenceEdit, async (req, res, next) => { try { const seqR = await pool.query(`SELECT * FROM sequences WHERE id = $1`, [req.params.id]); if (!seqR.rows.length) return res.status(404).json({ error: 'Sequence not found' }); diff --git a/services/mam-api/src/routes/settings.js b/services/mam-api/src/routes/settings.js index f7cefdc..82fbe17 100644 --- a/services/mam-api/src/routes/settings.js +++ b/services/mam-api/src/routes/settings.js @@ -258,21 +258,45 @@ router.put('/transcoding', async (req, res, next) => { // while it's still being written; the promotion worker later moves the // finalized file to S3 and flips the asset to status='ready'. -const GROWING_KEYS = ['growing_enabled', 'growing_path', 'growing_smb_url', 'growing_promote_after_seconds']; +// Growing-files mode is now a PER-RECORDER setting (recorders.growing_enabled); +// the legacy global `growing_enabled` key is no longer read at recorder start. +// These global keys describe the shared SMB landing-zone infrastructure only: +// - growing_path container mount point (default /growing) +// - growing_smb_url smb://... display string for editors (Premiere) +// - growing_smb_mount //host/share CIFS source the capture container mounts +// - growing_smb_username SMB user for the system-side CIFS mount +// - growing_smb_password SMB password (WRITE-ONLY; never returned) +// - growing_smb_vers CIFS protocol version (default 3.0) +// - growing_promote_after_seconds idle threshold before S3 promotion +const GROWING_KEYS = [ + 'growing_path', 'growing_smb_url', 'growing_smb_mount', + 'growing_smb_username', 'growing_smb_vers', 'growing_promote_after_seconds', +]; +// growing_smb_password is handled separately: stored on PUT but NEVER returned +// on GET (only a *_exists flag), mirroring s3_secret_key. router.get('/growing', async (req, res, next) => { try { const result = await pool.query( `SELECT key, value FROM settings WHERE key = ANY($1)`, - [GROWING_KEYS] + [[...GROWING_KEYS, 'growing_smb_password']] ); const out = { - growing_enabled: 'false', growing_path: '/growing', growing_smb_url: '', + growing_smb_mount: '', + growing_smb_username: '', + growing_smb_vers: '3.0', growing_promote_after_seconds: '8', + growing_smb_password_exists: false, }; - for (const { key, value } of result.rows) out[key] = value; + for (const { key, value } of result.rows) { + if (key === 'growing_smb_password') { + out.growing_smb_password_exists = !!(value && value.length); + } else { + out[key] = value; + } + } res.json(out); } catch (err) { next(err); @@ -290,6 +314,19 @@ router.put('/growing', async (req, res, next) => { ); } } + // SMB password is write-only. A non-empty value sets/replaces it. To remove + // it, send growing_smb_password_clear:true. A blank/omitted password field + // leaves the stored value untouched (so operators don't retype it on every + // save). + if (req.body.growing_smb_password_clear === true) { + await pool.query(`DELETE FROM settings WHERE key = 'growing_smb_password'`); + } else if (typeof req.body.growing_smb_password === 'string' && req.body.growing_smb_password.length > 0) { + await pool.query( + `INSERT INTO settings (key, value, updated_at) VALUES ('growing_smb_password', $1, NOW()) + ON CONFLICT (key) DO UPDATE SET value = $1, updated_at = NOW()`, + [req.body.growing_smb_password] + ); + } res.json({ message: 'Growing-files settings saved' }); } catch (err) { next(err); diff --git a/services/mam-api/src/routes/storage.js b/services/mam-api/src/routes/storage.js index 9d767a8..493d87b 100644 --- a/services/mam-api/src/routes/storage.js +++ b/services/mam-api/src/routes/storage.js @@ -14,10 +14,12 @@ const exec = promisify(execCb); const router = express.Router(); // Defaults mirrored from settings.js so the overview never returns nulls. +// Growing-file mode is now per-recorder; "enabled" here means the shared SMB +// landing zone is CONFIGURED (a mount source is set), not a global on/off. const GROWING_DEFAULTS = { - growing_enabled: 'false', growing_path: '/growing', growing_smb_url: '', + growing_smb_mount: '', growing_promote_after_seconds: '8', }; @@ -100,7 +102,9 @@ router.get('/overview', async (req, res, next) => { try { // Growing files — merge defaults with whatever's in `settings`. const growingRaw = { ...GROWING_DEFAULTS, ...(await readSettings(Object.keys(GROWING_DEFAULTS))) }; - const growingEnabled = growingRaw.growing_enabled === 'true' || growingRaw.growing_enabled === true; + // "enabled" now means the shared SMB landing zone is configured (a mount + // source is set). Per-recorder toggles decide which recorders actually use it. + const growingEnabled = !!(growingRaw.growing_smb_mount && growingRaw.growing_smb_mount.trim()); const containerPath = growingRaw.growing_path || '/growing'; const mount = await probeGrowingPath(containerPath); @@ -117,6 +121,7 @@ router.get('/overview', async (req, res, next) => { // existing deploy uses this symlink — surface it for operator context. host_path: '/mnt/NVME/MAM/wild-dragon-growing', smb_url: growingRaw.growing_smb_url || '', + smb_mount: growingRaw.growing_smb_mount || '', promote_after_seconds: parseInt(growingRaw.growing_promote_after_seconds, 10) || 8, exists: mount.exists, writable: mount.writable, diff --git a/services/mam-api/src/routes/upload.js b/services/mam-api/src/routes/upload.js index fcb89ef..c1ab9f3 100644 --- a/services/mam-api/src/routes/upload.js +++ b/services/mam-api/src/routes/upload.js @@ -14,6 +14,7 @@ import { AbortMultipartUploadCommand, } from '@aws-sdk/client-s3'; import { getAmppConfig, ensureFolderPath } from '../ampp/client.js'; +import { assertProjectAccess, accessibleProjectIds } from '../auth/authz.js'; const router = express.Router(); @@ -138,16 +139,24 @@ function mediaTypeFromMime(mime = '') { return 'document'; } -// GET /api/v1/upload - List in-progress uploads (#68) +// GET /api/v1/upload - List in-progress uploads (#68). Scoped to projects the +// caller can see — admins are unfiltered; a scoped viewer/editor only sees +// uploads for projects they have access to (no enumeration of other projects' +// in-flight filenames). router.get('/', async (req, res, next) => { try { - const result = await pool.query( - `SELECT id, filename, display_name, project_id, bin_id, status, created_at, updated_at - FROM assets - WHERE status = 'ingesting' - ORDER BY created_at DESC - LIMIT 50` - ); + const access = await accessibleProjectIds(req.user); + let query = `SELECT id, filename, display_name, project_id, bin_id, status, created_at, updated_at + FROM assets + WHERE status = 'ingesting'`; + const params = []; + if (!access.all) { + if (access.ids.size === 0) return res.json([]); + query += ` AND project_id = ANY($1::uuid[])`; + params.push([...access.ids]); + } + query += ` ORDER BY created_at DESC LIMIT 50`; + const result = await pool.query(query, params); res.json(result.rows); } catch (err) { next(err); } }); @@ -163,6 +172,17 @@ router.post('/init', async (req, res, next) => { }); } + // Uploading creates an asset under a project — require edit on that project. + // Without this, any logged-in user could write into any project. + await assertProjectAccess(req.user, projectId, 'edit'); + if (binId) { + const bin = await pool.query('SELECT project_id FROM bins WHERE id = $1', [binId]); + if (bin.rows.length === 0) return res.status(400).json({ error: 'binId not found' }); + if (bin.rows[0].project_id !== projectId) { + return res.status(400).json({ error: 'binId belongs to a different project' }); + } + } + const assetId = uuidv4(); const s3Key = `originals/${assetId}/${filename}`; const tagsArray = tags ? (Array.isArray(tags) ? tags : [tags]) : []; @@ -326,6 +346,20 @@ router.post('/simple', upload.single('file'), async (req, res, next) => { }); } + // Same authz gate as /init. + await assertProjectAccess(req.user, projectId, 'edit'); + if (binId) { + const bin = await pool.query('SELECT project_id FROM bins WHERE id = $1', [binId]); + if (bin.rows.length === 0) { + unlinkPart(tmpPath); + return res.status(400).json({ error: 'binId not found' }); + } + if (bin.rows[0].project_id !== projectId) { + unlinkPart(tmpPath); + return res.status(400).json({ error: 'binId belongs to a different project' }); + } + } + const assetId = uuidv4(); const s3Key = `originals/${assetId}/${filename}`; const mimeType = contentType || req.file.mimetype; diff --git a/services/mam-api/src/s3/client.js b/services/mam-api/src/s3/client.js index 5c30a7f..374bbb9 100644 --- a/services/mam-api/src/s3/client.js +++ b/services/mam-api/src/s3/client.js @@ -1,3 +1,4 @@ +import { NodeHttpHandler } from '@smithy/node-http-handler'; import { S3Client, GetObjectCommand, DeleteObjectCommand, HeadBucketCommand, ListObjectsV2Command } from '@aws-sdk/client-s3'; import { getSignedUrl } from '@aws-sdk/s3-request-presigner'; import { Upload } from '@aws-sdk/lib-storage'; @@ -22,6 +23,9 @@ function buildClient(cfg) { secretAccessKey: cfg.secretKey, }, forcePathStyle: true, + // Hard request/connection timeouts so a stalled RustFS GET can't hang the + // /video and /hls endpoints forever (the original browser-playback hang). + requestHandler: new NodeHttpHandler({ requestTimeout: 30_000, connectionTimeout: 10_000 }), requestChecksumCalculation: 'WHEN_REQUIRED', responseChecksumValidation: 'WHEN_REQUIRED', }); diff --git a/services/mam-api/src/scheduler.js b/services/mam-api/src/scheduler.js index 8b6b305..e78fe41 100644 --- a/services/mam-api/src/scheduler.js +++ b/services/mam-api/src/scheduler.js @@ -9,6 +9,8 @@ import pool from './db/pool.js'; import { syncToAmpp } from './routes/upload.js'; +import { restartChannel } from './routes/playout.js'; +import { INTERNAL_TOKEN } from './middleware/auth.js'; const TICK_INTERVAL_MS = parseInt(process.env.SCHEDULER_TICK_MS || '15000', 10); const SELF_URL = process.env.MAM_API_SELF_URL || `http://127.0.0.1:${process.env.PORT || 3000}`; @@ -19,7 +21,10 @@ let _interval = null; async function callSelf(path, method = 'POST') { const res = await fetch(`${SELF_URL}${path}`, { method, - headers: { 'Content-Type': 'application/json' }, + headers: { + 'Content-Type': 'application/json', + 'x-internal-token': INTERNAL_TOKEN, + }, signal: AbortSignal.timeout(30000), }); if (!res.ok) { @@ -175,6 +180,13 @@ async function tick() { for (const row of ampps.rows) { await syncToAmpp(row.id, row.project_id, row.bin_id); } + + // 6) Playout channel health checks. Ping each running channel's sidecar + // /status; on success bump last_heartbeat_at, on failure increment a + // transient miss counter (in playout_sidecars.last_heartbeat_at age). + // Three consecutive misses → auto-restart on a healthy node (non- + // decklink), or alert-only for decklink. + await playoutHealthTick(client); } catch (err) { console.error('[scheduler] tick error:', err); } finally { @@ -201,6 +213,142 @@ async function enqueueNextOccurrence(schedule, client) { console.log(`[scheduler] queued next "${schedule.name}" → ${start.toISOString()}`); } +// ── Playout channel health + failover ──────────────────────────────────────── +// Tick step 6. Reuses the same advisory lock so only one replica probes the +// sidecars; multi-replica pings would just waste cycles. A missed probe is +// counted via last_heartbeat_at age: > 3 * TICK_INTERVAL means 3 consecutive +// misses. +// Persist the as-run compliance log for one channel from a sidecar /status +// payload. The sidecar reports the currently on-air item via currentItemId / +// currentClip / currentItemStartedAt (playout-manager.getStatus). We keep at +// most one "open" row (ended_at IS NULL) per channel: when the on-air item +// changes (or playout stops) we close the open row — stamping ended_at and a +// computed duration_s — and, if a new clip is on air, open a fresh row. +// +// playout_as_run columns (migration 029): id, channel_id, asset_id, item_id, +// clip_name, started_at, ended_at, duration_s, result. +async function writeAsRun(client, channelId, engine) { + const currentItemId = engine && engine.currentItemId ? engine.currentItemId : null; + + // The currently-open as-run row for this channel, if any. + const { rows: openRows } = await client.query( + `SELECT id, item_id, started_at FROM playout_as_run + WHERE channel_id = $1 AND ended_at IS NULL + ORDER BY started_at DESC LIMIT 1`, + [channelId] + ); + const open = openRows[0] || null; + + // Same clip still on air → nothing to do. + if (open && currentItemId && open.item_id === currentItemId) return; + // Nothing on air and nothing open → nothing to do. + if (!open && !currentItemId) return; + + // Close the previous open row (clip changed, or playout stopped). + if (open) { + await client.query( + `UPDATE playout_as_run + SET ended_at = NOW(), + duration_s = EXTRACT(EPOCH FROM (NOW() - started_at)) + WHERE id = $1`, + [open.id] + ); + } + + // Open a new row for the clip now on air. Resolve the item's asset_id so the + // compliance log links back to the source asset even after the playlist item + // is later deleted. + if (currentItemId) { + let assetId = null; + try { + const { rows } = await client.query( + 'SELECT asset_id FROM playout_items WHERE id = $1', [currentItemId] + ); + if (rows.length > 0) assetId = rows[0].asset_id; + } catch (_) { /* item may have been deleted; log without asset link */ } + + await client.query( + `INSERT INTO playout_as_run + (channel_id, asset_id, item_id, clip_name, started_at, result) + VALUES ($1, $2, $3, $4, COALESCE($5::timestamptz, NOW()), 'played')`, + [channelId, assetId, currentItemId, engine.currentClip || null, + engine.currentItemStartedAt || null] + ); + } +} + +async function playoutHealthTick(client) { + let channels; + try { + ({ rows: channels } = await client.query( + `SELECT id, output_type, container_meta, node_id, last_heartbeat_at, updated_at, restart_count + FROM playout_channels WHERE status = 'running'` + )); + } catch (err) { + // Migration 029 may not be applied yet — bail silently rather than crash. + if (err.code === '42P01') return; + throw err; + } + + const TIMEOUT_MS = TICK_INTERVAL_MS * 3 + 5000; + for (const ch of channels) { + const sidecarUrl = + ch.container_meta && ch.container_meta.sidecar_url + ? ch.container_meta.sidecar_url + : `http://playout-${ch.id}:3002`; + try { + const r = await fetch(`${sidecarUrl}/status`, { signal: AbortSignal.timeout(5000) }); + if (!r.ok) throw new Error(`status HTTP ${r.status}`); + await client.query( + 'UPDATE playout_channels SET last_heartbeat_at = NOW() WHERE id = $1', [ch.id] + ); + // As-run compliance log: the sidecar only tracks the on-air clip locally + // (playout-manager._reportAsRunStart). On every successful status poll we + // detect a clip change here and persist it to playout_as_run — close the + // previous open row and open a new one. Failures are swallowed so a logging + // hiccup never knocks a healthy channel into failover. + try { + const engine = await r.json().catch(() => null); + if (engine) await writeAsRun(client, ch.id, engine); + } catch (e) { + console.warn(`[scheduler] as-run write failed for ${ch.id}: ${e.message}`); + } + } catch (err) { + // When last_heartbeat_at is NULL (channel just spawned), fall back to + // updated_at (set to NOW() by spawnChannelSidecar). This prevents a + // brand-new channel from being failed over on the very first tick because + // epoch-0 age always exceeds TIMEOUT_MS. + const baseline = ch.last_heartbeat_at || ch.updated_at; + const lastSeen = baseline ? new Date(baseline).getTime() : Date.now(); + const ageMs = Date.now() - lastSeen; + if (ageMs < TIMEOUT_MS) continue; // not yet 3 misses + + if (ch.output_type === 'decklink') { + await client.query( + "UPDATE playout_channels SET status = 'error', error_message = $1 WHERE id = $2", + [`sidecar unreachable (${err.message}); decklink channels require manual recovery`, ch.id] + ); + console.error(`[scheduler] decklink channel ${ch.id} unreachable — alert-only, no auto-failover`); + continue; + } + + console.warn(`[scheduler] failover: channel ${ch.id} unreachable (${err.message}), restart #${ch.restart_count + 1}`); + try { + // restartChannel re-places the channel on a healthy node AND spawns the + // new sidecar directly (shared helper) — no /start self-call needed. + const res = await restartChannel(ch.id); + if (res.restarted) { + console.log(`[scheduler] failover: channel ${ch.id} re-placed on node ${res.new_node_id}`); + } else { + console.error(`[scheduler] failover: channel ${ch.id} restart skipped — ${res.reason}`); + } + } catch (err2) { + console.error(`[scheduler] failover error for ${ch.id}: ${err2.message}`); + } + } + } +} + export function startSchedulerLoop() { if (_interval) return; console.log(`[scheduler] tick loop started (interval=${TICK_INTERVAL_MS}ms)`); diff --git a/services/mam-api/test/auth/authz.test.js b/services/mam-api/test/auth/authz.test.js new file mode 100644 index 0000000..4ab34a3 --- /dev/null +++ b/services/mam-api/test/auth/authz.test.js @@ -0,0 +1,125 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; +import { + isAdmin, + accessibleProjectIds, + projectLevel, + assertProjectAccess, +} from '../../src/auth/authz.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +// ── isAdmin (pure, no DB) ─────────────────────────────────────────────────── +test('isAdmin true only for role admin', () => { + assert.equal(isAdmin({ role: 'admin' }), true); + assert.equal(isAdmin({ role: 'editor' }), false); + assert.equal(isAdmin({ role: 'viewer' }), false); + assert.equal(isAdmin(null), false); + assert.equal(isAdmin(undefined), false); +}); + +// Seed helpers shared across the DB-backed cases. +async function seed(pool) { + const proj = async (name) => + (await pool.query( + `INSERT INTO projects (name, s3_prefix) VALUES ($1, $1) RETURNING id`, [name] + )).rows[0].id; + const user = async (username, role) => + (await pool.query( + `INSERT INTO users (username, password_hash, role) VALUES ($1, 'x', $2) RETURNING id`, + [username, role] + )).rows[0].id; + const group = async (name) => + (await pool.query(`INSERT INTO groups (name) VALUES ($1) RETURNING id`, [name])).rows[0].id; + const grantUser = (pid, uid, level) => + pool.query( + `INSERT INTO project_access (project_id, subject_type, subject_id, level) + VALUES ($1, 'user', $2, $3)`, [pid, uid, level]); + const grantGroup = (pid, gid, level) => + pool.query( + `INSERT INTO project_access (project_id, subject_type, subject_id, level) + VALUES ($1, 'group', $2, $3)`, [pid, gid, level]); + const addToGroup = (uid, gid) => + pool.query(`INSERT INTO user_groups (user_id, group_id) VALUES ($1, $2)`, [uid, gid]); + return { proj, user, group, grantUser, grantGroup, addToGroup }; +} + +test('admin sees all projects, every project at edit', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + try { + const s = await seed(pool); + await s.proj('Alpha'); await s.proj('Beta'); + const admin = { id: await s.user('adm', 'admin'), role: 'admin' }; + + const acc = await accessibleProjectIds(admin, pool); + assert.equal(acc.all, true); + assert.equal(await projectLevel(admin, '00000000-0000-4000-8000-000000000001', pool), 'edit'); + await assertProjectAccess(admin, '00000000-0000-4000-8000-000000000001', 'edit', pool); // no throw + } finally { await pool.end(); } +}); + +test('direct user grant scopes access and respects level', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + try { + const s = await seed(pool); + const alpha = await s.proj('Alpha'); + const beta = await s.proj('Beta'); + const u = { id: await s.user('bob', 'editor'), role: 'editor' }; + await s.grantUser(alpha, u.id, 'view'); + + const acc = await accessibleProjectIds(u, pool); + assert.equal(acc.all, false); + assert.deepEqual([...acc.ids], [alpha]); + assert.equal(await projectLevel(u, alpha, pool), 'view'); + assert.equal(await projectLevel(u, beta, pool), null); + + await assertProjectAccess(u, alpha, 'view', pool); // ok + await assert.rejects(() => assertProjectAccess(u, alpha, 'edit', pool), e => e.status === 403); + await assert.rejects(() => assertProjectAccess(u, beta, 'view', pool), e => e.status === 403); + } finally { await pool.end(); } +}); + +test('group grant flows through membership', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + try { + const s = await seed(pool); + const alpha = await s.proj('Alpha'); + const u = { id: await s.user('carol', 'viewer'), role: 'viewer' }; + const g = await s.group('broadcasters'); + await s.addToGroup(u.id, g); + await s.grantGroup(alpha, g, 'edit'); + + assert.equal(await projectLevel(u, alpha, pool), 'edit'); + const acc = await accessibleProjectIds(u, pool); + assert.deepEqual([...acc.ids], [alpha]); + await assertProjectAccess(u, alpha, 'edit', pool); // ok via group + } finally { await pool.end(); } +}); + +test('effective level is the max of direct + group grants', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + try { + const s = await seed(pool); + const alpha = await s.proj('Alpha'); + const u = { id: await s.user('dan', 'editor'), role: 'editor' }; + const g = await s.group('team'); + await s.addToGroup(u.id, g); + await s.grantUser(alpha, u.id, 'view'); // direct: view + await s.grantGroup(alpha, g, 'edit'); // group: edit → wins + + assert.equal(await projectLevel(u, alpha, pool), 'edit'); + } finally { await pool.end(); } +}); + +test('user with no grants sees nothing', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + try { + const s = await seed(pool); + await s.proj('Alpha'); + const u = { id: await s.user('eve', 'viewer'), role: 'viewer' }; + + const acc = await accessibleProjectIds(u, pool); + assert.equal(acc.ids.size, 0); + } finally { await pool.end(); } +}); diff --git a/services/mam-api/test/auth/google-oauth.test.js b/services/mam-api/test/auth/google-oauth.test.js new file mode 100644 index 0000000..26766be --- /dev/null +++ b/services/mam-api/test/auth/google-oauth.test.js @@ -0,0 +1,40 @@ +// Unit tests for the config-gating + domain helpers in google-oauth.js. The +// token-exchange / ID-token-verify path requires Google's servers and is covered +// by manual verification (see .env.example); here we lock down the pure logic +// that decides whether the feature is on and which domain is allowed. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { isConfigured, allowedDomain } from '../../src/auth/google-oauth.js'; + +function withEnv(vars, fn) { + const saved = {}; + for (const k of Object.keys(vars)) { saved[k] = process.env[k]; + if (vars[k] === undefined) delete process.env[k]; else process.env[k] = vars[k]; } + try { return fn(); } + finally { + for (const k of Object.keys(vars)) { + if (saved[k] === undefined) delete process.env[k]; else process.env[k] = saved[k]; + } + } +} + +test('isConfigured is false unless client id, secret, and redirect are all set', () => { + withEnv({ GOOGLE_CLIENT_ID: undefined, GOOGLE_CLIENT_SECRET: undefined, OAUTH_REDIRECT_URL: undefined }, () => { + assert.equal(isConfigured(), false); + }); + withEnv({ GOOGLE_CLIENT_ID: 'x', GOOGLE_CLIENT_SECRET: undefined, OAUTH_REDIRECT_URL: undefined }, () => { + assert.equal(isConfigured(), false); + }); + withEnv({ GOOGLE_CLIENT_ID: 'x', GOOGLE_CLIENT_SECRET: 'y', OAUTH_REDIRECT_URL: undefined }, () => { + assert.equal(isConfigured(), false); + }); + withEnv({ GOOGLE_CLIENT_ID: 'x', GOOGLE_CLIENT_SECRET: 'y', OAUTH_REDIRECT_URL: 'https://h/cb' }, () => { + assert.equal(isConfigured(), true); + }); +}); + +test('allowedDomain normalizes and defaults to null', () => { + withEnv({ GOOGLE_ALLOWED_DOMAIN: undefined }, () => assert.equal(allowedDomain(), null)); + withEnv({ GOOGLE_ALLOWED_DOMAIN: '' }, () => assert.equal(allowedDomain(), null)); + withEnv({ GOOGLE_ALLOWED_DOMAIN: ' WildDragon.NET ' }, () => assert.equal(allowedDomain(), 'wilddragon.net')); +}); diff --git a/services/mam-api/test/auth/mfa-tickets.test.js b/services/mam-api/test/auth/mfa-tickets.test.js new file mode 100644 index 0000000..546a336 --- /dev/null +++ b/services/mam-api/test/auth/mfa-tickets.test.js @@ -0,0 +1,49 @@ +// MFA ticket binding tests — the second login step's tickets are bound to the +// issuing request's IP + User-Agent (hashed) so a stolen ticket replayed from +// a different origin can't complete the second factor. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { issueTicket, redeemTicket } from '../../src/auth/mfa-tickets.js'; + +test('ticket round-trips when ip + userAgent match', () => { + const id = issueTicket('user-1', { ip: '1.2.3.4', userAgent: 'curl/8' }); + assert.equal(redeemTicket(id, { ip: '1.2.3.4', userAgent: 'curl/8' }), 'user-1'); +}); + +test('ticket rejects redemption from a different IP', () => { + const id = issueTicket('user-1', { ip: '1.2.3.4', userAgent: 'curl/8' }); + assert.equal(redeemTicket(id, { ip: '9.9.9.9', userAgent: 'curl/8' }), null); +}); + +test('ticket rejects redemption with a different User-Agent', () => { + const id = issueTicket('user-1', { ip: '1.2.3.4', userAgent: 'curl/8' }); + assert.equal(redeemTicket(id, { ip: '1.2.3.4', userAgent: 'Mozilla/5.0' }), null); +}); + +test('ticket is single-use even on binding mismatch', () => { + // A wrong-binding probe must still burn the ticket — otherwise an attacker + // could try multiple IPs/UAs against the same ticket. + const id = issueTicket('user-1', { ip: '1.2.3.4', userAgent: 'curl/8' }); + assert.equal(redeemTicket(id, { ip: '9.9.9.9', userAgent: 'curl/8' }), null); + // Same ticket with correct bindings now also fails — it was consumed. + assert.equal(redeemTicket(id, { ip: '1.2.3.4', userAgent: 'curl/8' }), null); +}); + +test('redeemTicket returns null for missing or unknown id', () => { + assert.equal(redeemTicket(null), null); + assert.equal(redeemTicket(undefined), null); + assert.equal(redeemTicket(''), null); + assert.equal(redeemTicket('not-a-real-id', { ip: 'x', userAgent: 'y' }), null); +}); + +test('ticket is single-use on success', () => { + const id = issueTicket('user-1', { ip: '1.2.3.4', userAgent: 'curl/8' }); + assert.equal(redeemTicket(id, { ip: '1.2.3.4', userAgent: 'curl/8' }), 'user-1'); + assert.equal(redeemTicket(id, { ip: '1.2.3.4', userAgent: 'curl/8' }), null); +}); + +test('issueTicket without bindings still works (back-compat / tests)', () => { + const id = issueTicket('user-1'); + // No bindings on redeem either — both sides skip the check. + assert.equal(redeemTicket(id), 'user-1'); +}); diff --git a/services/mam-api/test/auth/totp.test.js b/services/mam-api/test/auth/totp.test.js new file mode 100644 index 0000000..f583213 --- /dev/null +++ b/services/mam-api/test/auth/totp.test.js @@ -0,0 +1,96 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { + base32Encode, base32Decode, generateSecret, generateToken, + verifyToken, otpauthURI, generateRecoveryCodes, +} from '../../src/auth/totp.js'; + +// ── base32 round-trips ────────────────────────────────────────────────────── +test('base32 encode/decode round-trips arbitrary bytes', () => { + for (const s of ['', 'f', 'fo', 'foo', 'foob', 'fooba', 'foobar']) { + const buf = Buffer.from(s); + assert.deepEqual(base32Decode(base32Encode(buf)), buf); + } +}); + +// ── RFC 6238 Appendix B test vectors (SHA-1, 8 digits in the RFC; we use the +// low 6 here, so compare the last 6 digits of each published value). ────────── +// The RFC uses the ASCII secret "12345678901234567890". We base32-encode it and +// check the 6-digit code at each published timestamp. +test('matches RFC 6238 SHA-1 vectors (low 6 digits)', () => { + const secret = base32Encode(Buffer.from('12345678901234567890')); + // [unix seconds, full 8-digit code from the RFC] → expect last 6 digits. + const vectors = [ + [59, '94287082'], + [1111111109, '07081804'], + [1111111111, '14050471'], + [1234567890, '89005924'], + [2000000000, '69279037'], + [20000000000, '65353130'], + ]; + for (const [secs, full8] of vectors) { + const got = generateToken(secret, secs * 1000); + assert.equal(got, full8.slice(-6), `t=${secs}`); + } +}); + +// ── verify with drift window ──────────────────────────────────────────────── +// verifyToken returns the matched counter (truthy) or null (falsy). +test('verifyToken accepts the current code and ±1 step of drift', () => { + const secret = generateSecret(); + const now = 1_700_000_000_000; + const code = generateToken(secret, now); + const baseCounter = Math.floor(now / 1000 / 30); + assert.equal(verifyToken(secret, code, now), baseCounter); + // 30s earlier / later still inside ±1 window — the *issued* code matches the + // baseCounter, but at now+30s we're in step baseCounter+1, so the issued + // code matches as drift = -1 step → returns baseCounter. + assert.equal(verifyToken(secret, code, now + 30_000), baseCounter); + assert.equal(verifyToken(secret, code, now - 30_000), baseCounter); + // 2 steps away → rejected. + assert.equal(verifyToken(secret, code, now + 90_000), null); +}); + +test('verifyToken rejects malformed / empty input without throwing', () => { + const secret = generateSecret(); + assert.equal(verifyToken(secret, ''), null); + assert.equal(verifyToken(secret, 'abcdef'), null); + assert.equal(verifyToken(secret, '12345'), null); // too short + assert.equal(verifyToken(secret, '1234567'), null); // too long + assert.equal(verifyToken('', '123456'), null); +}); + +test('verifyToken tolerates spaces in the user-entered code', () => { + const secret = generateSecret(); + const now = 1_700_000_000_000; + const code = generateToken(secret, now); + const expected = Math.floor(now / 1000 / 30); + assert.equal(verifyToken(secret, code.slice(0, 3) + ' ' + code.slice(3), now), expected); +}); + +test('verifyToken returns the matched counter (for replay protection)', () => { + const secret = generateSecret(); + const now = 1_700_000_000_000; + const code = generateToken(secret, now); + const counter = verifyToken(secret, code, now); + assert.ok(typeof counter === 'number' && counter > 0); + assert.equal(counter, Math.floor(now / 1000 / 30)); +}); + +// ── otpauth URI ───────────────────────────────────────────────────────────── +test('otpauthURI embeds secret, issuer, and account', () => { + const uri = otpauthURI('JBSWY3DPEHPK3PXP', 'alice', 'Dragonflight'); + assert.match(uri, /^otpauth:\/\/totp\/Dragonflight%3Aalice\?/); + assert.match(uri, /secret=JBSWY3DPEHPK3PXP/); + assert.match(uri, /issuer=Dragonflight/); + assert.match(uri, /digits=6/); + assert.match(uri, /period=30/); +}); + +// ── recovery codes ────────────────────────────────────────────────────────── +test('generateRecoveryCodes returns N distinct formatted codes', () => { + const codes = generateRecoveryCodes(10); + assert.equal(codes.length, 10); + assert.equal(new Set(codes).size, 10); + for (const c of codes) assert.match(c, /^[0-9a-f]{5}-[0-9a-f]{5}$/); +}); diff --git a/services/mam-api/test/routes/assets-access.test.js b/services/mam-api/test/routes/assets-access.test.js new file mode 100644 index 0000000..99aa598 --- /dev/null +++ b/services/mam-api/test/routes/assets-access.test.js @@ -0,0 +1,79 @@ +// Regression test: GET /api/v1/assets must be scoped to the caller's accessible +// projects. A pre-fix bug returned every asset across every project to any +// authenticated user, defeating RBAC v2. +// +// Like project-access.test.js, the assets router uses the singleton pool, so we +// point DATABASE_URL at TEST_DATABASE_URL and seed through the same pool. +// Skips when TEST_DATABASE_URL is unset. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import express from 'express'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +async function appWithAssets(user) { + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + process.env.AUTH_ENABLED = 'true'; + // Importing the assets router constructs BullMQ queues; they connect lazily, + // and the list route only touches Postgres, so no Redis is needed here. + const { default: assetsRouter } = await import('../../src/routes/assets.js?v=' + Date.now()); + const app = express(); + app.use(express.json()); + app.use((req, _res, next) => { req.user = user; next(); }); + app.use('/api/v1/assets', assetsRouter); + return new Promise(r => { + const srv = app.listen(0, '127.0.0.1', () => + r({ baseUrl: 'http://127.0.0.1:' + srv.address().port, close: () => new Promise(rs => srv.close(rs)) })); + }); +} + +async function seed(pool) { + const proj = async (n) => (await pool.query(`INSERT INTO projects (name, s3_prefix) VALUES ($1,$1) RETURNING id`, [n])).rows[0].id; + const alpha = await proj('Alpha'); + const beta = await proj('Beta'); + const asset = (pid, name) => pool.query( + `INSERT INTO assets (project_id, filename, display_name, media_type, status) + VALUES ($1, $2, $2, 'video', 'ready')`, [pid, name]); + await asset(alpha, 'a1'); await asset(alpha, 'a2'); await asset(beta, 'b1'); + const admin = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('adm','x','admin') RETURNING id`)).rows[0].id, role: 'admin' }; + const scoped = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('vu','x','viewer') RETURNING id`)).rows[0].id, role: 'viewer' }; + await pool.query( + `INSERT INTO project_access (project_id, subject_type, subject_id, level) VALUES ($1,'user',$2,'view')`, + [alpha, scoped.id]); + return { alpha, beta, admin, scoped }; +} + +test('GET /assets returns only assets in granted projects for scoped users', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { admin, scoped } = await seed(pool); + + // Admin sees all three. + let a = await appWithAssets(admin); + let body = await (await fetch(a.baseUrl + '/api/v1/assets')).json(); + assert.equal(body.assets.length, 3, 'admin should see every asset'); + await a.close(); + + // Scoped viewer (granted only Alpha) sees the two Alpha assets, not Beta's. + let s = await appWithAssets(scoped); + body = await (await fetch(s.baseUrl + '/api/v1/assets')).json(); + assert.equal(body.assets.length, 2, 'scoped user should see only granted-project assets'); + assert.ok(body.assets.every(x => x.display_name !== 'b1'), 'must not leak ungranted-project asset'); + await s.close(); + } finally { await pool.end(); } +}); + +test('GET /assets returns nothing for a user with no grants', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + await seed(pool); + const nobody = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('no','x','viewer') RETURNING id`)).rows[0].id, role: 'viewer' }; + const s = await appWithAssets(nobody); + const body = await (await fetch(s.baseUrl + '/api/v1/assets')).json(); + assert.deepEqual(body, { assets: [], total: 0 }); + await s.close(); + } finally { await pool.end(); } +}); diff --git a/services/mam-api/test/routes/comments-access.test.js b/services/mam-api/test/routes/comments-access.test.js new file mode 100644 index 0000000..d056760 --- /dev/null +++ b/services/mam-api/test/routes/comments-access.test.js @@ -0,0 +1,76 @@ +// RBAC coverage for asset comments: the guard resolves the project via the +// asset, requiring view to read and edit to write. Also verifies the author id +// is recorded from req.user (regression for the old req.session.userId bug). +// Skips without TEST_DATABASE_URL. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import express from 'express'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +async function appWithComments(user) { + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + process.env.AUTH_ENABLED = 'true'; + const { default: commentsRouter } = await import('../../src/routes/comments.js?v=' + Date.now()); + const app = express(); + app.use(express.json()); + app.use((req, _res, next) => { req.user = user; next(); }); + // Mirror index.js mount so :assetId flows through (mergeParams in the router). + app.use('/api/v1/assets/:assetId/comments', commentsRouter); + return new Promise(r => { + const srv = app.listen(0, '127.0.0.1', () => + r({ baseUrl: 'http://127.0.0.1:' + srv.address().port, close: () => new Promise(rs => srv.close(rs)) })); + }); +} + +async function seed(pool) { + const proj = async (n) => (await pool.query(`INSERT INTO projects (name, s3_prefix) VALUES ($1,$1) RETURNING id`, [n])).rows[0].id; + const alpha = await proj('Alpha'); + const beta = await proj('Beta'); + const asset = async (pid, name) => (await pool.query( + `INSERT INTO assets (project_id, filename, display_name, media_type, status) VALUES ($1,$2,$2,'video','ready') RETURNING id`, [pid, name])).rows[0].id; + const assetA = await asset(alpha, 'a1'); + const assetB = await asset(beta, 'b1'); + const viewer = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('vu','x','viewer') RETURNING id`)).rows[0].id, role: 'viewer' }; + const editor = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('ed','x','editor') RETURNING id`)).rows[0].id, role: 'editor' }; + await pool.query(`INSERT INTO project_access (project_id, subject_type, subject_id, level) VALUES ($1,'user',$2,'view')`, [alpha, viewer.id]); + await pool.query(`INSERT INTO project_access (project_id, subject_type, subject_id, level) VALUES ($1,'user',$2,'edit')`, [alpha, editor.id]); + return { assetA, assetB, viewer, editor }; +} + +test('comments require view to read and block ungranted assets', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { assetA, assetB, viewer } = await seed(pool); + const s = await appWithComments(viewer); + assert.equal((await fetch(s.baseUrl + '/api/v1/assets/' + assetA + '/comments')).status, 200); + assert.equal((await fetch(s.baseUrl + '/api/v1/assets/' + assetB + '/comments')).status, 403); + await s.close(); + } finally { await pool.end(); } +}); + +test('posting a comment requires edit and records the author id from req.user', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { assetA, viewer, editor } = await seed(pool); + + // viewer (view-only) cannot post. + let s = await appWithComments(viewer); + let r = await fetch(s.baseUrl + '/api/v1/assets/' + assetA + '/comments', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ body: 'hi' }) }); + assert.equal(r.status, 403); + await s.close(); + + // editor can post, and the author id is the editor (not null). + let e = await appWithComments(editor); + r = await fetch(e.baseUrl + '/api/v1/assets/' + assetA + '/comments', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ body: 'looks good' }) }); + assert.equal(r.status, 201); + const created = await r.json(); + assert.equal(created.user_id, editor.id, 'comment author must be req.user.id'); + await e.close(); + } finally { await pool.end(); } +}); diff --git a/services/mam-api/test/routes/google-link.test.js b/services/mam-api/test/routes/google-link.test.js new file mode 100644 index 0000000..9c5e069 --- /dev/null +++ b/services/mam-api/test/routes/google-link.test.js @@ -0,0 +1,63 @@ +// Security regression test for resolveGoogleUser: a Google sign-in must NEVER +// adopt a pre-existing local account by matching email (that would be account +// takeover). It links only by google_sub, otherwise provisions a fresh viewer. +// Skips without TEST_DATABASE_URL. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; +import { hashPassword } from '../../src/auth/passwords.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +async function loadResolve() { + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + return (await import('../../src/routes/auth.js?v=' + Date.now())).resolveGoogleUser; +} + +test('a Google login with an email matching a local admin does NOT take over that account', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + // Pre-existing local admin with a password and the same email the attacker controls. + const adminId = (await pool.query( + `INSERT INTO users (username, password_hash, role, email) + VALUES ('boss', $1, 'admin', 'boss@wilddragon.net') RETURNING id`, + [await hashPassword('a-real-password-12')])).rows[0].id; + + const resolveGoogleUser = await loadResolve(); + const user = await resolveGoogleUser({ sub: 'google-attacker-sub', email: 'boss@wilddragon.net', name: 'Not The Boss' }); + + // Must be a brand-new account, NOT the admin. + assert.notEqual(user.id, adminId, 'Google login must not resolve to the existing admin'); + const row = (await pool.query(`SELECT role, google_sub FROM users WHERE id = $1`, [user.id])).rows[0]; + assert.equal(row.role, 'viewer', 'auto-provisioned account must be a viewer'); + assert.equal(row.google_sub, 'google-attacker-sub'); + // The admin row is untouched (no google_sub linked onto it). + const admin = (await pool.query(`SELECT google_sub FROM users WHERE id = $1`, [adminId])).rows[0]; + assert.equal(admin.google_sub, null, 'the existing admin must not have been linked'); + } finally { await pool.end(); } +}); + +test('a returning Google user resolves to the same account by google_sub', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const resolveGoogleUser = await loadResolve(); + const first = await resolveGoogleUser({ sub: 'sub-123', email: 'alice@wilddragon.net', name: 'Alice' }); + const second = await resolveGoogleUser({ sub: 'sub-123', email: 'alice@wilddragon.net', name: 'Alice' }); + assert.equal(first.id, second.id, 'same google_sub must map to the same user'); + const count = (await pool.query(`SELECT COUNT(*)::int AS n FROM users WHERE google_sub = 'sub-123'`)).rows[0].n; + assert.equal(count, 1, 'must not create a duplicate on second login'); + } finally { await pool.end(); } +}); + +test('username collisions get a numeric suffix', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('sam', 'x', 'viewer')`); + const resolveGoogleUser = await loadResolve(); + const u = await resolveGoogleUser({ sub: 'sub-sam', email: 'sam@wilddragon.net', name: 'Sam' }); + assert.match(u.username, /^sam\d+$/, 'colliding username should get a numeric suffix'); + } finally { await pool.end(); } +}); diff --git a/services/mam-api/test/routes/project-access.test.js b/services/mam-api/test/routes/project-access.test.js new file mode 100644 index 0000000..49d3bf1 --- /dev/null +++ b/services/mam-api/test/routes/project-access.test.js @@ -0,0 +1,125 @@ +// Integration test for per-project RBAC: the grant-management API on the +// projects router + scoped enforcement on GET /projects and GET /projects/:id. +// +// NOTE: the routers use the singleton pool (src/db/pool.js), which reads +// DATABASE_URL. We point DATABASE_URL at the throwaway TEST_DATABASE_URL and +// seed through that same singleton pool so the router and the test share one +// database. Skips cleanly when TEST_DATABASE_URL is unset. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import express from 'express'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +// Build an app that injects a chosen user as req.user (simulating requireAuth), +// then mounts the real projects router with the same admin gate index.js uses. +async function appWithProjects(user) { + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + process.env.AUTH_ENABLED = 'true'; + const { default: projectsRouter } = await import('../../src/routes/projects.js?v=' + Date.now()); + const app = express(); + app.use(express.json()); + app.use((req, _res, next) => { req.user = user; next(); }); + app.use('/api/v1/projects', projectsRouter); + return new Promise(r => { + const srv = app.listen(0, '127.0.0.1', () => + r({ baseUrl: 'http://127.0.0.1:' + srv.address().port, close: () => new Promise(rs => srv.close(rs)) })); + }); +} + +async function seedBaseline(pool) { + const alpha = (await pool.query(`INSERT INTO projects (name, s3_prefix) VALUES ('Alpha','alpha') RETURNING id`)).rows[0].id; + const beta = (await pool.query(`INSERT INTO projects (name, s3_prefix) VALUES ('Beta','beta') RETURNING id`)).rows[0].id; + const admin = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('adm','x','admin') RETURNING id`)).rows[0].id, role: 'admin' }; + const scoped = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('vu','x','viewer') RETURNING id`)).rows[0].id, role: 'viewer' }; + return { alpha, beta, admin, scoped }; +} + +test('admin grants a project, scoped user then sees only it; revoke removes access', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { alpha, beta, admin, scoped } = await seedBaseline(pool); + + // Scoped viewer initially sees nothing. + let s = await appWithProjects(scoped); + let list = await (await fetch(s.baseUrl + '/api/v1/projects')).json(); + assert.equal(list.length, 0, 'scoped user should see no projects before any grant'); + // And cannot read Alpha directly. + let direct = await fetch(s.baseUrl + '/api/v1/projects/' + alpha); + assert.equal(direct.status, 403); + await s.close(); + + // Admin grants the scoped user 'view' on Alpha. + const a = await appWithProjects(admin); + const grant = await fetch(a.baseUrl + '/api/v1/projects/' + alpha + '/access', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ subject_type: 'user', subject_id: scoped.id, level: 'view' }), + }); + assert.equal(grant.status, 201); + const grantList = await (await fetch(a.baseUrl + '/api/v1/projects/' + alpha + '/access')).json(); + assert.equal(grantList.length, 1); + assert.equal(grantList[0].subject_id, scoped.id); + await a.close(); + + // Scoped viewer now sees exactly Alpha (not Beta), can GET it, cannot PATCH (view-only). + s = await appWithProjects(scoped); + list = await (await fetch(s.baseUrl + '/api/v1/projects')).json(); + assert.deepEqual(list.map(p => p.id), [alpha]); + assert.equal((await fetch(s.baseUrl + '/api/v1/projects/' + alpha)).status, 200); + assert.equal((await fetch(s.baseUrl + '/api/v1/projects/' + beta)).status, 403); + const patch = await fetch(s.baseUrl + '/api/v1/projects/' + alpha, { + method: 'PATCH', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ description: 'hacked' }), + }); + assert.equal(patch.status, 403, 'view-level grant must not allow edit'); + await s.close(); + + // Admin revokes; scoped viewer goes back to seeing nothing. + const a2 = await appWithProjects(admin); + const del = await fetch(a2.baseUrl + '/api/v1/projects/' + alpha + '/access/user/' + scoped.id, { method: 'DELETE' }); + assert.equal(del.status, 204); + await a2.close(); + + s = await appWithProjects(scoped); + list = await (await fetch(s.baseUrl + '/api/v1/projects')).json(); + assert.equal(list.length, 0); + await s.close(); + } finally { await pool.end(); } +}); + +test('non-admin cannot reach the grant-management API', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { alpha, scoped } = await seedBaseline(pool); + const s = await appWithProjects(scoped); + // requireAdmin sits on the access sub-routes; a viewer is 403. + const r = await fetch(s.baseUrl + '/api/v1/projects/' + alpha + '/access'); + assert.equal(r.status, 403); + await s.close(); + } finally { await pool.end(); } +}); + +test('edit-level grant allows PATCH', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { alpha, admin, scoped } = await seedBaseline(pool); + const a = await appWithProjects(admin); + await fetch(a.baseUrl + '/api/v1/projects/' + alpha + '/access', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ subject_type: 'user', subject_id: scoped.id, level: 'edit' }), + }); + await a.close(); + + const s = await appWithProjects(scoped); + const patch = await fetch(s.baseUrl + '/api/v1/projects/' + alpha, { + method: 'PATCH', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ description: 'updated by editor' }), + }); + assert.equal(patch.status, 200); + await s.close(); + } finally { await pool.end(); } +}); diff --git a/services/mam-api/test/routes/recorders-access.test.js b/services/mam-api/test/routes/recorders-access.test.js new file mode 100644 index 0000000..ee143ba --- /dev/null +++ b/services/mam-api/test/routes/recorders-access.test.js @@ -0,0 +1,107 @@ +// RBAC coverage for recorders: list is scoped to accessible projects, /:id +// asserts view, mutators assert edit, null-project recorders are admin-only. +// Same harness as assets-access.test.js — singleton pool on TEST_DATABASE_URL, +// req.user injected, router dynamic-imported. Skips without TEST_DATABASE_URL. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import express from 'express'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +async function appWithRecorders(user) { + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + process.env.AUTH_ENABLED = 'true'; + const { default: recordersRouter } = await import('../../src/routes/recorders.js?v=' + Date.now()); + const app = express(); + app.use(express.json()); + app.use((req, _res, next) => { req.user = user; next(); }); + app.use('/api/v1/recorders', recordersRouter); + return new Promise(r => { + const srv = app.listen(0, '127.0.0.1', () => + r({ baseUrl: 'http://127.0.0.1:' + srv.address().port, close: () => new Promise(rs => srv.close(rs)) })); + }); +} + +async function seed(pool) { + const proj = async (n) => (await pool.query(`INSERT INTO projects (name, s3_prefix) VALUES ($1,$1) RETURNING id`, [n])).rows[0].id; + const alpha = await proj('Alpha'); + const beta = await proj('Beta'); + const rec = async (pid, name) => (await pool.query( + `INSERT INTO recorders (name, source_type, project_id) VALUES ($1, 'srt', $2) RETURNING id`, [name, pid])).rows[0].id; + const recA = await rec(alpha, 'Cam A'); + const recB = await rec(beta, 'Cam B'); + const recNull = (await pool.query( + `INSERT INTO recorders (name, source_type, project_id) VALUES ('Unassigned','srt',NULL) RETURNING id`)).rows[0].id; + const admin = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('adm','x','admin') RETURNING id`)).rows[0].id, role: 'admin' }; + const scoped = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('ed','x','editor') RETURNING id`)).rows[0].id, role: 'editor' }; + await pool.query(`INSERT INTO project_access (project_id, subject_type, subject_id, level) VALUES ($1,'user',$2,'view')`, [alpha, scoped.id]); + return { alpha, beta, recA, recB, recNull, admin, scoped }; +} + +test('recorders list is scoped; admin sees all incl. null-project, scoped sees only granted', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { recA, admin, scoped } = await seed(pool); + + let a = await appWithRecorders(admin); + let list = await (await fetch(a.baseUrl + '/api/v1/recorders')).json(); + assert.equal(list.length, 3, 'admin sees all three recorders'); + await a.close(); + + let s = await appWithRecorders(scoped); + list = await (await fetch(s.baseUrl + '/api/v1/recorders')).json(); + assert.deepEqual(list.map(r => r.id), [recA], 'scoped editor sees only the granted-project recorder'); + await s.close(); + } finally { await pool.end(); } +}); + +test('recorder /:id enforces view; mutators enforce edit', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { recA, recB, recNull, scoped } = await seed(pool); + const s = await appWithRecorders(scoped); + + // view-granted on Alpha: can GET recA, cannot GET recB (other project) or recNull (admin-only). + assert.equal((await fetch(s.baseUrl + '/api/v1/recorders/' + recA)).status, 200); + assert.equal((await fetch(s.baseUrl + '/api/v1/recorders/' + recB)).status, 403); + assert.equal((await fetch(s.baseUrl + '/api/v1/recorders/' + recNull)).status, 403); + + // view-only grant cannot PATCH (edit) or start. + const patch = await fetch(s.baseUrl + '/api/v1/recorders/' + recA, { + method: 'PATCH', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ name: 'x' }) }); + assert.equal(patch.status, 403, 'view-level grant must not allow edit'); + await s.close(); + } finally { await pool.end(); } +}); + +test('creating a recorder requires edit; null-project create is admin-only', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { alpha, admin, scoped } = await seed(pool); + + // scoped editor has only 'view' on Alpha → create denied. + let s = await appWithRecorders(scoped); + let r = await fetch(s.baseUrl + '/api/v1/recorders', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ name: 'New', source_type: 'srt', project_id: alpha }) }); + assert.equal(r.status, 403); + // null project → admin-only, still denied for the editor. + r = await fetch(s.baseUrl + '/api/v1/recorders', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ name: 'New2', source_type: 'srt' }) }); + assert.equal(r.status, 403); + await s.close(); + + // admin can create in any project and with no project. + let a = await appWithRecorders(admin); + r = await fetch(a.baseUrl + '/api/v1/recorders', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ name: 'AdminRec', source_type: 'srt' }) }); + assert.equal(r.status, 201); + await a.close(); + } finally { await pool.end(); } +}); diff --git a/services/mam-api/test/routes/sequences-access.test.js b/services/mam-api/test/routes/sequences-access.test.js new file mode 100644 index 0000000..6849977 --- /dev/null +++ b/services/mam-api/test/routes/sequences-access.test.js @@ -0,0 +1,103 @@ +// RBAC coverage for sequences: list/create assert on the query/body project, +// /:id asserts view, mutators assert edit. Skips without TEST_DATABASE_URL. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import express from 'express'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +async function appWithSequences(user) { + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + process.env.AUTH_ENABLED = 'true'; + const { default: sequencesRouter } = await import('../../src/routes/sequences.js?v=' + Date.now()); + const app = express(); + app.use(express.json()); + app.use((req, _res, next) => { req.user = user; next(); }); + app.use('/api/v1/sequences', sequencesRouter); + return new Promise(r => { + const srv = app.listen(0, '127.0.0.1', () => + r({ baseUrl: 'http://127.0.0.1:' + srv.address().port, close: () => new Promise(rs => srv.close(rs)) })); + }); +} + +async function seed(pool) { + const proj = async (n) => (await pool.query(`INSERT INTO projects (name, s3_prefix) VALUES ($1,$1) RETURNING id`, [n])).rows[0].id; + const alpha = await proj('Alpha'); + const beta = await proj('Beta'); + const seqB = (await pool.query(`INSERT INTO sequences (project_id, name) VALUES ($1,'B seq') RETURNING id`, [beta])).rows[0].id; + const viewer = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('vu','x','viewer') RETURNING id`)).rows[0].id, role: 'viewer' }; + const editor = { id: (await pool.query(`INSERT INTO users (username, password_hash, role) VALUES ('ed','x','editor') RETURNING id`)).rows[0].id, role: 'editor' }; + await pool.query(`INSERT INTO project_access (project_id, subject_type, subject_id, level) VALUES ($1,'user',$2,'view')`, [alpha, viewer.id]); + await pool.query(`INSERT INTO project_access (project_id, subject_type, subject_id, level) VALUES ($1,'user',$2,'edit')`, [alpha, editor.id]); + return { alpha, beta, seqB, viewer, editor }; +} + +const asset = (pool, pid, name) => pool.query( + `INSERT INTO assets (project_id, filename, display_name, media_type, status) VALUES ($1,$2,$2,'video','ready') RETURNING id`, + [pid, name]).then(r => r.rows[0].id); + +test('GET /sequences?project_id 403s on an ungranted project', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { alpha, beta, viewer } = await seed(pool); + const s = await appWithSequences(viewer); + assert.equal((await fetch(s.baseUrl + '/api/v1/sequences?project_id=' + alpha)).status, 200); + assert.equal((await fetch(s.baseUrl + '/api/v1/sequences?project_id=' + beta)).status, 403); + await s.close(); + } finally { await pool.end(); } +}); + +test('POST /sequences requires edit; viewer denied, editor allowed; /:id view enforced', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { alpha, seqB, viewer, editor } = await seed(pool); + + // viewer (view-only on Alpha) cannot create. + let s = await appWithSequences(viewer); + let r = await fetch(s.baseUrl + '/api/v1/sequences', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ project_id: alpha, name: 'X' }) }); + assert.equal(r.status, 403); + // viewer cannot read a sequence in an ungranted project (Beta). + assert.equal((await fetch(s.baseUrl + '/api/v1/sequences/' + seqB)).status, 403); + await s.close(); + + // editor can create in Alpha and then PUT it. + let e = await appWithSequences(editor); + r = await fetch(e.baseUrl + '/api/v1/sequences', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ project_id: alpha, name: 'Mine' }) }); + assert.equal(r.status, 201); + const seqId = (await r.json()).id; + const put = await fetch(e.baseUrl + '/api/v1/sequences/' + seqId, { + method: 'PUT', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ name: 'Renamed' }) }); + assert.equal(put.status, 200); + await e.close(); + } finally { await pool.end(); } +}); + +test('PUT /:id/clips rejects assets from outside the sequence project (cross-project leak guard)', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + const { alpha, beta, editor } = await seed(pool); + const seqA = (await pool.query(`INSERT INTO sequences (project_id, name) VALUES ($1,'A seq') RETURNING id`, [alpha])).rows[0].id; + const assetA = await asset(pool, alpha, 'a-clip'); + const assetB = await asset(pool, beta, 'b-clip'); // editor has NO access to project Beta + + const e = await appWithSequences(editor); + const clip = (aid) => ({ asset_id: aid, track: 0, timeline_in_frames: 0, timeline_out_frames: 10, source_in_frames: 0, source_out_frames: 10 }); + + // Same-project asset is accepted. + let r = await fetch(e.baseUrl + '/api/v1/sequences/' + seqA + '/clips', { + method: 'PUT', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify([clip(assetA)]) }); + assert.equal(r.status, 200); + + // Splicing in a Beta asset must be rejected — it would leak B's media via GET /:id. + r = await fetch(e.baseUrl + '/api/v1/sequences/' + seqA + '/clips', { + method: 'PUT', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify([clip(assetA), clip(assetB)]) }); + assert.equal(r.status, 400, 'foreign-project asset must be rejected'); + await e.close(); + } finally { await pool.end(); } +}); diff --git a/services/mam-api/test/routes/totp.test.js b/services/mam-api/test/routes/totp.test.js new file mode 100644 index 0000000..84f5c4c --- /dev/null +++ b/services/mam-api/test/routes/totp.test.js @@ -0,0 +1,143 @@ +// Integration test for the TOTP two-step login + recovery codes. +// +// Mounts the real auth router with a session store on the throwaway test DB. +// Drives: enroll (setup → enable) → logout → password login returns mfa_required +// → complete with a generated code → and the recovery-code single-use path. +// Skips when TEST_DATABASE_URL is unset. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import express from 'express'; +import session from 'express-session'; +import { isTestDbConfigured, setupTestDb } from '../helpers/setup-db.js'; +import { hashPassword } from '../../src/auth/passwords.js'; +import { generateToken } from '../../src/auth/totp.js'; + +const SKIP = !isTestDbConfigured() && 'TEST_DATABASE_URL not set'; + +async function appWithAuth(pool) { + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + process.env.AUTH_ENABLED = 'true'; + process.env.SESSION_SECRET = 'test'; + const ConnectPg = (await import('connect-pg-simple')).default(session); + const { default: authRouter } = await import('../../src/routes/auth.js?v=' + Date.now()); + const app = express(); + app.use(express.json()); + app.use(session({ + store: new ConnectPg({ pool, tableName: 'sessions' }), + secret: 'test', name: 'dragonflight.sid', + cookie: { httpOnly: true, sameSite: 'lax', secure: false, maxAge: 8 * 3600 * 1000 }, + rolling: false, resave: false, saveUninitialized: false, + })); + app.use('/api/v1/auth', authRouter); + return new Promise(r => { + const srv = app.listen(0, '127.0.0.1', () => + r({ baseUrl: 'http://127.0.0.1:' + srv.address().port, close: () => new Promise(rs => srv.close(rs)) })); + }); +} + +const J = (cookie, body) => ({ + method: 'POST', + headers: { 'Content-Type': 'application/json', ...(cookie ? { cookie } : {}) }, + body: JSON.stringify(body), +}); + +async function loginPassword(baseUrl, username, password) { + const r = await fetch(baseUrl + '/api/v1/auth/login', J(null, { username, password })); + const cookie = (r.headers.get('set-cookie') || '').split(';')[0]; + return { r, body: await r.json().catch(() => ({})), cookie }; +} + +test('enable TOTP, then password login requires a second factor', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + await pool.query(`INSERT INTO users (username, password_hash) VALUES ('alice', $1)`, [await hashPassword('correct-horse-battery')]); + const { baseUrl, close } = await appWithAuth(pool); + + // 1. Password login (no TOTP yet) → 200 with a session cookie. + const first = await loginPassword(baseUrl, 'alice', 'correct-horse-battery'); + assert.equal(first.r.status, 200); + assert.ok(!first.body.mfa_required); + const cookie = first.cookie; + + // 2. Enroll: setup returns a secret; enable confirms with a live code. + const setup = await (await fetch(baseUrl + '/api/v1/auth/totp/setup', J(cookie, {}))).json(); + assert.match(setup.secret, /^[A-Z2-7]+$/); + const enableRes = await fetch(baseUrl + '/api/v1/auth/totp/enable', J(cookie, { code: generateToken(setup.secret) })); + assert.equal(enableRes.status, 200); + const enableBody = await enableRes.json(); + assert.equal(enableBody.enabled, true); + assert.equal(enableBody.recovery_codes.length, 10); + + // 3. Fresh password login now returns mfa_required + a ticket, NO session cookie. + const second = await loginPassword(baseUrl, 'alice', 'correct-horse-battery'); + assert.equal(second.r.status, 200); + assert.equal(second.body.mfa_required, true); + assert.ok(second.body.ticket); + assert.ok(!second.cookie, 'no session cookie should be set before the second factor'); + + // 4. Wrong code → 401; the ticket is now spent. + const bad = await fetch(baseUrl + '/api/v1/auth/login/totp', J(null, { ticket: second.body.ticket, code: '000000' })); + assert.equal(bad.status, 401); + + // 5. New login + correct code → 200 with a session cookie. + const third = await loginPassword(baseUrl, 'alice', 'correct-horse-battery'); + const ok = await fetch(baseUrl + '/api/v1/auth/login/totp', J(null, { ticket: third.body.ticket, code: generateToken(setup.secret) })); + assert.equal(ok.status, 200); + assert.match(ok.headers.get('set-cookie') || '', /dragonflight\.sid=/); + + await close(); + } finally { await pool.end(); } +}); + +test('a recovery code logs in once and cannot be reused', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + await pool.query(`INSERT INTO users (username, password_hash) VALUES ('bob', $1)`, [await hashPassword('correct-horse-battery')]); + const { baseUrl, close } = await appWithAuth(pool); + + const first = await loginPassword(baseUrl, 'bob', 'correct-horse-battery'); + const setup = await (await fetch(baseUrl + '/api/v1/auth/totp/setup', J(first.cookie, {}))).json(); + const enableBody = await (await fetch(baseUrl + '/api/v1/auth/totp/enable', J(first.cookie, { code: generateToken(setup.secret) }))).json(); + const recovery = enableBody.recovery_codes[0]; + + // Use a recovery code to complete a fresh login. + const login1 = await loginPassword(baseUrl, 'bob', 'correct-horse-battery'); + const use1 = await fetch(baseUrl + '/api/v1/auth/login/totp', J(null, { ticket: login1.body.ticket, code: recovery })); + assert.equal(use1.status, 200, 'recovery code should complete login once'); + + // The same recovery code must NOT work a second time. + const login2 = await loginPassword(baseUrl, 'bob', 'correct-horse-battery'); + const use2 = await fetch(baseUrl + '/api/v1/auth/login/totp', J(null, { ticket: login2.body.ticket, code: recovery })); + assert.equal(use2.status, 401, 'a spent recovery code must be rejected'); + + await close(); + } finally { await pool.end(); } +}); + +test('disable TOTP returns login to single-factor', { skip: SKIP }, async () => { + const pool = await setupTestDb(); + process.env.DATABASE_URL = process.env.TEST_DATABASE_URL; + try { + await pool.query(`INSERT INTO users (username, password_hash) VALUES ('carol', $1)`, [await hashPassword('correct-horse-battery')]); + const { baseUrl, close } = await appWithAuth(pool); + + const first = await loginPassword(baseUrl, 'carol', 'correct-horse-battery'); + const setup = await (await fetch(baseUrl + '/api/v1/auth/totp/setup', J(first.cookie, {}))).json(); + await fetch(baseUrl + '/api/v1/auth/totp/enable', J(first.cookie, { code: generateToken(setup.secret) })); + + // Disabling requires the password. + const wrongPw = await fetch(baseUrl + '/api/v1/auth/totp/disable', J(first.cookie, { password: 'nope' })); + assert.equal(wrongPw.status, 400); + const disabled = await fetch(baseUrl + '/api/v1/auth/totp/disable', J(first.cookie, { password: 'correct-horse-battery' })); + assert.equal(disabled.status, 204); + + // Password login is single-factor again. + const relog = await loginPassword(baseUrl, 'carol', 'correct-horse-battery'); + assert.equal(relog.r.status, 200); + assert.ok(!relog.body.mfa_required); + + await close(); + } finally { await pool.end(); } +}); diff --git a/services/playout/Dockerfile b/services/playout/Dockerfile new file mode 100644 index 0000000..d82953f --- /dev/null +++ b/services/playout/Dockerfile @@ -0,0 +1,109 @@ +# Wild Dragon Playout sidecar — CasparCG Server + Node AMCP control shim. +# +# CasparCG's mixer needs an OpenGL context. On a node with a real GPU we'd pass +# the device + driver through; for the headless / no-GPU case we run a virtual +# framebuffer (Xvfb) so the GL context initialises. The container is launched +# --privileged by mam-api (same as capture) so DeckLink / NDI hardware is +# reachable when present. +# +# CasparCG 2.4.x no longer ships a self-contained Linux tarball — the GitHub +# release provides either Ubuntu .deb packages or an "ubuntu22" zip that bundles +# the binary + its .so files under bin/ and lib/. We use the zip on an +# ubuntu:22.04 base so the bundled libs match the host glibc/abi, then install +# Node 20 from NodeSource on top. +# +# NDI + DeckLink SDKs are NOT redistributable. They are fetched at build time +# from a URL supplied as a build arg (mirror it into your own artifact store); +# the build still succeeds without it (NDI/DeckLink consumers simply won't be +# available — SRT/RTMP/test output still work). + +FROM ubuntu:22.04 + +ARG CASPAR_VERSION=2.4.0-stable +ARG CASPAR_URL=https://github.com/CasparCG/server/releases/download/v2.4.0-stable/casparcg-server-v2.4.0-stable-ubuntu22.zip +ARG NDI_SDK_URL= + +ENV DEBIAN_FRONTEND=noninteractive + +# CasparCG 2.4 runtime deps + Xvfb for headless GL + CEF (HTML producer) deps + +# Node 20 (NodeSource). +# +# NOTE: we deliberately do NOT `apt-get install ffmpeg`. That package drags in +# ~80 transitive shared libraries (libav*, libx264, libdrm, libva, ...) that +# perturb CasparCG 2.4.0's runtime linking and make its headless startup abort +# with SIGABRT (exit 134) on nearly every launch. A self-contained STATIC +# ffmpeg binary (installed below) gives us the standalone CLI the preview +# re-muxer needs with ZERO new shared libs, keeping CasparCG's environment +# identical to the known-good image. +RUN apt-get update && apt-get install -y --no-install-recommends \ + ca-certificates curl unzip tar xz-utils gnupg \ + xvfb libgl1-mesa-dri libglu1-mesa fonts-dejavu-core \ + libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 \ + libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 \ + libgbm1 libpango-1.0-0 libcairo2 libasound2 libatspi2.0-0 \ + && mkdir -p /etc/apt/keyrings \ + && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key \ + | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \ + && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" \ + > /etc/apt/sources.list.d/nodesource.list \ + && apt-get update && apt-get install -y --no-install-recommends nodejs \ + && rm -rf /var/lib/apt/lists/* + +# ── Standalone STATIC ffmpeg CLI (for the HLS preview re-muxer) ─────────────── +# john van sickle's static build is fully self-contained (no shared-lib deps), +# so it can't perturb CasparCG's runtime linking. Override FFMPEG_URL to mirror +# this into your own artifact store if upstream availability is a concern. +ARG FFMPEG_URL=https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz +RUN set -eux; \ + curl -fsSL "$FFMPEG_URL" -o /tmp/ffmpeg.tar.xz; \ + mkdir -p /tmp/ffmpeg; \ + tar xJf /tmp/ffmpeg.tar.xz -C /tmp/ffmpeg --strip-components=1; \ + cp /tmp/ffmpeg/ffmpeg /tmp/ffmpeg/ffprobe /usr/local/bin/; \ + chmod +x /usr/local/bin/ffmpeg /usr/local/bin/ffprobe; \ + rm -rf /tmp/ffmpeg /tmp/ffmpeg.tar.xz; \ + /usr/local/bin/ffmpeg -version | head -1 + +# ── CasparCG Server (ubuntu22 zip bundle) ──────────────────────────────────── +# The zip extracts to /opt/casparcg_server with the binary at bin/casparcg and +# its bundled .so files under lib/ (added to LD_LIBRARY_PATH by entrypoint.sh). +# Symlink to /opt/casparcg so the config/entrypoint paths stay stable. +WORKDIR /tmp/caspar +RUN set -eux; \ + curl -fsSL "$CASPAR_URL" -o caspar.zip; \ + unzip -q caspar.zip -d /opt; \ + chmod +x /opt/casparcg_server/bin/casparcg /opt/casparcg_server/scanner 2>/dev/null || true; \ + ls /opt/casparcg_server/; \ + test -x /opt/casparcg_server/bin/casparcg; \ + ln -sfn /opt/casparcg_server /opt/casparcg; \ + echo "caspar binary: /opt/casparcg_server/bin/casparcg"; \ + cd /; rm -rf /tmp/caspar + +# ── NDI runtime (optional) ─────────────────────────────────────────────────── +# If an NDI SDK tarball URL is provided, extract its libs to /opt/ndi-lib and +# point CasparCG at them via NDI_RUNTIME_DIR_V6. Pin the SDK version to what the +# server expects (the common docker failure is a libndi .so version mismatch). +RUN if [ -n "$NDI_SDK_URL" ]; then \ + mkdir -p /opt/ndi-lib && \ + curl -fsSL "$NDI_SDK_URL" -o /tmp/ndi.tar.gz && \ + tar xzf /tmp/ndi.tar.gz -C /tmp && \ + find /tmp -name 'libndi*.so*' -exec cp -a {} /opt/ndi-lib/ \; && \ + rm -f /tmp/ndi.tar.gz && ldconfig /opt/ndi-lib || true; \ + fi +ENV NDI_RUNTIME_DIR_V6=/opt/ndi-lib + +# CasparCG media folder — mam-api stages assets from S3 into this volume. +RUN mkdir -p /media + +# ── Node control shim ──────────────────────────────────────────────────────── +WORKDIR /app +COPY package*.json ./ +RUN npm install --omit=dev +COPY . . + +# CasparCG config + entrypoint +COPY casparcg.config /opt/casparcg/casparcg.config +COPY entrypoint.sh /entrypoint.sh +RUN chmod +x /entrypoint.sh + +EXPOSE 3002 5250 +ENTRYPOINT ["/entrypoint.sh"] diff --git a/services/playout/casparcg.config b/services/playout/casparcg.config new file mode 100644 index 0000000..5fc806a --- /dev/null +++ b/services/playout/casparcg.config @@ -0,0 +1,22 @@ + + + + /media/ + /media/casparcg/log/ + /media/casparcg/data/ + /media/templates/ + + + + 1080i5994 + + + + + + + 5250 + AMCP + + + diff --git a/services/playout/entrypoint.sh b/services/playout/entrypoint.sh new file mode 100644 index 0000000..7cde7f1 --- /dev/null +++ b/services/playout/entrypoint.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Headless GL: start a virtual framebuffer unless a real DISPLAY is provided +# (a GPU node may pass through an X socket). CasparCG's mixer needs a GL context. +if [ -z "${DISPLAY:-}" ]; then + echo "[entrypoint] starting Xvfb on :99" + Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp & + export DISPLAY=:99 + for i in $(seq 1 20); do + [ -e /tmp/.X11-unix/X99 ] && break + sleep 0.25 + done +fi + +# Ensure the HLS preview directory exists before the re-mux ffmpeg writes to it +# (mam-api serves /live//* from the shared media volume). +if [ -n "${CHANNEL_ID:-}" ]; then + mkdir -p "/media/live/${CHANNEL_ID}" +fi + +mkdir -p /media/casparcg/log /media/casparcg/data /media/templates + +# CEF (HTML producer) initialises an NSS database at /root/.pki/nssdb and +# Chrome caches under HOME. Pre-create writable dirs so CEF doesn't SIGABRT +# ~30s into the run when it first lazily inits. +mkdir -p /root/.pki/nssdb /root/.cache /tmp/cef-cache +chmod 700 /root/.pki/nssdb +export HOME=/root + +# 2.4.x zip bundles its own .so files under lib/ — add to LD_LIBRARY_PATH. +export LD_LIBRARY_PATH="/opt/casparcg/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" + +cd /opt/casparcg +CASPAR_CFG=/opt/casparcg/casparcg.config +# 2.4.x: binary at bin/casparcg. 2.5.x: symlinked to casparcg at root. +if [ -x "./bin/casparcg" ]; then CASPAR_BIN="./bin/casparcg"; +elif [ -x "./casparcg" ]; then CASPAR_BIN="./casparcg"; +elif [ -x "./CasparCG Server" ]; then CASPAR_BIN="./CasparCG Server"; +elif command -v casparcg >/dev/null; then CASPAR_BIN="casparcg"; +else echo "[entrypoint] ERROR: casparcg binary not found"; exit 1; fi +echo "[entrypoint] launching CasparCG: $CASPAR_BIN $CASPAR_CFG" +"$CASPAR_BIN" "$CASPAR_CFG" & +CASPAR_PID=$! + +term() { + echo "[entrypoint] terminating CasparCG ($CASPAR_PID)" + kill -TERM "$CASPAR_PID" 2>/dev/null || true + wait "$CASPAR_PID" 2>/dev/null || true + exit 0 +} +trap term SIGTERM SIGINT + +cd /app +node src/index.js & +NODE_PID=$! + +wait -n "$CASPAR_PID" "$NODE_PID" +term diff --git a/services/playout/package.json b/services/playout/package.json new file mode 100644 index 0000000..fff4480 --- /dev/null +++ b/services/playout/package.json @@ -0,0 +1,18 @@ +{ + "name": "wild-dragon-playout", + "version": "1.0.0", + "description": "Wild Dragon MAM playout sidecar — wraps a CasparCG Server instance and drives it over AMCP for master-control playout (SDI / NDI / SRT / RTMP).", + "type": "module", + "main": "src/index.js", + "scripts": { + "start": "node src/index.js" + }, + "engines": { + "node": ">=18" + }, + "dependencies": { + "express": "^4.18.0", + "cors": "^2.8.0", + "dotenv": "^16.4.0" + } +} diff --git a/services/playout/src/amcp.js b/services/playout/src/amcp.js new file mode 100644 index 0000000..d60754a --- /dev/null +++ b/services/playout/src/amcp.js @@ -0,0 +1,182 @@ +import net from 'node:net'; + +// Minimal AMCP (Advanced Media Control Protocol) client for CasparCG. +// +// AMCP is a line-based TCP protocol: each command is a single CRLF-terminated +// line, and the server replies with a status line ("201 PLAY OK\r\n") optionally +// followed by data lines. We keep one persistent socket per CasparCG instance +// and serialize commands through a FIFO queue — CasparCG processes one command +// at a time per connection, so interleaving replies would otherwise be +// ambiguous. +// +// We only implement the subset the playout sidecar needs (PLAY / LOADBG / STOP / +// CLEAR / INFO / ADD / REMOVE). Responses are returned raw; callers parse the +// status code where they care. + +const CRLF = '\r\n'; + +export class AmcpClient { + constructor({ host = '127.0.0.1', port = 5250 } = {}) { + this.host = host; + this.port = port; + this.socket = null; + this.connected = false; + this._buffer = ''; + this._queue = []; // pending { command, resolve, reject, timer } + this._active = null; // command currently awaiting a reply + this._reconnectTimer = null; + } + + connect() { + if (this.socket) return; + const socket = net.createConnection({ host: this.host, port: this.port }); + socket.setEncoding('utf8'); + socket.setKeepAlive(true, 10000); + + socket.on('connect', () => { + this.connected = true; + console.log(`[amcp] connected to ${this.host}:${this.port}`); + }); + socket.on('data', (chunk) => this._onData(chunk)); + socket.on('error', (err) => { + console.error(`[amcp] socket error: ${err.message}`); + }); + socket.on('close', () => { + this.connected = false; + this.socket = null; + // Fail any in-flight + queued commands so callers don't hang. + const pending = this._active ? [this._active, ...this._queue] : [...this._queue]; + this._active = null; + this._queue = []; + for (const p of pending) { + clearTimeout(p.timer); + p.reject(new Error('AMCP connection closed')); + } + this._scheduleReconnect(); + }); + + this.socket = socket; + } + + _scheduleReconnect() { + if (this._reconnectTimer) return; + this._reconnectTimer = setTimeout(() => { + this._reconnectTimer = null; + console.log('[amcp] reconnecting...'); + this.connect(); + }, 2000); + } + + // Wait until the socket is usable, up to timeoutMs. + async waitReady(timeoutMs = 30000) { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (this.connected) return true; + if (!this.socket) this.connect(); + await new Promise((r) => setTimeout(r, 250)); + } + throw new Error('AMCP not ready within timeout'); + } + + _onData(chunk) { + this._buffer += chunk; + // A CasparCG reply is a status line, optionally followed by data lines. + // The simplest robust framing: a command's reply is complete when we see a + // status line AND (for 2-line "200" multi-line replies) the terminating + // blank line. For our command subset, single-status-line replies dominate; + // we treat a reply as complete at each newline and let the active command + // decide whether it has enough. To keep this correct for INFO (multi-line), + // we accumulate until the buffer ends with a known terminator. + if (!this._active) { + // Unsolicited data (e.g. connection banner) — discard. + this._buffer = ''; + return; + } + // CasparCG ends multi-line replies with CRLF on an empty line. Single-line + // replies (201/202/4xx/5xx) end with a single CRLF. Resolve when we have at + // least one complete line; for "200 ... OK" (list follows) wait for the + // blank-line terminator. + const firstLineEnd = this._buffer.indexOf(CRLF); + if (firstLineEnd === -1) return; + const statusLine = this._buffer.slice(0, firstLineEnd); + const code = parseInt(statusLine, 10); + + if (code === 200) { + // Multi-line: data lines until an empty line. + const term = this._buffer.indexOf(CRLF + CRLF); + if (term === -1) return; // wait for more + const full = this._buffer.slice(0, term); + this._buffer = this._buffer.slice(term + 4); + this._finishActive(null, full); + return; + } + + if (code === 201 || code === 202) { + // 201: one data line follows the status line. 202: status only. + if (code === 201) { + const secondLineEnd = this._buffer.indexOf(CRLF, firstLineEnd + 2); + if (secondLineEnd === -1) return; + const full = this._buffer.slice(0, secondLineEnd); + this._buffer = this._buffer.slice(secondLineEnd + 2); + this._finishActive(null, full); + } else { + const full = this._buffer.slice(0, firstLineEnd); + this._buffer = this._buffer.slice(firstLineEnd + 2); + this._finishActive(null, full); + } + return; + } + + // 4xx / 5xx error, or any other single-line status. + const full = this._buffer.slice(0, firstLineEnd); + this._buffer = this._buffer.slice(firstLineEnd + 2); + if (code >= 400) this._finishActive(new Error(`AMCP error: ${full}`), full); + else this._finishActive(null, full); + } + + _finishActive(err, data) { + const active = this._active; + this._active = null; + if (active) { + clearTimeout(active.timer); + if (err) active.reject(err); + else active.resolve(data); + } + this._pump(); + } + + _pump() { + if (this._active || this._queue.length === 0) return; + const next = this._queue.shift(); + this._active = next; + try { + this.socket.write(next.command + CRLF); + } catch (err) { + this._active = null; + clearTimeout(next.timer); + next.reject(err); + } + } + + // Send a single AMCP command and resolve with the raw reply string. + send(command, { timeoutMs = 15000 } = {}) { + return new Promise((resolve, reject) => { + const entry = { command, resolve, reject, timer: null }; + entry.timer = setTimeout(() => { + // Drop from queue if still pending; if active, detach so the next + // reply doesn't get misrouted. + if (this._active === entry) this._active = null; + else this._queue = this._queue.filter((e) => e !== entry); + reject(new Error(`AMCP command timed out: ${command}`)); + }, timeoutMs); + this._queue.push(entry); + this._pump(); + }); + } + + close() { + if (this._reconnectTimer) { clearTimeout(this._reconnectTimer); this._reconnectTimer = null; } + if (this.socket) { try { this.socket.destroy(); } catch (_) {} this.socket = null; } + this.connected = false; + } +} diff --git a/services/playout/src/index.js b/services/playout/src/index.js new file mode 100644 index 0000000..accbd2d --- /dev/null +++ b/services/playout/src/index.js @@ -0,0 +1,85 @@ +import express from 'express'; +import cors from 'cors'; +import dotenv from 'dotenv'; +import playoutManager from './playout-manager.js'; + +dotenv.config(); + +const app = express(); +const PORT = process.env.PORT || 3002; + +app.use(cors()); +app.use(express.json()); + +app.get('/health', (req, res) => res.json({ status: 'ok' })); + +// Start the channel's output consumer. Body: { outputType, outputConfig, videoFormat } +app.post('/channel/start', async (req, res) => { + try { + const out = await playoutManager.startChannel(req.body || {}); + res.json(out); + } catch (err) { + console.error('[playout] /channel/start error:', err.message); + res.status(500).json({ error: err.message }); + } +}); + +app.post('/channel/stop', async (req, res) => { + try { res.json(await playoutManager.stopChannel()); } + catch (err) { res.status(500).json({ error: err.message }); } +}); + +// Load + start a playlist. Body: { items: [...], loop } +app.post('/playlist/load', async (req, res) => { + try { + const { items = [], loop = false } = req.body || {}; + res.json(await playoutManager.loadPlaylist({ items, loop })); + } catch (err) { + console.error('[playout] /playlist/load error:', err.message); + res.status(500).json({ error: err.message }); + } +}); + +app.post('/transport/skip', async (req, res) => { try { res.json(await playoutManager.skip()); } catch (e) { res.status(500).json({ error: e.message }); } }); +app.post('/transport/pause', async (req, res) => { try { res.json(await playoutManager.pause()); } catch (e) { res.status(500).json({ error: e.message }); } }); +app.post('/transport/resume', async (req, res) => { try { res.json(await playoutManager.resume()); } catch (e) { res.status(500).json({ error: e.message }); } }); + +app.get('/status', (req, res) => res.json(playoutManager.getStatus())); + +// Auto-start: when the sidecar is spawned by mam-api with channel env, bring up +// the output consumer immediately so the container is "on air idle" (black/slate) +// the moment it boots, mirroring the capture sidecar's bootstrap pattern. +async function bootstrap() { + const outputType = process.env.OUTPUT_TYPE; + if (!outputType) { + console.log('[bootstrap] no OUTPUT_TYPE — on-demand sidecar, waiting for /channel/start'); + return; + } + let outputConfig = {}; + try { outputConfig = JSON.parse(process.env.OUTPUT_CONFIG || '{}'); } + catch (err) { console.error('[bootstrap] bad OUTPUT_CONFIG json:', err.message); } + const videoFormat = process.env.VIDEO_FORMAT || '1080i5994'; + try { + await playoutManager.startChannel({ outputType, outputConfig, videoFormat }); + } catch (err) { + console.error('[bootstrap] channel start failed:', err.message); + } +} + +const server = app.listen(PORT, () => { + console.log(`Wild Dragon Playout Service listening on port ${PORT}`); + // Give CasparCG a moment to come up (started by the container entrypoint). + playoutManager.amcp.connect(); + bootstrap(); +}); + +function shutdown(sig) { + console.log(`[playout] ${sig} — shutting down`); + playoutManager.stopChannel().catch(() => {}).finally(() => { + playoutManager.amcp.close(); + server.close(() => process.exit(0)); + setTimeout(() => process.exit(0), 5000); + }); +} +process.on('SIGTERM', () => shutdown('SIGTERM')); +process.on('SIGINT', () => shutdown('SIGINT')); diff --git a/services/playout/src/playout-manager.js b/services/playout/src/playout-manager.js new file mode 100644 index 0000000..c846483 --- /dev/null +++ b/services/playout/src/playout-manager.js @@ -0,0 +1,444 @@ +import { AmcpClient } from './amcp.js'; +import { spawn } from 'node:child_process'; +import { mkdirSync } from 'node:fs'; + +// Playout manager — owns one CasparCG channel's lifecycle inside this sidecar. +// +// One sidecar container == one CasparCG Server == one logical channel (channel +// index 1 in CasparCG terms). We add the output consumer (DeckLink / NDI / SRT +// / RTMP) at start, then walk a playlist by cueing the next clip on a background +// layer (LOADBG ... AUTO) so CasparCG performs a gapless transition at end of +// the current clip. +// +// Media is referenced by a path relative to CasparCG's configured media folder +// (/media inside the container). The mam-api stages assets from S3 to that +// shared volume and passes the resolved relative path on each item. + +const CHANNEL = 1; // single CasparCG channel per sidecar +const FG_LAYER = 10; // foreground (on-air) layer +const MEDIA_ROOT = process.env.CASPAR_MEDIA_ROOT || '/media'; + +// Channel-id-derived HLS preview path. The mam-api proxies /live// +// to this directory (shared media volume) so the UI's existing HLS player +// (capture's /live/ plumbing) works for playout monitors with zero new +// transport. +const CHANNEL_ID = process.env.CHANNEL_ID || ''; +const HLS_DIR = CHANNEL_ID ? `${MEDIA_ROOT}/live/${CHANNEL_ID}` : ''; + +// Loopback UDP port CasparCG's preview STREAM consumer publishes mpegts to, and +// the standalone ffmpeg re-muxer reads from. One CasparCG per sidecar, so a +// fixed port is fine; allow override for parallel local testing. +const PREVIEW_UDP_PORT = parseInt(process.env.PREVIEW_UDP_PORT || '9710', 10); +const PREVIEW_UDP_URL = `udp://127.0.0.1:${PREVIEW_UDP_PORT}`; + +// CasparCG SEEK / LENGTH are in frames, not seconds. Capture standard is 59.94; +// SD/film modes need their own values. Default 60000/1001 matches both +// '1080p5994' and '1080i5994'. +function fpsFor(videoFormat) { + const f = String(videoFormat || '').toLowerCase(); + if (f.endsWith('5994')) return 60000 / 1001; + if (f.endsWith('p60') || f.endsWith('i60')) return 60; + if (f.endsWith('p50') || f.endsWith('i50')) return 50; + if (f.endsWith('2997')) return 30000 / 1001; + if (f.endsWith('p30')) return 30; + if (f.endsWith('p25')) return 25; + if (f.endsWith('p24') || f.endsWith('2398')) return 24000 / 1001; + return 60000 / 1001; // safe default for the house standard +} + +// CasparCG transition syntax fragments keyed by our item.transition value. +function transitionArgs(transition, ms, fps) { + if (!transition || transition === 'cut' || !ms) return ''; + const frames = Math.max(1, Math.round((ms / 1000) * fps)); + if (transition === 'mix') return ` MIX ${frames}`; + if (transition === 'wipe') return ` WIPE ${frames}`; + return ''; +} + +// Turn an absolute /media path (or a relative one) into the token CasparCG +// expects: a path relative to MEDIA_ROOT, without extension, forward-slashed. +// CasparCG resolves "subdir/clip" against its media folder + probes extensions. +function toCasparToken(mediaPath) { + let p = String(mediaPath || ''); + if (p.startsWith(MEDIA_ROOT)) p = p.slice(MEDIA_ROOT.length); + p = p.replace(/^\/+/, ''); + p = p.replace(/\.[^/.]+$/, ''); // strip extension + return p; +} + +export class PlayoutManager { + constructor() { + this.amcp = new AmcpClient({ + host: process.env.CASPAR_HOST || '127.0.0.1', + port: parseInt(process.env.CASPAR_PORT || '5250', 10), + }); + this.state = { + running: false, + outputType: null, + outputConfig: null, + videoFormat: null, + playlist: [], // resolved items in play order + currentIndex: -1, + loop: false, + currentClip: null, + startedAt: null, + lastError: null, + }; + this._advanceTimer = null; + this._hlsProc = null; // standalone ffmpeg re-mux child process + this._hlsRestartTimer = null; + } + + async _consumerCommand(outputType, cfg) { + // Returns the AMCP ADD argument string for the requested output target. + if (outputType === 'decklink') { + const dev = cfg.device_index || 1; + return `DECKLINK DEVICE ${dev} EMBEDDED_AUDIO`; + } + if (outputType === 'ndi') { + const name = cfg.ndi_name || 'DRAGONFLIGHT'; + return `NDI NAME "${name}"`; + } + if (outputType === 'srt' || outputType === 'rtmp') { + // CasparCG 2.3 streams via the FFMPEG consumer, invoked with the STREAM + // keyword (FILE/STREAM are interchangeable aliases for it; the bare word + // "FFMPEG" is the PRODUCER and is NOT a valid consumer keyword). Args must + // use ffmpeg's -param:stream form (-codec:v, not -vcodec) or CasparCG + // rejects them. The channel feeds the consumer as RGBA, so a + // format=yuv420p filter is required before libx264. + const url = cfg.url || ''; + if (outputType === 'srt') { + const latency = cfg.latency || 200; + const full = url.includes('latency=') ? url : `${url}${url.includes('?') ? '&' : '?'}latency=${latency}`; + return `STREAM "${full}" -format mpegts -codec:v libx264 -preset:v veryfast -tune:v zerolatency -b:v 6M -codec:a aac -b:a 192k -filter:v format=yuv420p`; + } + const target = cfg.key ? `${url}/${cfg.key}` : url; + return `STREAM "${target}" -format flv -codec:v libx264 -preset:v veryfast -tune:v zerolatency -b:v 6M -codec:a aac -b:a 192k -filter:v format=yuv420p`; + } + throw new Error(`Unknown output_type: ${outputType}`); + } + + // Start the channel: bring up CasparCG's primary output consumer for the + // target, plus a second FFMPEG consumer writing HLS for the UI preview + // monitor (~4-6s lag, reuses capture's /live/ plumbing). + // + // The primary consumer failure is NON-FATAL. CasparCG can decode and route + // media through its pipeline even without an output consumer. This means: + // - NDI channels work (load/play/transport) even if libndi.so is absent. + // - SRT/RTMP channels work even if the destination URL is unreachable. + // - The HLS preview consumer is always attempted independently. + // + // state.consumerError is set when the primary consumer fails so the mam-api + // can surface a warning in the channel status without blocking operation. + async startChannel({ outputType, outputConfig = {}, videoFormat = '1080p5994' }) { + await this.amcp.waitReady(30000); + + // Set the channel video mode first. + try { await this.amcp.send(`SET ${CHANNEL} MODE ${videoFormat}`); } + catch (err) { console.warn(`[playout] SET MODE failed (continuing): ${err.message}`); } + + // Primary output consumer — non-fatal. + let consumerError = null; + try { + const consumer = await this._consumerCommand(outputType, outputConfig); + await this.amcp.send(`ADD ${CHANNEL} ${consumer}`); + } catch (err) { + consumerError = err.message; + console.warn(`[playout] primary consumer ADD failed (continuing without output): ${err.message}`); + } + + // HLS preview consumer — always attempt, independently non-fatal. + if (HLS_DIR) { + try { + await this._addHlsConsumer(); + console.log(`[playout] HLS preview at ${HLS_DIR}/index.m3u8`); + } catch (err) { + console.warn(`[playout] HLS preview consumer failed: ${err.message}`); + } + } + + this.state.running = true; + this.state.outputType = outputType; + this.state.outputConfig = outputConfig; + this.state.videoFormat = videoFormat; + this.state.fps = fpsFor(videoFormat); + this.state.startedAt = new Date().toISOString(); + this.state.lastError = consumerError; + console.log(`[playout] channel started output=${outputType} mode=${videoFormat} fps=${this.state.fps.toFixed(3)}${consumerError ? ' ⚠ consumer: ' + consumerError : ''}`); + return this.getStatus(); + } + + // HLS preview for the web UI confidence monitor. + // + // ── Why not CasparCG's own HLS (FILE/STREAM ".../index.m3u8") ────────────── + // CasparCG's bundled FFMPEG consumer muxes a BROKEN audio track into the HLS: + // ffprobe reports `aac, sample_rate=0` and ffmpeg decoding the playlist fails + // with "Invalid data ... abuffer: Value inf for parameter 'time_base' ... + // time_base 1/0". That corrupt audio prevents BOTH ffmpeg and hls.js from + // decoding, so the browser