Commit graph

122 commits

Author SHA1 Message Date
75c23448b4 fix(#65): GET /schedules returns 400 for unknown status query param 2026-05-25 17:40:58 -04:00
af905cf936 fix: bin creation 500 error + add drag-and-drop + project rename
- Fix 500 error when creating bins: missing updated_at column on bins table
  (migration 013 adds the column, schema.sql updated)
- Add drag-and-drop support for moving asset cards/list rows onto bin rail items
  with visual droppable highlight
- Add right-click context menu on project rail items (Rename/Delete)
- Expose RenameProjectModal via window so Library screen can reuse it
- Bins context menu already existed — was hidden by the 500 error
2026-05-24 13:27:24 -04:00
c312991bac feat: implement advanced features (conform, auto-relink, GUI redesign, docs, tests)
- #30 FCP XML Export & Conform: slide panel UI, preset system, FCP XML generation,
  conform job submission with progress polling via BullMQ
- #31 Hi-Res Auto-Relink: clip list with checkboxes, batch-trim server endpoint,
  trimWorker with frame-accurate FFmpeg trimming, auto-relink in Premiere via
  ExtendScript, temp segment signed URL endpoint
- #32 GUI Redesign: complete rewrite with Wild Dragon OKLCH design tokens
  (accent oklch(45% 0.20 266)), slide panels, preset cards, chip components
- #34 Cleanup Task: existing task validated and properly registered
- #35 Testing: comprehensive 33-scenario E2E test plan
- #36 Documentation: advanced features guide with workflows, troubleshooting,
  presets table, and architecture overview
- #24 PR merge: verified mergeable

All server endpoints, worker queues, and ExtendScript functions wired together
2026-05-24 13:19:24 -04:00
77130ac769 feat(server): temp segments cleanup task
Hourly cron that deletes expired temp_segments from S3 and DB.
Implements issue #34. Registered alongside scheduler in index.js.
2026-05-24 12:43:08 -04:00
a016175fc8 feat(db): migration 012 — advanced features schema
Add 'trim' to job_type enum, create temp_segments table with
expiry/job/asset indexes, and add conform_source_sequence_id
to assets for lineage tracking.

Closes #33
2026-05-24 12:42:22 -04:00
91325a4267 fix(jobs): real cancel for active jobs + multi-threaded thumbnail worker
DELETE /jobs/:id was throwing "404 not found" when the operator tried to
cancel a running job. BullMQ refuses job.remove() while a job is in the
active state; the route caught that error and fell through to the
404 branch, which was misleading because the job actually exists — the
queue was just refusing to drop it from under the worker.

Fix:
- Detect 'active' state explicitly and call moveToFailed(err, '0', false)
  first. Token '0' bypasses the per-worker lock check (the operator-side
  cancel doesn't hold the worker lock). That transitions active -> failed
  and frees the queue's concurrency slot.
- If moveToFailed itself fails (lock owned by a live worker), fall back
  to job.discard() so at least the result is thrown away.
- If remove() then fails (stalled, broken state), drop the job's Redis
  key directly via queue.client. Last-resort obliteration.
- Stop swallowing getJob() errors — if Redis is sad, surface it via
  next(err) instead of returning a misleading 404.
- Return { cancelled: true } when the job was active, so the client
  can show "Cancelled" rather than "Removed" in any future toast.

While here: thumbnail jobs now run with concurrency 4 by default
(proxy 2, conform 1, import 1 unchanged). Every queue defaulted to
concurrency 1 before, so a single stalled job blocked the entire queue.
All three are overridable via PROXY_CONCURRENCY / THUMBNAIL_CONCURRENCY
/ CONFORM_CONCURRENCY env vars for nodes with more headroom.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 17:23:07 -04:00
9a6ae3b786 fix(jobs): backfill asset_name from DB so non-YouTube jobs show their asset
The Jobs screen only displayed an asset name when the enqueueing code
stuffed assetName into the BullMQ job data. YouTube imports did that;
upload-triggered proxy/thumbnail jobs didn't — so everything except
YouTube showed em dashes in the Asset column.

Fix it centrally: after we collect jobs from BullMQ, look up names
in one bulk SELECT against the assets table for any job that has an
assetId but no asset_name. Applies to /jobs, /jobs/:id, and the SSE
events stream. Lookup failures fall through silently rather than
500-ing the whole list.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 16:23:29 -04:00
9ad88e4df4 feat(ingest): YouTube importer — paste link, asset travels normal pipeline
Adds Ingest → YouTube. UI takes a URL + project, API enqueues a BullMQ
"import" job, worker shells out to yt-dlp, lands the MP4 in S3 at the
same originals/{assetId}/... path uploads use, then hands off to the
existing proxy queue. Imported assets share one lifecycle with uploads
from that point on.

Worker container picks up yt-dlp + python3 (apk on alpine, apt on the
GPU variant). The new 'import' queue is registered in jobs.js so it
appears in the Jobs SSE stream and retry/delete work for free.

Spec: docs/superpowers/specs/2026-05-23-youtube-importer-design.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 16:05:41 -04:00
6a1d271576 feat(ui): polish round 2 — live refresh, schedule calendar, jobs times, real sidebar user
- recorders: dispatch df:recorders-changed on create/start/stop/delete so the
  list updates immediately instead of waiting for the 10s poll tick
- library: poll every 4s while any asset is live/processing (15s otherwise) and
  listen for df:assets-changed so a stopped recorder's LIVE badge drops and
  the thumbnail appears without a manual refresh
- auth: synthetic /auth/me (AUTH_ENABLED=false) now uses LOCAL_OPERATOR / USER /
  USERNAME instead of hardcoding "Admin", and flags synthetic:true
- shell: Sidebar takes `me` as a prop, drops the misleading "Admin" fallback,
  and surfaces an "auth off" hint when the response is synthetic
- jobs: replace the always-empty ETA column with a Time column that shows
  queued/started/done/failed N ago (full timestamp on hover); widen column
- schedule: new month-calendar view (default) with events plotted on day cells
  by status; clicking a day pre-fills the new-schedule modal with a 30-min
  window on that day; List view kept behind a toggle

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 14:52:04 -04:00
1afb150237 feat(assets): cleanup-live-orphans + retry handles non-error states
Two changes for issue #7 (HLS cleanup + orphan reaper) and the user's
"SRT clips ingest but won't play" complaint:

1) New POST /assets/cleanup-live-orphans — lists every directory under
   /live/<uuid>/ and deletes the ones whose UUIDs don't match an asset
   row. These accumulate when a recorder crashes mid-capture: the live
   HLS dir is created but no asset is ever finalized in the DB, so the
   files just sit on disk forever.

2) POST /assets/:id/retry now also works for assets that are 'ready'
   or 'archived' but have no proxy_s3_key. The original behavior (only
   re-queue when status='error') made it impossible to re-generate a
   proxy for older recorder captures that landed without one — the
   user could see a thumbnail in the library but the player would just
   show "Preview not yet available" with no retry path.
2026-05-23 10:28:42 -04:00
claude
90a9e4361a feat(comments): persistent frame-anchored comments on asset detail
- migration 010: asset_comments table (id, asset_id, user_id, body,
  frame_ms, resolved, timestamps) with index on asset_id+created_at
- new routes mounted at /api/v1/assets/:assetId/comments — GET/POST/
  PATCH/DELETE with author join (display_name + initials), nullable
  user_id so comments still attach when AUTH_ENABLED is off
- Asset detail loads comments from the API on mount instead of the
  empty ZAMPP_DATA.COMMENTS seed; addComment POSTs and merges the
  returned row; resolved-toggle and delete are wired
- CommentsList: new trash-icon delete action per comment, helpful
  empty-state copy ('Add one below to mark a frame'), tooltips on
  the timestamp and resolved buttons

Now editor comments survive page reload, are visible to other users
via the same API, and pin reliably to frame_ms (integer) instead of
a parsed HH:MM:SS:FF string.
2026-05-23 04:21:11 +00:00
claude
24820e921e polish: schedule past-time confirm, recorder name sanitization, asset detail player controls
- Schedule: if start_at is more than a minute in the past, confirm()
  before submitting (operator may want to fire immediately, but
  shouldn't do it accidentally)
- Recorders: generateClipName now sanitizes the recorder name so the
  S3 key / SMB path / ffmpeg arg stays clean — spaces become
  underscores, anything outside [A-Za-z0-9._-] is dropped, capped at 40
- Asset detail: audio mute + fullscreen buttons now key off streamUrl
  state (rather than videoRef.current which is null on first render)
  so they reliably appear when a stream is available
2026-05-23 04:12:42 +00:00
claude
47ad01d0b2 polish(projects,jobs,bins): row menus, real status bars, bulk retry
Projects:
- Per-row 3-dot menu in list view: Open / Rename / Delete (PATCH + DELETE)
- ProjectCard's bottom bar now shows real ready/in-flight/error counts
  for the project's assets instead of fake 70/20 segments
- After mutations, project list refreshes from /projects + recomputes
  asset counts client-side

Bins:
- GET /api/v1/bins now returns every bin across every project when
  no project_id is supplied; result rows include project_name + asset_count
- Asset right-click 'Move to bin' filters to bins in the same project as
  the asset and surfaces project_name as a tooltip

Jobs:
- 'Retry all failed' button in the header appears when there are
  failed jobs and POSTs /retry for each one in parallel
- Failed-row error message now clips with title= tooltip so 3KB
  ffmpeg stderr doesn't blow out the row layout

window.PROJECT_COLORS exposed for cross-screen access.
2026-05-23 04:09:13 +00:00
claude
992fbdfa20 fix(recorders,library): empty-capture handling + right-click context menu
Proxy failures ("moov atom not found"):
- root cause: failed/aborted SRT/RTMP recordings still uploaded 0-byte
  (or ftyp-only ~1KB) objects to S3, which ffmpeg can't probe
- worker proxy.js now bails on inputs < 4 KiB with a clear message
  before handing the file to ffmpeg
- capture-manager.stop() returns framesReceived + empty flag
- capture shutdown handler skips POST /assets entirely on empty
  sessions, instead calls new POST /assets/:id/mark-empty to flip
  the pre-created live asset to 'error' with a note

Library asset right-click menu:
- new AssetContextMenu component on screens-library.jsx; right-click
  any asset in grid or list view to open
- actions: Open, Rename, Move to bin (lists up to 10 bins), Remove
  from bin, Copy asset ID, Delete permanently (hard=true)
- viewport-aware positioning (won't clip past window edges)
- dismisses on outside click / contextmenu / scroll
- Library now refreshes via /assets after mutations; normalizeAsset
  exposed on window so the re-fetch shape matches boot
- ctx-menu styles in styles-rest.css
2026-05-23 03:52:30 +00:00
claude
9877ed351f fix(recorders): queue proxy on finalize + custom clip names
- POST /api/v1/assets: when transitioning from 'live' to 'processing'
  with a hi-res key but no proxy, queue a proxy job instead of just
  flipping status='ready'. Recorder-captured clips now get a proxy
  + thumbnail like upload-path assets do
- POST /api/v1/recorders/:id/start now accepts { clipName } in the body;
  operator-supplied name (sanitized to [A-Za-z0-9 ._-], capped at 80)
  overrides the auto-generated <recorder>_<timestamp> fallback
- RecorderRow gets a 'Clip name (optional)' input visible when stopped;
  Enter triggers Record, value sent on POST start, cleared on stop
- New POST /api/v1/assets/:id/generate-proxy and
  POST /api/v1/assets/backfill-proxies for one-shot cleanup of pre-fix
  clips that have a hi-res master but no proxy
2026-05-23 03:41:03 +00:00
claude
b128c9f5a9 fix(metrics): use real job_status enum values (queued/processing/complete) 2026-05-23 03:31:14 +00:00
claude
ef4c301149 feat(home,users): real metrics, working Users row actions + Groups CRUD
- Home: new /api/v1/metrics/home endpoint buckets last 24h of assets,
  jobs done/failed into hourly counts; sparklines now render real
  time-series instead of decorative sine waves
- Home stat cards are now clickable (route to relevant page) and the
  delta lines show real activity ("+N added in last 24h", "N completed")
- Home live-feed tiles use HlsPreview for recorders with a live_asset_id
- Users: row 3-dot menu is now a real popover with Rename / Reset
  password / Delete actions wired to PATCH /users/:id and DELETE
- Users: role is now an inline <select> that PATCHes immediately
- Users: Created column replaces fake 'last active' (no last_login
  tracking yet); group count is real
- Groups tab is now functional — list groups, create, expand to
  show + manage members (add/remove), delete; backed by existing
  /api/v1/groups CRUD
- Policies tab is now an honest 'coming soon' stub
- New icons: key, lock, edit; new .row-menu popover styles
2026-05-23 03:30:10 +00:00
claude
53196d38ce feat(scheduler): recorder scheduling — UI, CRUD, tick loop, recurrence
- New Ingest → Schedule page: upcoming/past/all tabs, status badges
  (pending / recording / completed / cancelled / failed), 10s
  auto-refresh, cancel/delete actions
- New Schedule modal: name, recorder dropdown, datetime-local start/end,
  recurrence (one-shot / daily / weekly), sensible defaults (+5min / +35min)
- Backend: migration 009 (recorder_schedules), routes/schedules.js
  (list/create/edit/cancel/delete), scheduler.js tick loop polling every
  15s; transitions trigger /recorders/:id/start and /stop via in-process
  HTTP so we reuse the full container orchestration path
- Recurring schedules: tick loop auto-queues the next occurrence on
  completion (daily = +24h, weekly = +7d)
- Sidebar + app.jsx route wired in, schedule-row table style added
2026-05-23 03:19:24 +00:00
claude
6398879b56 feat: SDK deployment UI, proxy encoding global settings, S3 env fallback
- Settings: drop AMPP tab, rename GPU/Transcoding → Proxy encoding
  with explicit 'applied to every ingested file' wording, expose
  CPU codec/preset options when GPU is off
- New Capture SDKs tab (Settings): upload Blackmagic / AJA / Deltacast
  SDK archives (.zip / .tar.gz) staged to /sdk/<vendor>/ inside mam-api;
  BMD is fully wired into the FFmpeg build pipeline, AJA + Deltacast
  staging-only pending FFmpeg patches
- mam-api: new /api/v1/sdk routes (multer upload, extract, list, delete);
  Dockerfile gets unzip+tar; docker-compose mounts /mnt/NVME/MAM/sdk:/sdk
- proxy worker now reads proxy-encoding settings from DB on every job,
  builds args for libx264 / NVENC / VAAPI, falls back to libx264 on
  hardware-encode failure
- settings GET /s3 falls back to S3_* env vars when DB is empty so the
  UI reflects what's actually wired (fixes 'not configured' false alarm)
2026-05-23 02:58:32 +00:00
328f7b4f31 feat: live HLS preview, proxy worker fixes, Settings tabs, growing-files + Premier panel
- worker/proxy: scale-to-even filter, analyzeduration 100M, skip images, hasAudio
- worker/promotion: SMB landing zone -> S3 on idle, queues proxy job, status='ready'
- web-ui screens-ingest: HlsPreview component replaces fake LiveStrip/FauxFrame
- web-ui screens-admin: functional Settings tabs (S3, GPU, Growing, SDI, AMPP)
- mam-api /settings/growing: GET/PUT growing-files config
- mam-api /assets/:id/live-path: SMB UNC/POSIX path for live growing assets
- capture-manager: GROWING_ENABLED -> write hires to /growing instead of S3 stream
- recorders.js: pass GROWING_ENABLED to capture container, bind /growing mount
- docker-compose: mount /mnt/NVME/MAM/wild-dragon-growing on mam-api + worker
- premiere-plugin: Mount Live button, Relink-to-HiRes, live->ready status poll
2026-05-22 19:12:53 -04:00
ceceedf201 probe fallback: basic TCP/UDP connectivity when capture service is offline 2026-05-22 17:26:26 -04:00
4864db03f3 probe: fallback to basic TCP/UDP connectivity check when capture service is offline 2026-05-22 17:22:30 -04:00
4afd0c7b21 feat: add /cluster/containers endpoint via Docker socket
Lists all containers on the local host; supports POST /containers/:id/restart.
Falls back to [] gracefully if Docker socket is unavailable.
2026-05-22 16:57:33 -04:00
ddb4cf0c51 feat: add POST /jobs/:id/retry endpoint for re-queuing failed BullMQ jobs 2026-05-22 12:18:53 -04:00
994fd799d0 fix: stop endpoint handles missing/dead containers gracefully 2026-05-22 11:32:44 -04:00
37767f9939 fix(cluster): pickIp() only treats 172.17.x as docker bridge, not all of RFC1918 172.16/12 2026-05-21 21:27:15 -04:00
bbed2a7059 fix(decklink): mount /dev/blackmagic in sidecar + remote node routing via node-agent
Two bugs fixed:
1. SDI capture sidecar never had /dev/blackmagic bound — ffmpeg opened the
   decklink input inside a container with no device nodes, so frame=0.
   Fix: local spawns now push '/dev/blackmagic:/dev/blackmagic' onto Binds
   when source_type='sdi'.

2. recorders.js always spawned sidecars against the local Docker socket
   (zampp1), even when a recorder's node_id pointed at zampp2 (where the
   card is). Fix: resolveNodeTarget() looks up the recorder's cluster node;
   if it's a different hostname the sidecar is spawned via a new
   POST /sidecar/start endpoint on the remote node-agent.

node-agent gains three new routes (all talk to the local Docker socket):
  POST   /sidecar/start         — create + start container (host network,
                                   privileged, /dev/blackmagic bind for sdi)
  DELETE /sidecar/:id           — stop + remove
  GET    /sidecar/:id/status    — inspect + poll capture service

docker-compose.worker.yml: add /var/run/docker.sock and LIVE_DIR to
node-agent so it can spawn sidecars, and document build-capture prerequisite.: recorders.js
2026-05-21 18:51:10 -04:00
4a3a672cbe cluster: stable hostname for mam-api, jq-based smoke test
mam-api self-heartbeat now reads NODE_HOSTNAME so primary rows survive container restarts instead of resurrecting with the random container ID. test-cluster.sh rewritten to use jq (the python f-strings had a parse bug that silently passed the IP check) and limited the docker-bridge alarm to 172.17.x since the user LAN occupies 172.18.0.0/16.
2026-05-21 11:50:52 +00:00
4c65753358 recorders route: accept full codec field set + node/port pinning
POST/PATCH now persist all new codec columns via a whitelist. /start
forwards every codec setting to the capture container as an env var. The
live-asset created during /start now uses the recorder's container ext
(.mov vs .mp4 etc.) instead of always assuming .mov.
2026-05-21 00:17:45 -04:00
0efef0d81b cluster route: fallback IP from request + /devices/blackmagic endpoint
Heartbeat handler now overrides 172.x docker bridge IPs with the
request's source address when the request itself came from a real LAN.
Adds GET /devices/blackmagic that flattens every node's capabilities so
the recorder UI can show a card picker spanning the whole cluster.
2026-05-21 00:16:36 -04:00
049beb8818 recorders: add granular codec / container / port columns
Expands recorders with video bitrate, framerate, audio codec / bitrate
/ channels, container format, and a node_id/device_index pair so the UI
can pin SDI recorders to a specific node + DeckLink port instead of
relying on a flat "BM1/BM2" index. capture-manager.js consumes these via
env vars and builds ffmpeg args from them.
2026-05-21 00:14:11 -04:00
a39c9831c5 cluster: dedupe rows + enforce unique hostname index
Migration 004 wrapped table creation in IF NOT EXISTS, so deploys with
a pre-existing cluster_nodes table never picked up the inline UNIQUE
constraint and accumulated duplicate hostnames on every container
restart. This migration purges older duplicates and adds the unique
index idempotently so the ON CONFLICT (hostname) upsert finally works.
2026-05-21 00:14:01 -04:00
066b9b17d3 feat: expand GPU transcoding settings (extension, framerate, rc mode, audio) 2026-05-20 23:41:42 -04:00
74299629e6 feat: detect GPUs via nvidia-smi and populate cluster_nodes capabilities 2026-05-20 17:25:11 -04:00
a4b9b5be82 fix: prefer NODE_IP env var in getLocalIp() for Docker deployments 2026-05-20 16:16:09 -04:00
a926da1c30 feat: add settings key-value table migration 2026-05-20 15:57:23 -04:00
7032cee6b3 feat: call loadS3ConfigFromDb() on startup after migrations 2026-05-20 15:53:26 -04:00
02cfa68b92 fix(assets): replace static S3_BUCKET with getS3Bucket() for dynamic config 2026-05-20 15:49:40 -04:00
737e69d72f fix(upload): replace static S3_BUCKET with getS3Bucket() for dynamic config 2026-05-20 15:48:48 -04:00
ab504841c3 feat(settings): add S3 / object-storage settings routes (GET, PUT, test) 2026-05-20 15:48:14 -04:00
b1457f0aad feat(s3): dynamic DB-driven config with rebuildS3Client + Proxy export 2026-05-20 15:47:40 -04:00
1725ec1de9 feat: settings routes for hardware inventory, GPU transcoding, capture service URL 2026-05-20 14:18:43 -04:00
dd3c2894f6 feat: cluster heartbeat stores capabilities (GPU/BMD hardware detection) 2026-05-20 14:18:22 -04:00
86d2960b60 feat: add capabilities column to cluster_nodes (migration 005) 2026-05-20 14:17:44 -04:00
5161644205 fix(capture): handle non-JSON responses from capture service gracefully 2026-05-20 13:55:06 -04:00
4dd377e28d feat(cluster): add GET /:id/ping to probe node agent reachability and latency 2026-05-20 13:49:56 -04:00
a855ea7885 feat(api): add GET /assets/:id/hires endpoint for original file download
Returns presigned S3 URL + filename/ext/file_size for the original
hi-res source so the Premiere plugin can download and import it.
2026-05-20 00:34:18 -04:00
090452969c feat(api): register system + cluster routes; add self-heartbeat on startup 2026-05-19 23:50:19 -04:00
66844b93d3 feat(cluster): node registry API — heartbeat, list, deregister 2026-05-19 23:46:16 -04:00
bd8b492ff6 feat(db): cluster_nodes table for multi-server registry 2026-05-19 23:46:06 -04:00