The HLS-VOD work made GET /assets/:id/stream return the HLS playlist URL as
`url` whenever hls_s3_key was set. The Premiere plugin's "Import Proxy"
downloads `url` to a file and imports it — so it was saving an .m3u8 playlist
as .mp4, and Premiere rejected it ("unsupported compression type"). This hit
every YouTube asset (all get HLS generated), regardless of codec.
/stream now returns the directly-downloadable MP4 proxy as `url` (type mp4)
and the HLS playlist as a separate `hls_url`. The web player prefers `hls_url`
(so in-browser HLS playback is unchanged), while the already-installed plugin
gets a real MP4 again — no plugin reinstall needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 0.2 of the NVENC All-Intra HEVC ingest plan.
node-agent/handleSidecarStart:
- Accept useGpu: true in the sidecar start body
- When useGpu: adds Runtime=nvidia, DeviceRequests=[gpu], and injects
NVIDIA_VISIBLE_DEVICES=all + NVIDIA_DRIVER_CAPABILITIES=video,compute,utility
into the container env. CPU-codec recorders are unaffected (useGpu defaults false).
mam-api/recorders (start endpoint):
- Derive useGpu from recorder.recording_codec — true for hevc_nvenc/h264_nvenc
- Pass useGpu to remote sidecar start body
- Apply same Runtime/DeviceRequests to the local Docker spawn path
capture/capture-manager:
- Update hevc_nvenc codec entry with all-intra flags:
-g 1 -bf 0 (every frame IDR, no B-frames — required for growing-file
edit-while-record), -rc vbr, -profile:v main10, pixFmt p010le (10-bit 4:2:0)
Next: validation gate (§8) — test MXF OP1a then fragmented MOV on one
DeckLink channel, mount in Premiere while recording.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stuck-live fix: capture sidecar now finalises the pre-created live asset by id (new POST /assets/:id/finalize) instead of POSTing a new asset (409 collision); node-agent gives the sidecar a 180s stop grace so the S3 upload + callback complete; node-agent logs sidecar start/stop for diagnostics.
Live SDI monitor: HLS preview is now a 2nd output of the hires ffmpeg (single DeckLink read, split to ProRes/S3 + H.264/HLS); node-agent serves /live over HTTP; mam-api proxies GET /recorders/:id/live/* to the recorder node; web-ui HlsPreview loads from the proxied URL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- capture/src/index.js: read MAM_API_TOKEN from env; include
Authorization: Bearer header in shutdown callback fetches to mam-api
(POST /assets and POST /assets/:id/mark-empty). Without this, mam-api
AUTH_ENABLED=true rejects the callback with 401, leaving assets stuck in live
- recorders.js: pass MAM_API_TOKEN=${CAPTURE_TOKEN} in sidecar env so the
capture container receives the token at boot
- api_tokens: inserted capture-sidecar token (unbound, prefix b3d3d3c4)
- recorders.js: when isRemote=true, replace MAM_API_URL in sidecar env with
http://<NODE_IP>:<PORT_MAM_API> so capture containers on worker host network
can reach mam-api (fixes assets stuck in live status after recorder stop)
- cluster.js: add GET /api/v1/cluster/metrics endpoint returning per-node
cpu/ram/gpu utilization; update heartbeat handler to persist metrics JSONB
- web-ui: add Resources panel to dashboard with live CPU/RAM/GPU bars per node,
polling /api/v1/cluster/metrics every 5s
## capture service
- capture-manager.js: add 'deltacast' source_type to _buildInputArgs.
Uses 'deltacast://<index>' with ffmpeg deltacast demuxer when
/dev/deltacast<N> exists; falls back to lavfi testsrc2 + sine test card
(matching deltacast-sdi-recorder standalone app) when hardware absent.
- routes/capture.js: add GET /devices/deltacast endpoint (enumerates
/dev/deltacast* + DELTACAST_PORT_COUNT env fallback). Extend /probe to
handle source_type=deltacast.
## node-agent
- detectHardware(): add 'deltacast' array to capabilities payload.
Enumerates /dev/deltacast* nodes; falls back to DELTACAST_PORT_COUNT env.
Adds DELTACAST_MODEL env support. Logs dc= count in heartbeat line.
- sidecar /start: bind /dev/deltacast* device nodes into capture containers
when sourceType='deltacast'.
## mam-api
- cluster.js: add GET /cluster/devices/deltacast and
GET /cluster/devices/deltacast/signal endpoints — same shape as
blackmagic equivalents for UI parity.
- recorders.js /start: pass DELTACAST_PORT_COUNT env to capture container;
bind /dev/deltacast* device nodes on local spawn.
- migration 024: ALTER TYPE source_type ADD VALUE 'deltacast' (idempotent).
- schema.sql: add 'deltacast' to source_type ENUM for fresh installs.
## web-ui
- modal-new-recorder.jsx: add 'Deltacast' source type card; fetch
/cluster/devices/deltacast on selection; port picker with TEST CARD
badge when hardware absent; falls through to manual index entry if
no devices detected.
- capture-manager.js, routes/capture.js: fix ffmpeg -sources decklink
parse regex from v4l2 hex-address format (never matched DeckLink output)
to correct indented-line format. Port 2+ (index 1+) was falling through
to a wrong model-name fallback, causing ffmpeg to open the wrong input
and produce black frames. Now logs the detected device list and the
selected name at start.
- recorders.js (/start): accept per-take projectId override in request
body. If provided, clips go to that project instead of the recorder's
default project_id. Used for both the live-asset INSERT and the
PROJECT_ID env var passed to the capture container.
- screens-ingest.jsx (RecorderRow): add project dropdown shown when
recorder is stopped. Defaults to the recorder's configured project;
operator can change it before hitting Record without editing the
recorder config.
The conform worker's final step INSERTs the rendered output into the
assets table:
INSERT INTO assets (project_id, filename, display_name, …)
VALUES ($1, …)
-- project_id NOT NULL
It reads projectId from job.data, but the /sequences/:id/conform
endpoint never set it. Render finished cleanly, ffmpeg ran, output
uploaded to S3, then the final asset row INSERT failed:
null value in column "project_id" of relation "assets"
Pass seq.project_id from the loaded sequence row. The rendered output
lands as an asset under the same project as its source sequence —
the natural target.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two cooperating bugs left Export Timeline stuck at "Rendering Hi-Res"
forever:
A. worker emitted "Invalid FCP XML: no sequence element" because
Timeline.generateFcpXml produced fcpxml (FCP X schema:
<fcpxml><resources>/<library>/...) while the worker's parseFcpXml
expects xmeml (FCP 7 schema: <xmeml><sequence>...). Two completely
different formats.
Rewrite generateFcpXml to emit xmeml v5 with the structure the
parser walks:
xmeml/sequence/{name,duration,rate{timebase,ntsc},
media/video/{format/samplecharacteristics,
track[@currentExplodedTrackIndex]
/clipitem/{name,duration,rate,in,out,
start,end,file/{name,pathurl}}}}
Clipitem in/out are SOURCE frames (the underlying media in/out);
start/end are TIMELINE frames (the cut position). The worker uses
the rate timebase to parse them.
B. /api/v1/jobs/:id rejected the panel's polls with
"Invalid id — must be a UUID". The handlers below correctly parse
BullMQ-prefixed ids ("conform:42"), but router.param('id',
validateUuid('id')) ran first and 400'd everything that wasn't a
UUID. The panel's pollConform swallows the resulting fetch error
silently and polls forever.
Drop the validator. Comment in the file explains why.
Bumps panel to v2.2.2.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- requireAuth bearer path now selects api_tokens.bound_hostname and users.role,
populates req.tokenBoundHostname and req.user.role. /cluster/heartbeat can
now authenticate via a bound api_token (issued via POST /auth/tokens with
bound_hostname).
- routes/tokens.js POST accepts bound_hostname; GET returns it so users can
see which tokens are bound.
- Remove /cluster/heartbeat from SERVICE_PATHS so requireAuth runs on it (the
bearer auth handles the gate; the heartbeat handler still enforces the
body.hostname === bound match).
- /auth/me now returns role (final-review I2). Closes the gap where every
signed-in user appeared as 'viewer' in the UI regardless of actual role.
- loadUser SELECTs role for session auth.
- Backend tests still 37/15/0/22 — no test changes needed; existing token
CRUD tests stay passing since bound_hostname is optional.
Final-review findings:
- Mount usersRouter at /api/v1/users in addition to /api/v1/auth/users so the
existing SPA Users page works; add PATCH /:id for inline edits (display_name,
role, password).
- Add X-Requested-With: dragonflight-ui to raw XHR/fetch paths that bypass
apiFetch (file uploads, SDK uploads, EDL export) — without it, requireUiHeader
403s before reaching the route.
- Exempt SERVICE_PATHS (/cluster/heartbeat) from requireUiHeader so node-agent
heartbeats keep working when NODE_TOKEN is unset.
- Remove stale auth.js.bak.
Code-review feedback:
- Dummy hash for user-enumeration-defense timing was 63 chars (bcrypt strings
are 60 chars). Worked by accident because bcrypt 5.x is lenient about
trailing chars; a future tightening would silently regress the timing
defense. Replaced with a real pre-computed bcrypt hash.
- last_login_at UPDATE now logs errors instead of silently swallowing them,
matching the pattern in requireAuth for api_tokens.last_used_at.
- Removed dead import of comparePassword from auth.test.js.
- remove requireAuth from all route files
- delete auth.js, tokens.js, users.js routes
- delete auth middleware
- remove session middleware and all auth deps from index.js
- delete login.html and auth-guard.js from web-ui
Scope (locked in via planning Q&A):
- Identity: local accounts only (PG users table) + existing bearer
tokens for headless callers.
- Transport: httpOnly cookie session for browser, Bearer for API.
- RBAC: admin / editor / viewer roles, plus an orthogonal
is_client flag for external (agency, talent, customer) accounts.
- Bootstrap: ADMIN_BOOTSTRAP_USER + ADMIN_BOOTSTRAP_PASSWORD env
seed the first admin on a clean install. Set ADMIN_BOOTSTRAP_RESET
to force-reset the named user (break-glass).
- Rate limit: in-memory, 10 fails per 15min per (IP, username).
- Password policy: \u22658 chars, mixed case, digit, symbol; small
blocklist of common passwords; cannot equal username.
- Self-service: change own display name + password. Everything
else (role, is_client, other-user mgmt) is admin only.
- Audit log: append-only table, indexed by actor + event_type +
created_at, populated by every auth/admin event.
Files added:
- services/mam-api/src/db/migrations/022-auth-rework.sql
users.is_client + last_login_at + failed_attempts; audit_log
table with FK to users (ON DELETE SET NULL).
- services/mam-api/src/middleware/audit.js
Fire-and-forget audit() helper. Caller never awaits, failure
logs but never throws — auditing cannot break the request
that triggered it.
- services/mam-api/src/middleware/passwordPolicy.js
Shared checkPassword(pw, { username }) used by setup, user
create/update, and self-service password change.
- services/mam-api/src/tasks/bootstrapAdmin.js
Runs after migrations. No-ops unless ADMIN_BOOTSTRAP_USER +
ADMIN_BOOTSTRAP_PASSWORD are set AND (users table empty OR
ADMIN_BOOTSTRAP_RESET=true).
- services/mam-api/src/routes/audit.js
Admin-only GET /audit (paginated, filter by event_type /
actor / target / date) and GET /audit/event-types.
- services/web-ui/public/modal-account-settings.jsx
Profile + Password tabs. Triggered by sidebar user button.
Files rewritten:
- services/mam-api/src/routes/auth.js
- POST /login: regenerate(), no manual save(); audit success/
fail/lockout; updates last_login_at + failed_attempts.
- POST /logout: destroys session, audits logout.
- GET /me: returns is_client + last_login_at. Synthetic admin
when AUTH_ENABLED=false.
- GET /setup-status: drives login.html UI state.
- POST /setup: blocked once any user exists; password policy.
- POST /password: self-service. Requires current pw, runs
policy, audits, invalidates other sessions implicitly via
users.js if changed by admin.
- PATCH /me: self-service display_name update.
- services/mam-api/src/routes/users.js
- is_client field in create/update/list/get.
- Guardrails: cannot delete or demote last admin, cannot
delete self, admins cannot be flagged is_client.
- Password change invalidates all sessions for that user
(DELETE FROM sessions WHERE sess->>'userId' = id).
- Audit on every mutation.
- Password policy enforced.
- services/mam-api/src/middleware/auth.js
- requireAuth now exposes req.user.is_client.
- New requireRole(["admin","editor"], { rejectClients: true })
helper. Applied to cluster, sdk, capture routes (infra).
- Synthetic user when AUTH_ENABLED=false has is_client=false.
- services/mam-api/src/index.js
- Loads bootstrap admin after migrations.
- Wires /api/v1/audit.
- Cleans up an earlier comment block.
- services/web-ui/public/login.html
- Password hint added next to setup-mode password field.
- services/web-ui/public/shell.jsx
- Sidebar user footer is a button that opens AccountSettings.
- CLIENT badge next to role when is_client=true.
- Nav filters: clients lose ingest tree + jobs + editor;
viewers lose ingest + editor; only admins see the Admin
section. Power button hidden when synthetic user.
- services/web-ui/public/screens-admin.jsx
- Users table: new Client column with inline toggle.
- InviteUserModal: Client checkbox + password hint, gated
off when role=admin.
- Last login column replaces Created in primary view.
- CSV export includes client + last_login.
- services/web-ui/public/data.jsx
- ZAMPP_DATA.ME carries is_client + display_name.
- services/web-ui/public/index.html
- Loads dist/modal-account-settings.js.
- services/web-ui/public/styles-rest.css
- .user-row grid widened to 6 columns.
- docker-compose.yml
- Plumbs SESSION_COOKIE_SECURE + ADMIN_BOOTSTRAP_* env vars.
Deploy:
cd /opt/wild-dragon
git pull origin main
# In .env:
# AUTH_ENABLED=true
# SESSION_SECRET=<openssl rand -hex 48>
# ADMIN_BOOTSTRAP_USER=admin
# ADMIN_BOOTSTRAP_PASSWORD=<strong>
docker compose build mam-api web-ui
docker compose up -d --force-recreate --no-deps mam-api web-ui
Login was returning 200 + correct user JSON + writing a row to the
sessions table, but emitting zero Set-Cookie headers. Root cause:
session.regenerate() → set fields → session.save() → res.json()
Calling session.save() manually writes the store but bypasses
express-session's res.end() hook, which is the only path that adds
the Set-Cookie header to the response. The cookie was never sent to
the browser even though the session existed server-side — hence the
redirect loop.
Fix: remove the manual save(). Set the session fields and call
res.json() directly inside regenerate()'s callback; express-session
handles store write + Set-Cookie automatically on res.end().
The redirect loop after successful login was almost certainly the
`sessions` table never being created. `schema.sql` defines it but
only runs on first-init via the postgres entrypoint; instances
bootstrapped via mam-api's own migration loop never got the table.
express-session's `req.session.save()` then failed silently and the
cookie pointed at a sid that wasn't in the store — every subsequent
request looked like a brand-new visitor.
- New migration 021-ensure-sessions-table.sql (idempotent).
- connect-pg-simple now configured with `createTableIfMissing: true`
as belt-and-braces.
- `POST /auth/login` now explicitly waits for session.save() and
surfaces both regenerate() and save() errors instead of treating
them as 'success'. Logs sid + req.secure + req.protocol so we can
confirm trust-proxy is doing the right thing behind NPM.
Three concrete issues kept the login flow broken on dragonflight.live:
1. mam-api trusted no proxy headers, so behind nginx/Cloudflare the
session cookie's `secure` flag and the rate-limiter's IP keying
both saw the wrong values. Now sets `app.set('trust proxy', 1)`.
2. Session config was tied to NODE_ENV and lacked sameSite/name. Now:
- SESSION_COOKIE_SECURE env (default: true when AUTH_ENABLED) so a
site behind HTTPS gets Secure cookies regardless of NODE_ENV.
- `sameSite: 'lax'` for predictable post-login redirects.
- Renamed to `df.sid` so it's obvious in DevTools.
- `rolling: true` extends the 7-day TTL on active use.
- SESSION_SECRET is now required when AUTH_ENABLED=true; the
server refuses to start with a dev default in prod.
3. login.html silently showed the sign-in panel even when no users
exist or auth is off:
- New GET /auth/setup-status reports {needs_setup, user_count,
auth_enabled}.
- login.html calls it on load and auto-flips into setup mode when
needs_setup is true, or shows an explicit "auth is off" flash
when auth_enabled is false (the previous symptom: logout button
did nothing because /auth/me returned a synthetic admin no matter
what).
- Added a `.flash.info` style for the new neutral notice.
4. Sidebar logout used to call /auth/logout then `window.location
.reload()`. With auth off that reload landed back on the synthetic-
admin app and looked like nothing happened. It now redirects to
/login.html in all states so the operator sees feedback (and the
server-side messaging about auth being off) instead of a no-op.
Deploy notes for zampp1:
- Set AUTH_ENABLED=true and a random SESSION_SECRET in the
mam-api environment (e.g. /opt/wild-dragon/.env).
- Restart mam-api.
- First load of /login.html will auto-route to the setup form so
you can create the first admin.
RustFS returns empty bodies for ranged GETs whose start offset is past
~5.9 MB on single-file proxy MP4s. HEAD reports correct size, full GET
(`bytes=0-`) works, but `bytes=8179166-` comes back 206 + correct
Content-Range header with zero bytes. Confirmed via direct S3 probe
against broadcastmgmt.cloud/dragonmam (see scratch tests).
Workaround in mam-api `GET /api/v1/assets/:id/video` until the proxy
worker emits HLS (planned v1.2.1):
- HEAD the object first to learn total size (also gives ETag /
Last-Modified for conditional requests).
- No-Range / unparseable-Range / pre-EOF requests \u2192 plain pipe.
- Parsed `bytes=N-M` requests below RUSTFS_RANGE_SAFE_START
(default 5_500_000) \u2192 direct ranged GET, RustFS handles fine.
- Anything reaching into the broken zone \u2192 stream from offset 0,
drop bytes below start, stop at end. Memory stays flat; extra
bandwidth = (end+1 - requested-size) per seek.
- Genuinely out-of-range \u2192 416 with Cache-Control: no-store so the
browser doesn't poison its cache.
Also stashes (not yet wired up) the HLS pieces we'll need for the
follow-up: `segmentToHls` ffmpeg helper + `uploadDirectoryToS3`
worker s3 helper. Harmless additions; not referenced by any code path
yet.
Confirmed against the affected asset (a72aaa03-...): bytes=0-100k +
50% +100k native pass-through; 70% +100k and near-EOF previously hung
the browser, now stream correctly via the stitched path.
Refs #143.
Frontend / UX / a11y
- Sidebar collapse/expand toggle with localStorage persistence (#142)
- Settings sections wrap inputs in <form> with Enter-to-submit + native
validation; password autocomplete=new-password (#141, #138)
- Asset thumbnails get descriptive alt text (#140)
- Production deploy now precompiles JSX via esbuild and loads the
production React UMD instead of dev builds + in-browser Babel (#139,
#122)
- Search wrapper gets role=search; global search input gets aria-label,
role=combobox, aria-controls/aria-expanded/aria-activedescendant
wiring (#137, #135)
- Dashboard and Library no longer share the same nav icon (#136)
- Sidebar collapses off-canvas with a topbar menu button below 768 px;
mobile default is collapsed (#134)
- --text-3 bumped to #8B92A0 for WCAG AA contrast on --bg-0 (#133)
- Schedule and Library routes were rendering empty inside the .main
flex container — switched to flex:1 + min-height:0 (#131, #132,
editor + asset detail get the same fix)
- Jobs nav badge now polls /jobs?status=active every 10 s and reflects
the live count (#130, #113)
- aria-label sweep on every icon-only button (#126)
- Premiere panel release list moved to window.PREMIERE_RELEASES in
data.jsx; Editor + Settings read from the same source (#125)
- Typo setPgMclips → setPgmClips (#124)
- Stray console.error / console.warn calls gated behind
window.DF_LOG.{warn,error} (#123)
- Hardcoded /api/v1 paths route through window.ZAMPP_API_PREFIX (#115)
- Schedule rows no longer crash on null recorder_id (#117)
- EditorKeyboard guards against document.activeElement === null (#116)
- Unmount-safe timers for PasswordResetModal, Containers, Editor (#111)
- Player seek clamps below totalMs, server-side range clamping +
uncached 416 on EOF, client-side EOF-stall watchdog (#143)
- Duration badge overlap fix on narrow asset cards (#52)
Backend / security / reliability
- GET /recorders fixed N+1: single LATERAL JOIN for live_asset_id;
Docker inspects bounded to actually-recording rows (#121)
- Upload disk-storage (multer.diskStorage) streams parts to S3 instead
of buffering 500 MB in RAM (#120)
- /assets list clamps limit to MAX_LIMIT=500 to prevent OOM (#119)
- SDK upload archive listing + post-extract sanitize block zip-slip /
tar-slip and symlink escapes (#118)
- Migrations track applied state in schema_migrations, run in a
transaction, and exit non-zero on failure (#107)
- node-agent BMD_COUNT override uses BMD_DEVICE_PREFIX; filesystem
detection wins (#109, #127)
- GPU_COUNT override now merges with nvidia-smi enrichment (#108)
- /cluster/heartbeat requires a node-bound token or admin user;
tokens carry bound_hostname (#106)
- /recorders/:id/start error responses no longer echo the Docker
create payload — env vars stay out of client responses (#105)
- /recorders/probe restricts schemes (srt/rtmp/rtsp/udp/rtp), blocks
private + loopback hosts for non-admins, denies common service
ports (#104)
- Scheduler tick guarded by a Postgres advisory lock; pending/running
rows claimed via UPDATE...RETURNING + FOR UPDATE SKIP LOCKED to
survive multi-node deploys (#103)
- UUID validateUuid('id') param middleware on every /:id route (#102)
- Error handler scrubs Postgres error messages and 5xx detail (#101)
- Graceful SIGTERM/SIGINT shutdown — stops scheduler, drains the HTTP
server, ends the pool, 25 s force-exit watchdog (#100)
- AMPP sync moved from fire-and-forget to a persisted retry queue
(ampp_sync_status / attempts / next_attempt_at + scheduler retry
loop with exponential backoff) (#77)
Migrations
- 019: api_tokens.bound_hostname (#106)
- 020: assets.ampp_sync_status + retry bookkeeping (#77)
Other
- Defer #92 Growing-files per-upload toggle, #80 Audio tab, #57
Dashboard redesign, #56 Editor SPA polish phase 3, #114 S3
migration tool to v1.3
Adds per-port video signal state to the admin Cluster panel:
- New GET /cluster/devices/blackmagic/signal endpoint joins recorders by
node_id+device_index and queries each active capture container's
/capture/status (local: http://recorder-<id>:3001, remote: api_url/
sidecar/<container_id>/status). Returns receiving/connecting/lost/
error/idle/no-recorder per port plus framesReceived and currentFps.
- bmd-card.js render() now accepts portSignals (Map or object) and
overlays a colored dot on each BNC connector with pulse animation
for receiving/connecting states.
- screens-admin.jsx Cluster panel polls the new endpoint every 5s,
feeds the signal map into both the port chips (now show
RECEIVING/CONNECTING/LOST + fps) and the BMD SVG card diagram
rendered below them via a new BmdCardPanel component.
- styles-fixes.css adds bmd-card-* styles for the SVG diagram and
bmd-port-signal --pulse animation.
mam-api /video endpoint:
- S3 InvalidRange (httpStatusCode 416) was being caught and returned as 500
via next(err), which the video element treats as a fatal load error and
freezes the player. Now we catch the specific 416 case, do a no-range
HEAD-equivalent to learn the real file size, and return proper 416 with
Content-Range: bytes */<total> so the browser can recover.
screens-asset.jsx — player health + buffer visualization:
- New states: playerState ('idle'|'loading'|'playing'|'paused'|'seeking'|
'waiting'|'stalled'|'error'), playerError, buffered (array of {start,end}
ms from HTMLMediaElement.buffered), stallStart, stallElapsedMs
- Wired video element events: onProgress, onWaiting, onStalled, onPlaying,
onCanPlay, onCanPlayThrough, onSeeking, onSeeked, onError
- onError captures MediaError code+message into a console.error and the
on-screen badge so freeze causes are now visible
- Status badge overlay (top-right of player): shows SEEKING / BUFFERING /
STALLED / ERROR + elapsed seconds since the stall began
- PlaybackBar renders buffered ranges as translucent grey segments so you
can see what the browser has loaded vs. what's still pending — makes
seek-related freezes immediately obvious
S3 at broadcastmgmt.cloud (RustFS/openresty) returns 403 on range
requests that include an Origin header on presigned URLs. The HMAC
signature only covers 'host' in X-Amz-SignedHeaders, so the browser's
cross-origin Origin header breaks signature validation.
Reverted: /stream and /video no longer redirect to signed S3 URLs.
Fixed: /video now pipes through Node with:
Cache-Control: private, max-age=3600
ETag and Last-Modified forwarded from S3
This means the browser caches video segments for 1h. On seek the
browser checks its cache first — only uncached byte ranges hit the
server. Combined with the 1.5Mbps proxy (was 4Mbps), seeks should
be responsive for clips under ~10 minutes.
- GET /assets/:id/stream now returns a signed S3 URL directly (4h TTL)
instead of pointing to the /video pipe endpoint. Browser streams
directly from S3 — no Node.js bottleneck, S3 handles range requests
natively for smooth seeking.
- GET /assets/:id/video now redirects (302) to a signed S3 URL.
Belt-and-suspenders: any code still calling /video gets redirected.
- proxy.js: default bitrate changed from 10Mbps to 1.5Mbps, audio
default from 192kbps to 128kbps. DB settings already updated to
1.5Mbps. Cuts proxy file size ~6x for the same quality content.
Existing proxies need re-generation at new bitrate.
Root causes found:
1. Scheduler crashing every 15s: assets table has no error_message column.
Fix: remove error_message from UPDATE in scheduler.js (#66 regression).
2. Clip freezing: client-side filmstrip seek loop runs on main thread,
seeks same proxy the player is streaming → both stall → freeze.
Fix: replace browser seek loop entirely with server-side FFmpeg worker.
3. No dedicated filmstrip worker: filmstrip was never pre-built server-side.
Changes:
- services/mam-api/src/db/migrations/018-add-filmstrip-s3-key.sql
Add filmstrip_s3_key TEXT column to assets table
- services/worker/src/workers/filmstrip.js (new)
BullMQ worker: downloads proxy, runs FFmpeg fps filter to extract
28 evenly-spaced JPEG frames, base64-encodes them, uploads JSON
array to S3 at filmstrips/<assetId>.json, stores key in DB
- services/worker/src/workers/thumbnail.js
Queue filmstrip job automatically after thumbnail completes
- services/worker/src/index.js
Register filmstrip worker (concurrency=2), export filmstripQueue
singleton, close it on SIGTERM
- services/mam-api/src/routes/assets.js
- filmstripQueue added
- POST /reprocess?type=filmstrip now supported
- GET /:id/filmstrip returns signed S3 URL for JSON frames
- services/mam-api/src/routes/jobs.js
filmstrip queue visible in Jobs UI
- services/web-ui/public/screens-asset.jsx
Replace browser seek loop with fetch of /assets/:id/filmstrip
→ fetch S3 JSON → render frames. Zero browser-side video seeking.
Right-click and Files tab re-generate via API endpoint.
Bug fixes:
- #91: dockerApi() 10s socket timeout (Docker daemon hang)
- #77: await syncToAmpp() with .catch() — no longer fire-and-forget
- #75: migration 016 — add 'proxy','import' to job_type enum; add 'completed' to job_status
- #73: BullMQ orphan job cleanup on hard asset delete
- #70: batch-trim jobs table gets expires_at; trim-status auto-expires stale rows
- #66: scheduler tick marks stale live assets (>2h) as error
- #63: migration 017 — partial unique index prevents concurrent live asset overwrite
- #61: recorders.js uses getS3Bucket() not stale process.env.S3_BUCKET
- #60: already fixed (copy nulls proxy/thumbnail keys, requeues proxy)
- #40: already fixed (All projects clears openProject)
- #64: already fixed (sourceType/needsProxy handled)
- #90: GET /jobs now includes DB jobs table (trim jobs visible in UI)
- #74: nginx Content-Type header preserved; multer 500MB file size limit
- #68: GET /upload returns in-progress ingesting assets
- #58: /stream and /video endpoints fall back to original file for all video types
- #55: recorder poll .catch() logs auth errors cleanly; redirect stops interval
- #52: thumb-status and thumb-duration moved inside position:relative wrapper
- #50: ProjectCard gets onContextMenu handler with rename/delete menu
- #49: project context menu dismisses on contextmenu + scroll events
Features:
- #93: POST /assets/:id/reprocess?type=proxy|thumbnail — force re-queue any asset
Asset ⋯ menu now shows 'Re-generate proxy' and 'Re-generate thumbnail' buttons
UI:
- Logo: brightness(0) invert(1) filter applied consistently in sidebar, launcher,
and login — white logo pops on dark UI; inline style removed from login.html
DELETE /jobs/:id was throwing "404 not found" when the operator tried to
cancel a running job. BullMQ refuses job.remove() while a job is in the
active state; the route caught that error and fell through to the
404 branch, which was misleading because the job actually exists — the
queue was just refusing to drop it from under the worker.
Fix:
- Detect 'active' state explicitly and call moveToFailed(err, '0', false)
first. Token '0' bypasses the per-worker lock check (the operator-side
cancel doesn't hold the worker lock). That transitions active -> failed
and frees the queue's concurrency slot.
- If moveToFailed itself fails (lock owned by a live worker), fall back
to job.discard() so at least the result is thrown away.
- If remove() then fails (stalled, broken state), drop the job's Redis
key directly via queue.client. Last-resort obliteration.
- Stop swallowing getJob() errors — if Redis is sad, surface it via
next(err) instead of returning a misleading 404.
- Return { cancelled: true } when the job was active, so the client
can show "Cancelled" rather than "Removed" in any future toast.
While here: thumbnail jobs now run with concurrency 4 by default
(proxy 2, conform 1, import 1 unchanged). Every queue defaulted to
concurrency 1 before, so a single stalled job blocked the entire queue.
All three are overridable via PROXY_CONCURRENCY / THUMBNAIL_CONCURRENCY
/ CONFORM_CONCURRENCY env vars for nodes with more headroom.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Jobs screen only displayed an asset name when the enqueueing code
stuffed assetName into the BullMQ job data. YouTube imports did that;
upload-triggered proxy/thumbnail jobs didn't — so everything except
YouTube showed em dashes in the Asset column.
Fix it centrally: after we collect jobs from BullMQ, look up names
in one bulk SELECT against the assets table for any job that has an
assetId but no asset_name. Applies to /jobs, /jobs/:id, and the SSE
events stream. Lookup failures fall through silently rather than
500-ing the whole list.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds Ingest → YouTube. UI takes a URL + project, API enqueues a BullMQ
"import" job, worker shells out to yt-dlp, lands the MP4 in S3 at the
same originals/{assetId}/... path uploads use, then hands off to the
existing proxy queue. Imported assets share one lifecycle with uploads
from that point on.
Worker container picks up yt-dlp + python3 (apk on alpine, apt on the
GPU variant). The new 'import' queue is registered in jobs.js so it
appears in the Jobs SSE stream and retry/delete work for free.
Spec: docs/superpowers/specs/2026-05-23-youtube-importer-design.md
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
