79 lines
7.9 KiB
Markdown
79 lines
7.9 KiB
Markdown
|
---
|
||
|
|
name: Dragonflight MAM platform
|
||
|
|
description: Self-hosted media asset management replacing Grass Valley AMPP FramelightX. Now deployed directly on TrueNAS (zampp1/zampp2 retained as remote DeckLink workers). Repo renamed to forge.wilddragon.net/zgaetano/dragonflight on 2026-05-23.
|
||
|
|
type: project
|
||
|
|
originSessionId: 544a289a-0493-4194-9fbd-112ed250e221
|
||
|
|
---
|
||
|
|
Self-hosted MAM platform under active development. Stack: Node.js/Express, vanilla HTML/CSS/JS frontend, PostgreSQL 16, Redis 7 + BullMQ, S3-compatible (RustFS), FFmpeg, Blackmagic DeckLink SDK, Docker Compose. Repo: `forge.wilddragon.net/zgaetano/dragonflight` (renamed from `wild-dragon` on 2026-05-23 — Forgejo handles redirects for the old URL). On-disk checkout on TrueNAS is still at `/mnt/NVME/MAM/wild-dragon/` because docker-compose paths reference it; rename later if desired. Remote `origin` already updated to the new SSH URL.
|
||
|
|
|
||
|
|
**Why:** Replacing Grass Valley AMPP FramelightX with an in-house alternative. Test deployment is the production deployment — the user authorizes pushing directly to `main` and deploying to zampp1 + zampp2 for continued testing.
|
||
|
|
|
||
|
|
**How to apply:** Treat zampp1/zampp2 as authorized targets for MeshCentral / live deploy. Push to `main` directly when changes are validated. The repo's `deploy/onboard-node.sh` is the canonical worker provisioning path.
|
||
|
|
|
||
|
|
## Cluster topology (as of 2026-05-22)
|
||
|
|
- **zampp1** — primary; runs `mam-api`, `db`, `redis`, `web-ui`, `worker`, `capture` containers via `docker-compose.yml`. Postgres (5432) and Redis (6379) are now published on the host for worker-node connectivity.
|
||
|
|
- **zampp2** — worker ONLY; runs `node-agent` (host-network, `restart: unless-stopped`) via `docker-compose.worker.yml`. Does NOT run the primary stack. Has a **DeckLink Duo 2** (4 BNC ports at `/dev/blackmagic/io0..io3`).
|
||
|
|
- zampp2's `.env` points DATABASE_URL/REDIS_URL at `172.18.91.216` (zampp1). Using `.env` (not `.env.worker`); project name is `wild-dragon` (not the `wild-dragon-worker` that onboard-node.sh would create).
|
||
|
|
- Workers heartbeat to `POST /api/v1/cluster/heartbeat`; list at `GET /api/v1/cluster/`.
|
||
|
|
- Cluster registry: unique index on `cluster_nodes.hostname` (migration 007); `pickIp()` in `routes/cluster.js` now only flags `172.17.x` as docker bridge (NOT the full 172.16/12 block — real LAN 172.18.91.x was being falsely flagged).
|
||
|
|
|
||
|
|
## Codec selection (recorders)
|
||
|
|
Per-recorder columns added in migration 008 (`recording_*` and `proxy_*` for video bitrate, framerate, audio codec/bitrate/channels, container). `services/capture/src/capture-manager.js` exports `VIDEO_CODECS`, `AUDIO_CODECS`, `CONTAINER_FMT`, `CONTAINER_EXT` catalogs. `routes/recorders.js` `RECORDER_FIELDS` whitelist passes settings as env vars to the capture sidecar via `bootstrapAutoStart()`.
|
||
|
|
|
||
|
|
## DeckLink port picker
|
||
|
|
`services/web-ui/public/js/bmd-card.js` (window.BMDCards) renders an SVG diagram of the card with click-to-select ports. Models registered: Duo 2, Quad 2, Mini Recorder 4K, Mini Monitor 4K, UltraStudio 4K Mini. Backed by `GET /api/v1/cluster/devices/blackmagic` which flattens every node's DeckLink capabilities. **The `recorders.html` rewrite that consumes this was lost** in a context-compaction event — file on zampp1 still matches the old (pre-tabs, pre-SVG) version.
|
||
|
|
|
||
|
|
## LAN topology gotcha
|
||
|
|
The user's LAN is **172.18.91.0/24** — inside RFC1918 *and* inside Docker's reserved 172.16.0.0/12 range. zampp1=172.18.91.216, zampp2=172.18.91.217. `pickIp()` in `routes/cluster.js` was fixed (2026-05-22) to only flag 172.17.x as docker bridge, not all of 172.16/12. The smoke test in `deploy/test-cluster.sh` already had this fix.
|
||
|
|
|
||
|
|
## Pending work (as of 2026-05-22)
|
||
|
|
1. **`recorders.html` UI rewrite** — tabbed codec settings (Video/Audio/Container) for both master & proxy, node selector + BMD card SVG picker for SDI source. Renderer + endpoint are in `main`; the HTML that wires them up was lost to compaction and never reached Forgejo.
|
||
|
|
2. **GUI polish pass with flyonui MCP** — explicitly the LAST priority.
|
||
|
|
|
||
|
|
## DeckLink fix (issue #10, closed) — infra layer
|
||
|
|
- `recorders.js` now binds `/dev/blackmagic` for SDI local sidecars
|
||
|
|
- `resolveNodeTarget()` routes SDI sidecars to the correct remote node-agent
|
||
|
|
- `node-agent` v1.2.0: `POST /sidecar/start`, `DELETE /sidecar/:id`, `GET /sidecar/:id/status`
|
||
|
|
- `docker-compose.worker.yml`: node-agent gets `/var/run/docker.sock` + `LIVE_DIR`
|
||
|
|
- Sidecar port scheme for remote nodes: `7438 + deviceIndex` (device 0 → 7438, device 1 → 7439)
|
||
|
|
- All deployed to zampp1+zampp2 on 2026-05-22. FFmpeg now opens the card and reads audio.
|
||
|
|
|
||
|
|
## DeckLink end-to-end working (commit 30b4def, 2026-05-22)
|
||
|
|
- `services/capture/patch_decklink.py` now applies `services/capture/decklink-sdk16.patch` (~346 lines) via `git apply -R` instead of regex stunts
|
||
|
|
- The patch is `git diff release/7.1 origin/master -- 'libavdevice/decklink_{enc,dec,common}{.cpp,.h}'` — upstream's own SDK 16 migration that renames every IDeckLink* interface to its `_v14_2_1` versioned form consistently
|
||
|
|
- Verified zampp2 SDI1 recorder: 423 frames in 14 sec @ 29.97 fps, ProRes HQ 91 Mbps clean
|
||
|
|
- **Why the old patch failed:** Fix 1 renamed the allocator class to `_v14_2_1`, but Fixes 2+3 stubbed out the matching `SetVideoInputFrameMemoryAllocator` call and replaced `videoFrame->GetBytes()` with a QueryInterface-around-IDeckLinkVideoBuffer dance — mixing two different API generations. Compiled, didn't crash, silently dropped every video frame
|
||
|
|
- **How to apply:** When SDK or FFmpeg revs, regenerate the patch with `git diff <old> <new> -- libavdevice/decklink_*` against an FFmpeg tree built with `--enable-decklink` and drop into `services/capture/decklink-sdk16.patch`
|
||
|
|
- **Bisection trick:** Build the BMD-provided Linux Capture sample from `services/capture/sdk/extracted/Blackmagic DeckLink SDK 16.0/Linux/Samples/Capture` to ground-truth whether a frame-zero problem is signal vs FFmpeg
|
||
|
|
- DeckLink Duo Mini active ports are 0 and 1 (Duo (1) and Duo (2)); ports 2-3 are inactive on this card
|
||
|
|
|
||
|
|
## zampp1 vs zampp2 layout (as of 2026-05-22)
|
||
|
|
- Repos checked out at `/opt/wild-dragon` on both
|
||
|
|
- zampp2 hostname is `zampp2`; zampp1 hostname is **`zampp`** (mam-api sets `NODE_HOSTNAME=zampp1` so `cluster_nodes.hostname` is `zampp1`)
|
||
|
|
- zampp2 runs ONLY node-agent now (primary stack torn down 2026-05-22)
|
||
|
|
- DeckLink Duo Mini device paths are `/dev/blackmagic/io0..io3` (NOT `dv0..dv3`); only port io1 (device index 1) shows any signal during testing
|
||
|
|
- mam-api is published on **port 47432** on zampp2's `.env` (historical — zampp2 no longer runs mam-api)
|
||
|
|
- Git creds installed at `/root/.git-credentials` on both zampp1 and zampp2
|
||
|
|
|
||
|
|
## Pushed commits (already on `main`)
|
||
|
|
- `3b4af6e` node-agent: prefer host LAN IP, NODE_IP override
|
||
|
|
- `0efef0d` cluster route: pickIp() + /devices/blackmagic endpoint
|
||
|
|
- `a39c983` migration 007: dedupe hostnames + unique index
|
||
|
|
- `049beb8` migration 008: expanded codec columns
|
||
|
|
- `40a66ba` worker compose: network_mode: host for node-agent
|
||
|
|
- `f4a83ee` capture-manager: dynamic ffmpeg args
|
||
|
|
- `485af25` capture index.js: bootstrap reads codec env vars
|
||
|
|
- `4c65753` recorders route: full codec field whitelist
|
||
|
|
- `0ebb3cf` onboard-node: auto-detect host LAN IP
|
||
|
|
- `d39f86d` web-ui: bmd-card.js
|
||
|
|
- `97628bb` chore: remove cloudflare rate-limit probe (.touchtest)
|
||
|
|
- `8aa3783` deploy: test-cluster.sh + .touchtest cleanup
|
||
|
|
- `4a3a672` cluster: stable hostname for mam-api (NODE_HOSTNAME env), jq-based smoke test
|
||
|
|
- `486e3c2` fix(decklink): mount /dev/blackmagic in sidecar + remote node routing via node-agent
|
||
|
|
- `30b4def` fix(decklink): proper SDK 16 patch via git apply -R
|
||
|
|
- `00f3f29` feat(cluster): expose db/redis ports for worker-node connectivity
|
||
|
|
- `37767f9` fix(cluster): pickIp() only treats 172.17.x as docker bridge
|
||
|
|
|
||
|
|
## Git creds on zampp1
|
||
|
|
`/root/.git-credentials` holds the Forgejo PAT for user `zgaetano`. `credential.helper=store` is set in `/root/.gitconfig`. Future pushes from zampp1 just need `HOME=/root` exported (the MeshCentral agent's default $HOME is `/usr/local/mesh_services/Mesh Dragon/MeshDragon`). Rotate the PAT by overwriting that one file.
|