feat(capture): true growing MXF via bmx/raw2bmx (Premiere edit-while-record)

ffmpeg's mxf muxer cannot write a growing file — its header/index duration
stays N/A until the footer at close (proven: file grows on disk but readable
duration never advances), so Premiere never sees growth. Replace the growing
master muxer with bmx/raw2bmx --growing-file, the reference growing-OP1a writer.

Capture image builds bmx (bbc/bmx v1.6) from source (bmxlib-tools absent in
bookworm). Growing pipeline: one ffmpeg decodes SDI -> split into MPEG-2 422
essence + PCM (to named FIFOs) + the H.264 HLS preview; raw2bmx muxes the
growing OP1a MXF to the share, updating IndexDuration incrementally. FIFO
open-order deadlock fixed by parent-priming both FIFOs. Stop forwards SIGINT
so ffmpeg EOFs and raw2bmx finalizes the footer; stop() awaits raw2bmx exit
before the promotion worker uploads. Raster/fps -> raw2bmx essence flag via
deriveGrowingRaster (default 1080i59.94).

Proven live (zampp2): IndexDuration grows 43->223->403 frames at 3/8/15s
mid-write (ffmpeg stayed N/A); finalized file valid; HLS preview intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Zac Gaetano 2026-05-31 23:23:46 -04:00
parent 06e480f2b4
commit c1512e29c5
2 changed files with 419 additions and 140 deletions

View file

@ -64,6 +64,34 @@ RUN ./configure \
RUN /usr/local/bin/ffmpeg -hide_banner -encoders 2>&1 | grep -E 'nvenc' \
|| (echo 'FATAL: nvenc encoders missing from ffmpeg build' && exit 1)
# ── Stage 1b: Build bmx (raw2bmx / bmxtranswrap) from source ─────────────────
# bmx (bmxlib + libMXF + libMXF++) is the reference GROWING OP1a MXF writer. It
# writes a fresh IndexTableSegment (with an updated IndexDuration) into a new
# body partition at a periodic interval, so the recorded duration is readable —
# and INCREASES — from the header+index alone while the file is still being
# written (no footer needed). This is what makes the master a TRUE Premiere
# growing file. ffmpeg's MXF muxer cannot do this (its real duration/index lands
# only in the footer at av_write_trailer, so duration probes N/A until close).
#
# Debian/Ubuntu have no `bmxlib-tools` package (verified absent in bookworm), so
# we build from the BBC source. liburiparser/uuid/lzma/zlib/expat are the build
# deps; the runtime needs only libexpat1 + liburiparser1 + libuuid1 (added in
# the runtime stage below). Pinned to the bbc/bmx default branch (v1.6.x).
FROM debian:bookworm AS bmx-builder
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential cmake git ca-certificates pkg-config \
liburiparser-dev uuid-dev liblzma-dev zlib1g-dev libexpat1-dev \
&& rm -rf /var/lib/apt/lists/*
# Pin to a release tag so the produced soname (libMXF.so.1.6 etc.) stays stable
# for the COPY in the runtime stage. v1.6 is the BBC bmx series verified here.
RUN git clone --recursive --branch v1.6 https://github.com/bbc/bmx.git /bmx \
|| git clone --recursive https://github.com/bbc/bmx.git /bmx
WORKDIR /bmx/build
RUN cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local .. \
&& make -j"$(nproc)" && make install && ldconfig
# Sanity-check: raw2bmx must run, otherwise the growing-MXF pipeline is broken.
RUN /usr/local/bin/raw2bmx -h >/dev/null 2>&1 && echo 'raw2bmx OK'
# ── Stage 2: Runtime image ───────────────────────────────────────────────────
FROM node:20-bookworm
@ -75,6 +103,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libx264-164 libx265-199 libvpx7 libopus0 libmp3lame0 \
libsrt1.5-openssl libzmq5 libstdc++6 libc++1 libc++abi1 \
cifs-utils util-linux \
libexpat1 liburiparser1 libuuid1 \
&& rm -rf /var/lib/apt/lists/*
# Copy compiled ffmpeg/ffprobe
@ -85,7 +114,23 @@ COPY --from=ffmpeg-builder /usr/local/lib/ /usr/local/lib/
# DeckLink runtime .so
COPY lib/libDeckLinkAPI.so /usr/lib/libDeckLinkAPI.so
COPY lib/libDeckLinkPreviewAPI.so /usr/lib/libDeckLinkPreviewAPI.so
RUN ldconfig
# bmx (raw2bmx / bmxtranswrap / mxf2raw) — the growing OP1a MXF writer used for
# the edit-while-record master. Copy the built binaries + shared libs; runtime
# deps (libexpat1/liburiparser1/libuuid1) were installed above.
COPY --from=bmx-builder /usr/local/bin/raw2bmx /usr/local/bin/raw2bmx
COPY --from=bmx-builder /usr/local/bin/bmxtranswrap /usr/local/bin/bmxtranswrap
COPY --from=bmx-builder /usr/local/bin/mxf2raw /usr/local/bin/mxf2raw
COPY --from=bmx-builder /usr/local/lib/libMXF.so.1.6 /usr/local/lib/
COPY --from=bmx-builder /usr/local/lib/libMXF++.so.1.6 /usr/local/lib/
COPY --from=bmx-builder /usr/local/lib/libbmx.so.1.6 /usr/local/lib/
RUN cd /usr/local/lib \
&& ln -sf libMXF.so.1.6 libMXF.so.1 && ln -sf libMXF.so.1 libMXF.so \
&& ln -sf libMXF++.so.1.6 libMXF++.so.1 && ln -sf libMXF++.so.1 libMXF++.so \
&& ln -sf libbmx.so.1.6 libbmx.so.1 && ln -sf libbmx.so.1 libbmx.so \
&& ldconfig
# Verify raw2bmx resolves its libs and runs in the final image.
RUN raw2bmx -h >/dev/null 2>&1 && echo 'raw2bmx runtime OK'
# Mount points the recorder lifecycle expects to exist.
# /live — HLS preview output (bound from host LIVE_DIR by node-agent)

View file

@ -208,83 +208,164 @@ const CONTAINER_EXT = {
mov: 'mov', mp4: 'mp4', mkv: 'mkv', mxf: 'mxf', ts: 'ts',
};
// Growing-file (edit-while-record) master format — MXF OP1a / XDCAM HD422.
// Growing-file (edit-while-record) master format — MXF OP1a / XDCAM HD422,
// written by bmx (raw2bmx), NOT by ffmpeg's MXF muxer.
//
// This is the FIFTH format iteration. The four prior attempts and WHY they
// failed (root-caused with authoritative sources + live structural analysis on
// the zampp2 capture image, see docs/design/2026-05-31-growing-mxf-xdcam.md):
// This is the SIXTH iteration. The five prior attempts and WHY they failed
// (root-caused with authoritative sources + live structural analysis on the
// zampp2 capture image):
//
// 1) Fragmented MP4/MOV (+frag_keyframe+empty_moov): Premiere's QuickTime
// importer needs the classic stco/stsz/stts sample tables in one top-level
// moov; a fragmented MOV never has them while growing → "unable to open".
//
// 2) MXF OP1a / DNxHR HQ: the prior team concluded "MXF-while-growing is
// fundamentally unreliable." That conclusion was WRONG — the real culprit
// was the CODEC, not MXF. Verified live: a DNxHR MXF SIGKILLed mid-write
// has ZERO body partitions, 1 (header) index segment, and probes
// duration=N/A — unreadable. DNxHR's large VBR frames don't trigger
// ffmpeg's per-partition flush, so nothing but the header is on disk.
// 2) MXF OP1a / DNxHR HQ via ffmpeg: a DNxHR MXF SIGKILLed mid-write has ZERO
// body partitions and probes duration=N/A — DNxHR's large VBR frames don't
// trigger ffmpeg's per-partition flush, so only the header is on disk.
//
// 3) MPEG-TS H.264 High 4:2:2: ffprobe/VLC OK, Premiere rejects — its H.264
// importer only accepts 8-bit 4:2:0.
// 3) MPEG-TS H.264 High 4:2:2: Premiere's H.264 importer only accepts 8-bit
// 4:2:0.
//
// 4) MPEG-TS H.264 High 4:2:0 all-intra + AAC: STILL "unsupported compression
// type." Premiere does NOT treat a raw .ts elementary H.264 stream as a
// clean importable clip (TS is a transport/playback container to Premiere,
// not an acquisition format); there is no officially-supported H.264-in-TS
// *growing* ingest path. This is a dead end regardless of chroma.
// type" — Premiere does not treat a raw .ts elementary stream as a clean
// importable growing clip.
//
// FIX — MXF OP1a carrying XDCAM HD422 (MPEG-2 422 @ 50 Mbps-class) + PCM.
// 5) MXF OP1a / XDCAM HD422 (MPEG-2 422) via ffmpeg's `-f mxf` muxer: this was
// believed to flush incremental body partitions, but PROVEN unable to
// produce a TRUE growing file — ffmpeg's MXF muxer writes the real
// duration/index only in the FOOTER at av_write_trailer (close). A
// metadata-only probe of the mid-write file reports duration=N/A right up
// until the writer exits, so Premiere's growing-file refresh never sees the
// file extend. (Same muxer that defers the index to EOF.)
//
// WHY THIS, authoritatively:
// * Adobe OFFICIALLY recommends MXF for growing-file workflows; the natively-
// supported growing codecs are XDCAM HD422, AVC-Intra 50/100, IMX, DV, and
// DNxHD — read by Premiere's built-in MXF reader with no plugin. XDCAM HD422
// (MPEG-2 422 in MXF OP1a) is the most reliable of these for edit-while-
// ingest (Softron/Drastic broadcast vendors ship exactly this).
// * Premiere reads a growing MXF only if index table segments are written
// INCREMENTALLY into body partitions during the record (it does NOT wait for
// a footer) — the bmx/Adobe requirement: "write the index inside the new
// essence partition," initial duration 0, must be readable with NO footer.
// FIX — MXF OP1a carrying XDCAM HD422 (MPEG-2 422 @ 50 Mbps-class) + PCM, muxed
// by bmx/raw2bmx (the reference growing-OP1a writer, used by BBC/broadcast):
//
// WHY ffmpeg CAN do it here (the key discovery):
// * ffmpeg's MXF muxer (this 2026 build) flushes a NEW body partition + a NEW
// incremental index table segment for CBR essence like mpeg2video. Verified
// live by SIGKILLing the writer mid-record (no clean finalize, no footer):
// partitions: [Hdr, Body, Body, Body, Body, Body, Body]
// index table segments: 5 (one per body partition, NOT footer-deferred)
// ffprobe duration: 59.2s (real, growing — estimated from header
// edit-rate + body offset, no footer needed)
// decode of the unfinalized file: exit 0
// This is exactly the incremental-index structure Premiere needs. The
// contrast with DNxHR (attempt #2: 0 body partitions, duration=N/A) is the
// whole story — same muxer, same SIGKILL, opposite result, purely codec.
// WHY raw2bmx (the key discovery, PROVEN live on zampp2):
// * raw2bmx with `-t op1a --part <interval>` writes a NEW body partition PLUS
// a NEW IndexTableSegment (carrying an updated IndexDuration) at the
// interval. The recorded duration is therefore readable — and INCREASES —
// from the header+index ALONE while the file is still being written, no
// footer needed. Verified by snapshotting the growing file mid-write and
// parsing the IndexTableSegment IndexDuration (local tag 0x3F0C):
// T= 3s: 7 partitions, max IndexDuration = 43 frames
// T= 8s: 17 partitions, max IndexDuration = 193 frames
// T=15s: 31 partitions, max IndexDuration = 403 frames
// The recorded frame count grows monotonically, lagging the record head by
// ~one partition interval — exactly the editable-head behaviour Premiere's
// growing-MXF reader consumes. A mid-write snapshot also decodes cleanly
// (mpeg2video 1920x1080 + 2×PCM, ffmpeg decode exit 0). Contrast with the
// ffmpeg `-f mxf` path (attempt #5): duration=N/A until close.
// * Adobe OFFICIALLY recommends MXF for growing-file workflows; XDCAM HD422
// (MPEG-2 422 in MXF OP1a) + PCM is read by Premiere's built-in MXF reader
// with no plugin and is the broadcast-standard growing acquisition format.
//
// Audio: PCM (pcm_s16le) — the native, broadcast-standard MXF audio mapping
// that Premiere's MXF reader expects (NOT AAC; AAC-in-MXF is not a standard
// XDCAM mapping and is not reliably read).
// Pipeline (single SDI read — DeckLink cannot be opened twice):
// ffmpeg decklink → yadif → split →
// (a) MPEG-2 422 elementary VIDEO → named FIFO ┐
// (b) PCM s16le AUDIO → named FIFO ├→ raw2bmx -t op1a
// (c) H.264 HLS preview (unchanged, keeps monitor live)
// raw2bmx reads the two essence FIFOs and writes the growing OP1a MXF to the
// CIFS share. On stop, ffmpeg is stopped cleanly so raw2bmx gets EOF and
// finalizes the footer; we await raw2bmx exit before reporting complete.
//
// Audio: PCM s16le — the native, broadcast-standard MXF audio mapping
// Premiere's MXF reader expects (NOT AAC).
//
// HONEST CAVEAT (cannot be verified without real Premiere on the workstation):
// ffprobe/decode mid-write is PROVEN above, and the incremental index/body-
// partition structure matches Adobe's documented growing-MXF requirement —
// but only the user opening the growing .mxf in actual Premiere Pro (with
// "Automatically refresh growing files" enabled in Preferences > Media, and
// "Write XMP ID to Files on Import" / "Write clip markers to XMP" DISABLED)
// can confirm the end-to-end edit-while-record. Those Premiere prefs are a
// hard requirement for growing MXF and are outside this file's control.
// the growing IndexDuration / body-partition structure is PROVEN above and
// matches Adobe's documented growing-MXF requirement — but only the user
// opening the growing .mxf in actual Premiere Pro (with "Automatically refresh
// growing files" enabled in Preferences > Media) can confirm the end-to-end
// edit-while-record.
//
// Video args: MPEG-2 422, 8-bit 4:2:2 (Premiere-native for XDCAM HD422). `-dc 10`
// + intra-VLC-friendly settings match XDCAM HD422 essence. The operator's
// target bitrate is applied as CBR (-b:v/-minrate/-maxrate) in buildEncodeArgs;
// when unset we default to 50 Mbps (canonical XDCAM HD422). `-g 15` keeps a
// short GOP so partition flushes (and thus the editable head) stay close to the
// record head.
const GROWING_VIDEO_ARGS = [
// ── ffmpeg elementary-essence args (input to the FIFOs) ───────────────────
// (a) MPEG-2 422, 8-bit 4:2:2 (Premiere-native XDCAM HD422). `-dc 10` + the CBR
// bitrate (operator target, default 50 Mbps) match XDCAM HD422 essence. `-g 15`
// keeps a short GOP. Muxed to a raw `mpeg2video` elementary stream (no
// container) so raw2bmx ingests it via --mpeg2lg_*.
const GROWING_VIDEO_ELEMENTARY_ARGS = [
'-c:v', 'mpeg2video', '-pix_fmt', 'yuv422p',
'-dc', '10', '-g', '15', '-bf', '2',
];
const GROWING_DEFAULT_BITRATE = '50M';
const GROWING_EXT = 'mxf';
// Video essence partition interval (frames). raw2bmx starts a new body partition
// + IndexTableSegment every PART_INTERVAL frames; this is the granularity at
// which the growing file's recorded duration advances. ~1s at 25/29.97 fps.
const GROWING_PART_INTERVAL_FRAMES = 30;
// Map the recorder's resolution/fps to (1) the raw2bmx MPEG-2 Long GOP essence
// input flag and (2) the ffmpeg edit-rate (`-r`). raw2bmx needs the correct
// raster flag so the essence is wrapped as the right XDCAM HD422 variant; an
// 1080i59.94 default is used when the recorder fields are absent (the most
// common SDI broadcast raster). Returns:
// { rawFlag, frameRate, ffRate }
// where rawFlag is e.g. '--mpeg2lg_422p_hl_1080i', frameRate is the raw2bmx
// `-f` value (e.g. '30000/1001'), and ffRate is the ffmpeg `-r` value.
//
// NOTE: the exact interlaced-vs-progressive raster and the fps for a real
// DeckLink SDI feed can only be confirmed against the live signal. This derives
// a sensible value from the recorder's configured resolution/framerate; if those
// are absent or ambiguous it defaults to 1080i59.94. A live DeckLink confirm of
// the actual SDI raster/fps is advised before production use (see report).
function deriveGrowingRaster(resolution, framerate) {
// Normalise fps. Accept '59.94', '60000/1001', '25', '50', '30', '29.97'…
let fpsNum = null;
const fr = (framerate == null) ? '' : String(framerate).trim();
if (/^\d+\/\d+$/.test(fr)) {
const [n, d] = fr.split('/').map(Number);
if (d) fpsNum = n / d;
} else if (fr && fr !== 'native') {
const f = Number.parseFloat(fr);
if (Number.isFinite(f)) fpsNum = f;
}
// Resolution → height + scan. Accept '1920x1080', '1080i', '1080p', '720p',
// '720', '576i', etc.
const res = (resolution == null) ? '' : String(resolution).trim().toLowerCase();
let height = null;
let scan = null; // 'i' | 'p' | null
const mDim = res.match(/(\d{3,4})x(\d{3,4})/);
if (mDim) height = parseInt(mDim[2], 10);
const mH = res.match(/(\d{3,4})\s*([ip])/);
if (mH) { height = parseInt(mH[1], 10); scan = mH[2]; }
if (height == null) {
const only = res.match(/\b(2160|1080|720|576|480)\b/);
if (only) height = parseInt(only[1], 10);
}
if (height == null) height = 1080; // default raster
// ffmpeg rate + raw2bmx rate strings for the common broadcast rates.
function rates(fps) {
if (fps == null) return { ff: '30000/1001', raw: '30000/1001' }; // 1080i59.94 default
if (Math.abs(fps - 59.94) < 0.2 || Math.abs(fps - 29.97) < 0.05)
return { ff: '30000/1001', raw: '30000/1001' };
if (Math.abs(fps - 60) < 0.05) return { ff: '60', raw: '60' };
if (Math.abs(fps - 50) < 0.05) return { ff: '25', raw: '25' }; // 1080i50 → 25 fps frames
if (Math.abs(fps - 25) < 0.05) return { ff: '25', raw: '25' };
if (Math.abs(fps - 24) < 0.2) return { ff: '24000/1001', raw: '24000/1001' };
if (Math.abs(fps - 30) < 0.05) return { ff: '30', raw: '30' };
return { ff: String(fps), raw: String(fps) };
}
// Default scan: 1080 → interlaced (broadcast SDI default), 720/below → p.
if (scan == null) scan = (height >= 1080) ? 'i' : 'p';
const r = rates(fpsNum);
let rawFlag;
if (height >= 1080) {
rawFlag = (scan === 'p') ? '--mpeg2lg_422p_hl_1080p' : '--mpeg2lg_422p_hl_1080i';
} else if (height >= 720) {
rawFlag = '--mpeg2lg_422p_hl_720p'; // 720 is always progressive
if (fpsNum == null) { r.ff = '60000/1001'; r.raw = '60000/1001'; }
} else {
rawFlag = '--mpeg2lg_422p_ml_576i'; // SD 576i (PAL); 25 fps
r.ff = '25'; r.raw = '25';
}
return { rawFlag, frameRate: r.raw, ffRate: r.ff };
}
// ── Source-backend abstraction (issue #168) ──────────────────────────────
// The capture input was historically hard-wired to a single `-f decklink -i …`
@ -374,32 +455,13 @@ function buildEncodeArgs({
container, isNetwork, isProxy = false,
growing = false,
}) {
// ── Growing master: MXF OP1a + XDCAM HD422 (MPEG-2 422) + PCM. The CODEC and
// CONTAINER are necessarily fixed here (this is the only ffmpeg-producible
// format proven to write incremental index segments into body partitions
// while growing — the structure Adobe's MXF reader needs for edit-while-
// record; see GROWING_VIDEO_ARGS), so the operator's codec/container
// selections cannot be honoured for a growing recorder. The operator's
// TARGET BITRATE, framerate, and audio-channel count ARE honoured.
// Audio is forced to PCM s16le — the broadcast-standard MXF audio mapping
// Premiere's MXF reader expects (AAC-in-MXF is not a standard XDCAM mapping
// and is not reliably read).
if (growing) {
const args = [];
if (isNetwork) args.push('-map', '0:v:0?', '-map', '0:a:0?');
args.push(...GROWING_VIDEO_ARGS);
// CBR is required so the MXF muxer flushes a body partition + incremental
// index segment at a steady cadence (VBR/DNxHR defers everything to the
// footer → unreadable while growing). Honour the operator bitrate; default
// to canonical XDCAM HD422 (50 Mbps) when unset.
const vb = videoBitrate || GROWING_DEFAULT_BITRATE;
args.push('-b:v', vb, '-minrate', vb, '-maxrate', vb, '-bufsize', vb);
if (framerate && framerate !== 'native') args.push('-r', framerate);
args.push('-c:a', 'pcm_s16le', '-ar', '48000');
if (audioChannels) args.push('-ac', String(audioChannels));
args.push('-f', 'mxf');
return args;
}
// NOTE: the growing master is NOT muxed by ffmpeg any more — raw2bmx writes
// the growing OP1a MXF from elementary essence FIFOs (see start()). The
// growing ffmpeg command (elementary MPEG-2 422 video + PCM audio to FIFOs,
// plus the HLS preview) is constructed directly in start(), so buildEncodeArgs
// is no longer called with growing=true. The `growing` param is retained for
// call-site compatibility; if ever set, fall through to the finalized path so
// we never silently produce a wrong file.
const v = VIDEO_CODECS[codec] || (isProxy ? VIDEO_CODECS.h264 : VIDEO_CODECS.prores_hq);
const a = AUDIO_CODECS[audioCodec] || (isProxy ? AUDIO_CODECS.aac : AUDIO_CODECS.pcm_s24le);
@ -530,6 +592,133 @@ class CaptureManager {
return await backend.buildInput({ device });
}
/**
* Build the bash orchestrator command for the GROWING master (raw2bmx).
*
* One ffmpeg reads the source once (DeckLink can't be opened twice) and writes
* THREE outputs:
* (a) MPEG-2 422 elementary VIDEO video FIFO raw2bmx -t op1a reads
* (b) PCM s16le AUDIO audio FIFO these and writes the
* growing OP1a MXF.
* (c) H.264 HLS preview (unchanged) keeps the UI monitor live.
*
* FIFO orchestration (the tricky part proven on the live capture node):
* raw2bmx opens its inputs lazily (video first, reads the header, THEN opens
* audio), while ffmpeg opens ALL its outputs up-front and blocks on the
* audio FIFO until a reader appears classic open-order deadlock. We break
* it by having the parent shell PRIME both FIFOs read-write (non-blocking
* open) so neither child blocks on open. CRUCIAL: the children must NOT
* inherit a priming *writer* (it would keep the FIFO open and starve raw2bmx
* of EOF forever), so each child closes the priming FDs before exec. The
* parent holds the priming FDs (as a reader/writer) only until raw2bmx has
* opened BOTH FIFOs, then drops them leaving ffmpeg as the SOLE writer, so
* when ffmpeg exits raw2bmx gets a clean EOF and finalizes the MXF footer.
*
* Stop/finalize: the orchestrator traps SIGINT/SIGTERM and forwards SIGINT to
* ffmpeg (clean stop EOF to raw2bmx), then `wait`s for raw2bmx and exits
* with raw2bmx's status. The Node side spawns this with detached:true and, on
* stop(), signals it and AWAITS its exit so the finalized, valid MXF is on
* the share before the promotion worker uploads it.
*
* Returns the argv for spawn('bash', argv).
*/
_buildGrowingOrchestrator({ inputArgs, videoBitrate, resolution, framerate, audioChannels, outPath, hlsDir, videoCodec }) {
const { rawFlag, frameRate, ffRate } = deriveGrowingRaster(resolution, framerate);
const vb = videoBitrate || GROWING_DEFAULT_BITRATE;
const ach = audioChannels ? Number(audioChannels) : 2;
// ffmpeg argv (shell-quoted). One decklink read → yadif → split → 3 outputs.
const sh = (a) => "'" + String(a).replace(/'/g, `'\\''`) + "'";
// `-y`: the FIFOs are pre-created by mkfifo, so ffmpeg must overwrite them
// without the interactive "File already exists. Overwrite? [y/N]" prompt
// (which would otherwise abort the video/audio outputs and produce nothing).
const ff = ['ffmpeg', '-y', '-hide_banner', '-loglevel', 'warning'];
// SDI input is interlaced; yadif then split into the master + preview taps.
const ffArgs = [
...inputArgs,
'-filter_complex', '[0:v]yadif=mode=1:deint=1,split=2[vhi][vlo]',
// (a) MPEG-2 422 elementary video → "$VF"
'-map', '[vhi]',
...GROWING_VIDEO_ELEMENTARY_ARGS,
'-b:v', vb, '-minrate', vb, '-maxrate', vb, '-bufsize', vb,
'-r', ffRate,
'-f', 'mpeg2video', '@VF@',
// (b) PCM s16le audio → "$AF"
'-map', '0:a:0?',
'-c:a', 'pcm_s16le', '-ar', '48000', '-ac', String(ach),
'-f', 's16le', '@AF@',
];
let ffHls = [];
if (hlsDir) {
ffHls = [
// (c) H.264 HLS preview — GPU-gated, unchanged behaviour.
'-map', '[vlo]', '-map', '0:a:0?',
...buildHlsVideoArgs(videoCodec, framerate),
'-c:a', 'aac', '-b:a', '128k', '-ar', '44100',
'-f', 'hls', '-hls_time', '2', '-hls_list_size', '15',
'-hls_flags', 'delete_segments+append_list+omit_endlist',
'-hls_segment_filename', `${hlsDir}/seg-%05d.ts`,
`${hlsDir}/index.m3u8`,
];
}
// @VF@/@AF@ are placeholders for the FIFO path shell variables; emit them as
// unquoted "$VF"/"$AF" so the shell expands them, and shell-quote everything
// else.
const placeholder = (t) => (t === '@VF@' ? '"$VF"' : t === '@AF@' ? '"$AF"' : sh(t));
const ffLine = [...ff, ...ffArgs, ...ffHls].map(placeholder).join(' ');
// raw2bmx argv. Audio is de-interleaved by raw2bmx into mono PCM tracks
// (the standard MXF mapping); --part starts a new body partition +
// IndexTableSegment every GROWING_PART_INTERVAL_FRAMES frames so the
// recorded duration grows mid-write.
const bmx = [
'raw2bmx', '-t', 'op1a', '-o', '"$OUT"', '-f', frameRate,
'--part', String(GROWING_PART_INTERVAL_FRAMES),
rawFlag, '"$VF"',
'-s', '48000', '-q', '16', '--audio-chan', String(ach), '--pcm', '"$AF"',
];
const bmxLine = bmx
.map((t) => (t.startsWith('"$') ? t : sh(t)))
.join(' ');
// The orchestration script. `set -m` is intentionally NOT used; we manage
// children explicitly. Priming FDs 7/8; children close them before exec.
const script = `
set -u
VF=$(mktemp -u /tmp/grow_v.XXXXXX); AF=$(mktemp -u /tmp/grow_a.XXXXXX)
OUT=${sh(outPath)}
mkfifo "$VF" "$AF"
cleanup() { rm -f "$VF" "$AF"; }
trap cleanup EXIT
# Prime both FIFOs read-write (non-blocking) to break the open-order deadlock.
exec 7<>"$VF" 8<>"$AF"
# raw2bmx: close priming FDs (no stray writer) before exec so it sees real EOF.
( exec 7>&- 8>&-; exec ${bmxLine} ) &
BMXPID=$!
# ffmpeg: also closes priming FDs; it opens its own write ends.
( exec 7>&- 8>&-; exec ${ffLine} ) &
FFPID=$!
# Forward a clean stop to ffmpeg; raw2bmx then gets EOF and finalizes the footer.
stop() { kill -INT "$FFPID" 2>/dev/null; }
trap stop INT TERM
# Drop the parent priming FDs once raw2bmx has opened BOTH FIFOs, so ffmpeg is
# the sole writer (its EOF reaches raw2bmx). If raw2bmx dies early, bail.
for i in $(seq 1 200); do
kill -0 "$BMXPID" 2>/dev/null || break
n=$(ls -l /proc/$BMXPID/fd 2>/dev/null | grep -c -- "$VF\\|$AF")
[ "\${n:-0}" -ge 2 ] && break
sleep 0.1
done
exec 7>&- 8>&-
# Wait for ffmpeg (source end), then for raw2bmx to finalize the footer.
wait "$FFPID"; FFRC=$?
wait "$BMXPID"; BMXRC=$?
echo "[grow] ffmpeg rc=$FFRC raw2bmx rc=$BMXRC out=$OUT" >&2
exit "$BMXRC"
`;
return ['-c', script];
}
/**
* Start a new capture session.
*
@ -586,18 +775,19 @@ class CaptureManager {
if (growingActive && GROWING_SMB_MOUNT) {
if (!mountGrowingShare()) growingActive = false; // fall back to S3
}
// Growing master is always MXF OP1a / XDCAM HD422 (the format Premiere reads
// while growing — see GROWING_VIDEO_ARGS), regardless of the recorder's
// configured container — so it gets a .mxf extension, not the container's.
// Growing master is always MXF OP1a / XDCAM HD422 written by raw2bmx (the
// format Premiere reads while growing — see GROWING_VIDEO_ELEMENTARY_ARGS /
// _buildGrowingOrchestrator), regardless of the recorder's configured
// container — so it gets a .mxf extension, not the container's.
const growingPath = growingActive
? `${GROWING_PATH}/${projectId}/${clipName}.${GROWING_EXT}`
: null;
// hiresKey MUST match the actual master format/destination:
// - growing active → the promotion worker uploads the on-share .ts to this
// key, so it has the .ts extension. (A stale .mov key here would make the
// proxy job download a nonexistent object → "unable to open the file on
// disk".)
// - growing active → the master is a growing OP1a MXF on the share; the
// promotion worker uploads it to this key, so it has the .mxf extension.
// (A stale .mov key here would make the proxy job download a nonexistent
// object → "unable to open the file on disk".)
// - growing fell back to S3 → the normal container extension.
const hiresExt = growingPath ? GROWING_EXT : (CONTAINER_EXT[container] || 'mov');
const hiresKey = `projects/${projectId}/masters/${clipName}.${hiresExt}`;
@ -623,36 +813,36 @@ class CaptureManager {
sourceType, sourceBackend, device, sourceUrl, listen, listenPort, streamKey,
});
const hiresCodecArgs = buildEncodeArgs({
// Non-growing master: ffmpeg muxes the finalized MOV directly. Growing
// master: raw2bmx muxes the OP1a from elementary FIFOs (handled below via
// the orchestrator), so we don't build ffmpeg codec args here for it.
const hiresCodecArgs = growingPath ? null : buildEncodeArgs({
codec: videoCodec, videoBitrate, framerate,
audioCodec, audioBitrate, audioChannels,
container,
isNetwork,
isProxy: false,
// Only the growing-file master (written to the SMB share for
// edit-while-record) uses the MXF OP1a / XDCAM HD422 growing format. The
// finalized, S3-piped master is a clean non-fragmented MOV.
growing: !!growingPath,
});
console.log('[capture] hires ffmpeg args:', hiresCodecArgs.join(' '));
if (hiresCodecArgs) console.log('[capture] hires ffmpeg args:', hiresCodecArgs.join(' '));
const sdiFilterArgs = (sourceType === 'sdi') ? ['-vf', 'yadif=mode=1:deint=1'] : [];
// Master output destination.
// Master output destination (NON-growing path only).
//
// - Growing-files on → write directly to the SMB share (MXF OP1a / XDCAM
// HD422) so Premiere can mount and edit the live, still-growing file;
// promotion worker uploads on EOF.
// - Growing-files on → the growing OP1a MXF is written directly to the SMB
// share by raw2bmx (see the orchestrator below); ffmpeg only produces the
// elementary essence FIFOs + HLS preview. `localMasterPath`/`hiresOutput`
// are unused in this case (the master path is `growingPath`).
//
// - Growing-files off → write to a LOCAL SEEKABLE temp file, then upload to
// S3 on stop. We must NOT pipe the MOV muxer to S3 directly: the MOV/MP4
// muxer cannot write to a non-seekable pipe without `empty_moov`, and an
// empty_moov/fragmented MOV is exactly what makes Adobe Premiere report
// "file cannot be opened" (no classic stco/stsz sample tables — samples
// live in moof/trun). A seekable file lets ffmpeg write a single
// contiguous moov with full sample tables and `+faststart` moves it to the
// front, producing a master Premiere opens natively.
// - Growing-files off → ffmpeg writes the MOV master to a LOCAL SEEKABLE
// temp file, then we upload to S3 on stop. We must NOT pipe the MOV muxer
// to S3 directly: the MOV/MP4 muxer cannot write to a non-seekable pipe
// without `empty_moov`, and an empty_moov/fragmented MOV is exactly what
// makes Adobe Premiere report "file cannot be opened" (no classic
// stco/stsz sample tables — samples live in moof/trun). A seekable file
// lets ffmpeg write a single contiguous moov with full sample tables and
// `+faststart` moves it to the front, producing a Premiere-native master.
const localMasterPath = growingPath
? null
: `/tmp/capture/${sessionId}.${hiresExt}`;
@ -660,44 +850,71 @@ class CaptureManager {
try { mkdirSync(dirname(localMasterPath), { recursive: true }); }
catch (err) { console.error('[capture] could not create temp master dir:', err.message); }
}
const hiresOutput = growingPath ? growingPath : localMasterPath;
// ffmpeg now writes a file (not stdout) in both modes → stdout is unused.
const hiresOutput = localMasterPath;
const hiresStdio = ['ignore', 'ignore', 'pipe'];
// For SDI we cannot open the DeckLink device a second time for a preview
// tee, so the live HLS preview is produced as a SECOND OUTPUT of the hires
// ffmpeg: one decklink read -> yadif -> split -> [ProRes/S3] + [H.264/HLS].
let sdiHlsDir = null;
let hiresArgs;
if (sourceType === 'sdi' && this._assetIdForHls) {
const fsMod = await import('node:fs');
sdiHlsDir = '/live/' + this._assetIdForHls;
try { fsMod.mkdirSync(sdiHlsDir, { recursive: true }); } catch (_) {}
hiresArgs = [
...inputArgs,
'-filter_complex', '[0:v]yadif=mode=1:deint=1,split=2[vhi][vlo]',
// Output 0 — ProRes master (S3 pipe or growing file)
'-map', '[vhi]', '-map', '0:a:0?',
...hiresCodecArgs,
hiresOutput,
// Output 1 — low-latency H.264 HLS preview for the UI monitor.
// GPU-encoded (h264_nvenc) when the GPU is attached to this sidecar,
// otherwise libx264 (issue #164). GOP is pinned to one IDR per HLS
// segment so segments start on keyframes (avoids black/flashing).
'-map', '[vlo]', '-map', '0:a:0?',
...buildHlsVideoArgs(videoCodec, framerate),
'-c:a', 'aac', '-b:a', '128k', '-ar', '44100',
'-f', 'hls', '-hls_time', '2', '-hls_list_size', '15',
'-hls_flags', 'delete_segments+append_list+omit_endlist',
'-hls_segment_filename', sdiHlsDir + '/seg-%05d.ts',
sdiHlsDir + '/index.m3u8',
];
console.log('[HLS] SDI preview as 2nd output -> ' + sdiHlsDir);
} else {
hiresArgs = [ ...inputArgs, ...sdiFilterArgs, ...hiresCodecArgs, hiresOutput ];
}
const hiresProcess = spawn('ffmpeg', hiresArgs, { stdio: hiresStdio });
let hiresProcess;
if (growingPath) {
// ── GROWING master: raw2bmx orchestrator ──────────────────────────
// One ffmpeg (single SDI read) → MPEG-2 422 elementary + PCM to FIFOs +
// the H.264 HLS preview; raw2bmx muxes the growing OP1a MXF from the FIFOs.
// Spawned via bash so the FIFO priming / EOF / stop-forwarding logic (see
// _buildGrowingOrchestrator) runs as one supervised unit. detached:true so
// it leads its own process group and we can clean-stop the whole pipeline.
const orchArgs = this._buildGrowingOrchestrator({
inputArgs,
videoBitrate,
// Recorder raster for the raw2bmx essence flag. recorders.js sets
// RECORDING_RESOLUTION (e.g. '1920x1080' / '1080i' / 'native'); when
// 'native'/absent, deriveGrowingRaster defaults to 1080i59.94.
resolution: process.env.RECORDING_RESOLUTION || null,
framerate,
audioChannels,
outPath: growingPath,
hlsDir: (sourceType === 'sdi') ? sdiHlsDir : null,
videoCodec,
});
console.log('[capture] growing master via raw2bmx; orchestrator script length=' + orchArgs[1].length);
hiresProcess = spawn('bash', orchArgs, { stdio: ['ignore', 'ignore', 'pipe'], detached: true });
} else {
// ── Finalized (non-growing) master: ffmpeg muxes the MOV directly ──
let hiresArgs;
if (sourceType === 'sdi' && this._assetIdForHls) {
hiresArgs = [
...inputArgs,
'-filter_complex', '[0:v]yadif=mode=1:deint=1,split=2[vhi][vlo]',
// Output 0 — ProRes/MOV master (local temp, uploaded to S3 on stop)
'-map', '[vhi]', '-map', '0:a:0?',
...hiresCodecArgs,
hiresOutput,
// Output 1 — low-latency H.264 HLS preview for the UI monitor.
// GPU-encoded (h264_nvenc) when the GPU is attached to this sidecar,
// otherwise libx264 (issue #164). GOP is pinned to one IDR per HLS
// segment so segments start on keyframes (avoids black/flashing).
'-map', '[vlo]', '-map', '0:a:0?',
...buildHlsVideoArgs(videoCodec, framerate),
'-c:a', 'aac', '-b:a', '128k', '-ar', '44100',
'-f', 'hls', '-hls_time', '2', '-hls_list_size', '15',
'-hls_flags', 'delete_segments+append_list+omit_endlist',
'-hls_segment_filename', sdiHlsDir + '/seg-%05d.ts',
sdiHlsDir + '/index.m3u8',
];
console.log('[HLS] SDI preview as 2nd output -> ' + sdiHlsDir);
} else {
hiresArgs = [ ...inputArgs, ...sdiFilterArgs, ...hiresCodecArgs, hiresOutput ];
}
hiresProcess = spawn('ffmpeg', hiresArgs, { stdio: hiresStdio });
}
// Growing-files: nothing to upload here (promotion worker handles S3).
// Non-growing: the master is uploaded from the finalized local file in
@ -792,17 +1009,34 @@ class CaptureManager {
const { processes, currentSession } = this.state;
// Send SIGINT and WAIT for ffmpeg to exit. This is what flushes the MOV
// trailer (writes the moov atom with the full sample tables). If we uploaded
// before ffmpeg finalized, the object would have no moov → "moov atom not
// found" / "file cannot be opened" in Premiere.
const isGrowing = !!currentSession.growingPath;
// Send SIGINT and WAIT for the master writer to exit cleanly.
// - Non-growing: SIGINT flushes ffmpeg's MOV trailer (the moov atom with
// full sample tables). Uploading before finalize → "moov atom not found".
// - Growing: `processes.hires` is the bash ORCHESTRATOR (detached group
// leader). SIGINT hits its trap, which forwards SIGINT to ffmpeg; ffmpeg
// stops → raw2bmx gets EOF → raw2bmx writes the OP1a FOOTER and exits;
// only then does the orchestrator exit. Awaiting it guarantees the
// finalized, valid MXF is on the share before the promotion worker
// uploads it. raw2bmx footer finalize of a long recording can take longer
// than a MOV trailer flush, so the growing safety-net is more generous.
const finalizeTimeoutMs = isGrowing ? 60000 : 15000;
const waitExit = (proc) => new Promise((resolve) => {
if (!proc || proc.exitCode !== null || proc.signalCode !== null) return resolve();
let done = false;
const finish = () => { if (!done) { done = true; resolve(); } };
proc.once('exit', finish);
// Safety net: don't hang stop() forever if ffmpeg refuses to exit.
setTimeout(() => { try { proc.kill('SIGKILL'); } catch (_) {} finish(); }, 15000);
// Safety net: don't hang stop() forever if the writer refuses to exit.
setTimeout(() => {
try {
// Detached orchestrator → kill the whole process group (ffmpeg +
// raw2bmx + bash); otherwise just the process.
if (isGrowing && proc.pid) { try { process.kill(-proc.pid, 'SIGKILL'); } catch (_) {} }
proc.kill('SIGKILL');
} catch (_) {}
finish();
}, finalizeTimeoutMs);
});
if (processes.hires) processes.hires.kill('SIGINT');