VHD_SlotExtractAudio on the JOINED slot returned -91dB silence when only
pAudioChannels[0] was declared. On this card/SDK declaring BOTH channel[0] and
channel[1] of the stereo pair makes the SDK land real PCM. Verified -13.3dB
audible against the live SDI source (was -91dB).
Re-engineer the framecache so each video frame carries its own SDI-embedded
audio through ONE transport, eliminating the "audio ahead of video" offset at
the root: there is no longer a second independent audio buffer/FIFO that can
race ahead of video.
slot.h (FC_VERSION 1 -> 2):
- Per ring entry data region is now [video frame_size][audio FC_MAX_AUDIO_BYTES].
- fc_frame_t: the former _pad u32 is REUSED as audio_size (header still 24B).
- Header gains audio_max_bytes / audio_rate / audio_channels (self-describing).
- New fc_entry_stride()/fc_frame_audio() helpers; shm size includes audio.
- Readers/writer check version == FC_VERSION and FAIL SAFE on mismatch so an
old reader against a new writer (or vice-versa) refuses rather than misparses.
slot.c: populate audio header fields; add fc_slot_write_av(); version gate in open.
fc_client.[ch]: fc_frame_ref_t gains audio/audio_size; copy buffer holds
video+audio; both copied from the SAME entry in one read -> frame-locked.
fc_pipe.c: now <slot_id> <wait_ms> <audio_fifo_path>; per ring entry writes
video -> stdout AND that frame's audio -> the audio FIFO IN LOCKSTEP from one
cursor read (no second buffer to drift). Auto-reattaches FIFO on EPIPE.
deltacast-bridge:
- SILENT-AUDIO FIX: audio_extract_init now configures ONLY pAudioChannels[0]
of group 0 as one stereo pair (Mode=STEREO, BufferFormat=AF_16, pData=buf),
leaving pAudioChannels[1] ZEROED, exactly like Deltacast's own FFmpeg fork
(libavdevice/videomaster_common.c init_audio_info). The prior JOINED code
ALSO set channel[1].Mode/BufferFormat=STEREO/AF_16, declaring a second pair
the signal does not carry -> VHD_SlotExtractAudio returned zero samples ->
-91 dB silent audio. DataSize is (re)set to capacity before each extract.
- VHD_SDI_SP_INTERFACE now set from the channel-detected interface
(VHD_SDI_CP_INTERFACE) before StartStream, per the same fork — required for
embedded-audio extraction on JOINED SDI streams.
- fc_writer.[ch]: add fc_writer_write_av(); struct/stride bumped to v2.
- video_thread (framecache path) extracts each frame's audio from the SAME
locked JOINED slot and writes BOTH via fc_writer_write_av. Silence fallback
at the source: a frame with no embedded audio gets one frame-interval of
silence so the audio timeline length always equals the video timeline length.
- The separate audio FIFO + audio_thread + apcm ring are retained ONLY for the
legacy (-DLEGACY_FIFO / framecache-unreachable) fallback; on the primary
framecache path the bridge no longer owns the audio FIFO.
capture-manager.js: deltacast/sdi framecache branch now CREATES the audio FIFO
and passes its path to fc_pipe (argv[3]); ffmpeg keeps two raw inputs
(rawvideo pipe:0 + s16le 48k input 1) but both are now fed frame-locked from
the same ring entry. Stale-audio pre-flush retained as harmless safety.
All changes versioned; mismatched binaries refuse to attach (fail safe).
Re-architect the Deltacast bridge from two independently-buffered VHD
streams (DISJOINED_VIDEO + a separate DISJOINED_ANC audio stream) to a
single VHD_SDI_STPROC_JOINED RX stream per port. Each locked slot now
carries both the video frame and that frame's SDI-embedded audio, so
audio is extracted (VHD_SlotExtractAudio) from the SAME slot the video
came from — eliminating the constant "audio ahead of video" offset at
its root instead of papering over it with --audio-delay-ms.
This mirrors Deltacast's own FFmpeg fork (libavdevice/videomaster_common.c):
open JOINED, then per frame LockSlot -> GetSlotBuffer(VIDEO) ->
SlotExtractAudio(same slot) -> Unlock.
Changes (main.c):
- main(): RX stream opened with VHD_SDI_STPROC_JOINED (was
VHD_SDI_STPROC_DISJOINED_VIDEO). SDI interface still propagated so
audio extraction yields real samples.
- video_thread(): becomes the single per-frame consumer. After locking
each JOINED slot it extracts the embedded audio (audio_extract_slot)
and pushes de-interleaved s16le stereo PCM into a new lock-free SPSC
ring (ApcmRing), then writes the video frame to the framecache ring
(or legacy video FIFO). Audio is extracted on BOTH the framecache and
legacy-FIFO paths, before the video packing check so it is never
dropped on a rare packing mismatch.
- audio_thread(): no longer opens any VHD stream. It is now purely the
audio-FIFO sink: drains the ApcmRing into the named audio FIFO
(ffmpeg input 1), flushes the ring to the live edge on reader attach,
survives ffmpeg restarts (EPIPE -> reopen), and emits wall-clock-paced
silence when the ring is empty (preserves the silence-fallback when the
signal carries no embedded audio).
- Audio stays 48 kHz stereo s16le to match ffmpeg's expectations.
- --audio-delay-ms / FC_AUDIO_DELAY_MS retained but now unnecessary
(should be left at 0); kept for compatibility / emergency tuning.
Compiles clean (only pre-existing %lu/ULONG format warnings in main()).
Not installed, not deployed — branch only.
The deltacast bridge captures audio (DISJOINED_ANC VHD stream -> FIFO) and
video (VHD stream -> framecache ring -> fc_pipe -> ffmpeg) on separate paths
with independent buffering. The video path is buffered deeper, so audio reaches
the muxer AHEAD of its matching video frame (user-confirmed: audio ahead of
video on lip-sync). --audio-delay-ms / FC_AUDIO_DELAY_MS prepends N ms of real
PCM silence to the audio stream once per reader-attach, shifting the audio
timeline N ms later to re-align. One value, all 8 ports, drift-free (ffmpeg
derives audio PTS from sample count). Default 0 = unchanged (non-destructive).
Operator tunes once against a lip-sync reference. Also fixes promotion worker
to reset orphaned processing assets on failure (was stuck forever).
The ~2.5s of leading silence at record start was the VHD audio slot QUEUE: while
the recorder is idle (no FIFO reader), the bridge blocks on open(O_WRONLY) but the
board keeps buffering audio slots. When the record ffmpeg attaches, the bridge
streamed that stale backlog first — heard as leading silence and pushing audio
out of alignment with the live video.
On each reader attach, drain slots that lock FAST (already-queued backlog) and
stop at the first lock that takes ~a frame period (= waiting on a live slot), so
the reader is handed the live edge, A/V aligned.
The 16ch interleave in the deltacast bridge produced audio at HALF the correct
sample rate (measured 24224 vs 48000 samples/s/ch), which broke A/V sync and
pitch. Per the working baseline (audio was clean before the channel selector),
revert the bridge audio thread to the original single-group 2ch extraction and
the capture-manager audio input to -ac 2 + wallclock + aresample.
KEPT the good fixes: long-GOP HEVC for non-growing (NVENC realtime, no frame
drops) and GPU-only codec list. 16ch/channel-select is shelved for a separate,
properly-validated change.
Taking the MAX sample count across the 4 audio groups could emit more audio
frames per slot than group 0 (the SDI-clock reference), drifting the audio
stream slightly longer than video — heard as a ~1% pitch-up. Group 0 paces the
timeline exactly as the original 2ch path did; shorter groups are silence-padded
to its length, never extending it.
- fc_writer.h/fc_writer.c: new framecache slot writer module
- Registers slot via POST /slots to framecache HTTP API on signal lock
- Opens shm file returned by API (O_RDWR + mmap MAP_SHARED)
- fc_writer_write(): atomic write_cursor advance + sem_post per frame
- fc_writer_close(): DELETE /slots/:id + munmap + sem_close
- HTTP calls via raw POSIX sockets (no libcurl dependency)
- Parses host:port from FC_URL env var or --fc-url arg
- main.c changes:
- PortState gains slot_id, fc_url, fc_writer fields
- --fc-url CLI arg + FC_URL env var (default http://localhost:7435)
- On signal lock: fc_writer_open() before thread launch;
falls back to FIFO if framecache unreachable (fc_writer == NULL)
- video_thread: shm path primary (fc_writer != NULL),
FIFO path fallback (fc_writer == NULL or LEGACY_FIFO=1)
- Format JSON now includes slot_id field for node-agent consumption
- Cleanup: fc_writer_close() before VHD_CloseBoardHandle
- CMakeLists.txt:
- Add fc_writer.c to build
- Link rt (shm_open, sem_open)
- LEGACY_FIFO option (OFF by default) for nodes without framecache
Audio thread unchanged — audio stays in FIFO (shm audio is roadmap).
Relying on the SDK default for VHD_CORE_SP_BUFFER_PACKING caused the board
to deliver raw slot buffers whose byte size did not match the width*height*2
contract declared to ffmpeg (pix_fmt=uyvy422). ffmpeg consumed frames at the
wrong stride, causing every frame to shift progressively, rolling and
shearing the picture ("bounces and bends").
Fix: explicitly set VHD_BUFPACK_VIDEO_YUV422_8 on the video stream before
StartStream so the board always delivers tightly-packed 8-bit UYVY.
Safety net: video thread now asserts sz==width*height*2 and skips+warns on
any mismatch so a future packing divergence is immediately visible in logs
rather than silently corrupting video.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a capture sidecar stopped/restarted, the bridge video thread got EPIPE
on the FIFO write, set g_port_stop[port]=1, and the port went dead — requiring
a full bridge restart to recover. Subsequent record attempts on that port would
hang in 'connecting' forever.
Fix: mirror the audio thread pattern — on EPIPE, close the FIFO and loop back
to open() blocking for the next reader. Hardware lock errors (SDK failures)
still stop the port via g_port_stop as before. Only reader-disconnect (EPIPE)
now recovers gracefully.
This was the cause of port 6 (Ghost) failure in the burn test.
🤖 Generated with Claude Code
ROOT CAUSE of 'connecting' hangs and intermittent port failures:
The DELTA-12G-e-h 8c is a bidirectional card. Without calling
VHD_SetBiDirCfg(board_index, VHD_BIDIR_80) before streaming, the
board remains in its default bi-dir config (likely 4RX/4TX) — so
RX stream opens fail with VHDERR_RESOURCEUNAVAILABLE on channels
configured as TX, causing random 'connecting' hangs per the SDK docs.
Per SDK Tools.cpp SetNbChannels() pattern:
1. Open temporary board handle
2. Check IS_BIDIR + channel counts
3. Call VHD_SetBiDirCfg(board_index, VHD_BIDIR_80) for 8ch bidir
4. Close temp handle, then open real board handle for streaming
Also add VHD_SetChannelProperty(VHD_CHANNEL_MODE_SDI) for ASI-type
channels per Sample_RX.cpp — required for 12G-ASI/3G-ASI channel
types to correctly detect incoming video standard.
🤖 Generated with Claude Code
Root cause A (main.c): audio_thread set the global g_stop flag on EPIPE
(ffmpeg reader died). This killed ALL port threads across the entire bridge
process. Bridge process then exited with all 8 ports gone.
Root cause B (node-agent/index.js): startDeltacastBridge() skipped respawn
when FIFOs existed in /dev/shm/deltacast, even if the bridge process was dead.
Next ffmpeg opened the audio FIFO read-end and blocked forever (no writer) →
no audio (or video) for any new recording.
Fix A (main.c):
- Add per-port atomic g_port_stop[MAX_PORTS] flags.
- Audio thread: on EPIPE, close the FIFO fd and loop back to reopen it.
The VHD ANC stream stays open across reconnects. Other ports unaffected.
- Video thread: on EPIPE or stream error, set only g_port_stop[port], not
the global g_stop. Other ports keep running.
- MAX_PORTS #define moved before globals so g_port_stop[MAX_PORTS] compiles.
Fix B (node-agent/index.js):
- Add _dcBridgeProcessAlive() — scans /proc/<pid>/cmdline for deltacast-bridge.
- startDeltacastBridge(): if FIFOs exist but no live bridge process is found,
spawn a fresh bridge instead of silently returning. Detects bridges started
externally (e.g. sudo on the host before node-agent started).
Requires: bridge rebuild + restart on zampp3. No capture image rebuild needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The old architecture spawned one deltacast-capture per recorder port; each
called VHD_OpenBoardHandle, triggering a BufMngr.c:781 OOB fault in the
delta_x300 kernel driver whenever two opens raced.
Fix: a single deltacast-bridge daemon opens the board once, opens RX
streams for all requested ports concurrently, and writes each port's
video/audio to named FIFOs (/dev/shm/deltacast/video-<N>.fifo,
/dev/shm/deltacast/audio-<N>.fifo). Capture sidecars read from those
FIFOs directly — no board handle, no race, no flock.
Changes:
services/capture/deltacast-bridge/main.c
- Complete rewrite: --ports csv arg, board opened once, one
video+audio thread pair per port, FIFO paths per port, format
JSON emitted per port on signal lock, SIGTERM clean shutdown.
- flock/serialize logic removed (no longer needed).
- --port single-port compat alias retained.
services/capture/deltacast-bridge/CMakeLists.txt
- Rename target deltacast-capture -> deltacast-bridge.
- POST_BUILD symlink deltacast-capture -> deltacast-bridge for compat.
services/capture/src/capture-manager.js
- deltacast _buildInputArgs: remove bridge spawn; wait up to 30s
for FIFOs to exist (bridge may be starting); return rawvideo +
s16le FIFO inputArgs. bridgeProcess=null.
- audioMap: keyed on sourceType instead of bridgeProcess (both
inputs are always present for deltacast).
- Remove readFirstStderrLine helper (no longer needed).
- Remove bridgeProcess.stdout.pipe / processes.bridge stop signal.
services/node-agent/index.js
- Add import spawn for bridge daemon management.
- Add startDeltacastBridge / stopDeltacastBridge: host-process
lifecycle for the shared bridge, ref-counted by sidecar count.
- handleSidecarStart: on deltacast, increment counter + start bridge;
decrement on container create/start failure.
- handleSidecarStop: decrement counter; stop bridge when last sidecar.
- _containerSourceType map tracks containerId->sourceType for stop.
- Old acquireDcLock mutex retained but no longer called.
The signal timeout deadline was set at process start before waiting for
the flock. Bridges queued behind earlier ports waited minutes for the
lock, then found their 30s signal deadline had already expired before
they even opened the board, causing false "no signal" failures on ports
that have live signal.
Fix: move clock_gettime deadline initialisation to AFTER flock acquired
and board opened, so the full sig_timeout is always available for signal
detection regardless of queue wait time.
Concurrent VHD_OpenBoardHandle calls from multiple capture sidecars
trigger delta_x300 BufMngr.c:781 array-index-out-of-bounds, wedging all
RX channels until the module is reloaded. The node-agent stagger only
delays container start — the bridge binary starts ~2s later and can still
race. This fix acquires an exclusive flock on /dev/shm/deltacast/bridge.lock
before VHD_OpenBoardHandle and holds it until signal lock succeeds (then
adds a 4s settle before releasing so the board's buffer queues stabilize).
Lock is released on signal failure too so the next bridge is never
permanently blocked. All 8 channels can now start safely by serializing
through the same lock file mounted into every sidecar.
ffmpeg opens all inputs before processing; input 1 is the audio FIFO. The
bridge previously opened the FIFO writer only after VHD_OpenStreamHandle +
VHD_StartStream succeeded, returning early on failure / no embedded audio and
never opening the FIFO -> ffmpeg blocked forever on input 1 -> 0 fps and an
empty HLS preview. Now the FIFO writer is opened unconditionally and first,
and the audio thread feeds a continuous, wall-clock-paced s16le stereo stream
(real samples when available, otherwise silence). SIGPIPE is ignored so a
dying ffmpeg returns EPIPE instead of killing the bridge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
