# Unified Framecache — Implementation Plan ## Context Replace the current named-FIFO-per-source architecture with a shared-memory ring buffer (framecache) that fans raw video frames from any ingest source to unlimited concurrent consumers with zero-copy reads. **Approved design:** docs/design/framecache/DESIGN.md **Branch:** feat/unified-framecache **Roadmap (out of scope here):** RDMA cross-node, AJA, growing-file-while-recording browser playback --- ## Migration Strategy Ship in 5 phases. Each phase is independently deployable and leaves the system in a working state. Existing recording workflows are unaffected until Phase 5 cuts over. --- ## Phase 1 — Framecache Container (foundation) **Goal:** Running framecache service with slot registry. No ingest writers yet. ### 1.1 — Create `services/framecache/` directory structure ``` services/framecache/ src/ framecache.c # main — slot manager + HTTP API slot.c / slot.h # shm ring buffer lifecycle registry.c # /dev/shm/framecache/registry.json writer http.c # lightweight HTTP server (libmicrohttpd) client/ fc_client.c / fc_client.h # consumer library fc_client_node/ binding.cc # Node.js N-API addon binding.gyp Dockerfile CMakeLists.txt ``` ### 1.2 — Shared memory layout (slot.h) Each slot lives at `/dev/shm/framecache/`: ```c #define FC_MAGIC 0x46524D43 // "FRMC" #define FC_RING_DEPTH 120 // ~2s at 59.94fps #define FC_HEADER_SIZE 4096 // 4KB header block typedef struct { uint32_t magic; uint32_t version; // = 1 uint32_t width; uint32_t height; uint32_t fps_num; uint32_t fps_den; uint32_t pixel_format; // FC_PIX_UYVY422 = 0 uint32_t frame_size; // width * height * 2 uint32_t ring_depth; // = FC_RING_DEPTH _Atomic uint64_t write_cursor; // monotonically increasing frame index _Atomic uint64_t dropped_frames; uint8_t _pad[FC_HEADER_SIZE - 48]; } fc_header_t; typedef struct { uint64_t pts_us; uint64_t wall_us; uint32_t size; uint8_t data[]; // frame_size bytes } fc_frame_t; ``` Semaphore: `sem_open("/framecache--write", ...)` — posted by writer on each new frame, consumers `sem_timedwait` on it. ### 1.3 — HTTP API (port 7435) ``` POST /slots body: {slot_id, width, height, fps_num, fps_den, source_type} creates shm region, writes registry entry 201 {slot_id, shm_path, sem_name} GET /slots 200 [{slot_id, width, height, fps_num, fps_den, source_type, write_cursor, dropped_frames, current_fps}] GET /slots/:id 200 slot detail DELETE /slots/:id destroys shm + semaphore, removes registry entry, 204 GET /health 200 {status: "ok"} ``` ### 1.4 — Registry file Written to `/dev/shm/framecache/registry.json` on every slot create/delete. ### 1.5 — Dockerfile ```dockerfile FROM debian:bookworm RUN apt-get update && apt-get install -y \ build-essential cmake libmicrohttpd-dev \ && rm -rf /var/lib/apt/lists/* COPY . /src RUN cmake -S /src -B /build -DCMAKE_BUILD_TYPE=Release \ && cmake --build /build -j$(nproc) EXPOSE 7435 CMD ["/build/framecache"] ``` ### 1.6 — docker-compose.worker.yml addition ```yaml framecache: build: ./services/framecache ipc: host shm_size: '60gb' environment: FC_SHM_SIZE: ${FC_SHM_SIZE:-64424509440} FC_PORT: 7435 ports: - "7435:7435" volumes: - /dev/shm:/dev/shm restart: unless-stopped ``` ### 1.7 — Consumer library (fc_client.c) ```c fc_slot_t *fc_open(const char *slot_id); int fc_read_frame(fc_slot_t *slot, fc_frame_t **out, uint64_t timeout_ms); void fc_close(fc_slot_t *slot); ``` **Commit:** `feat(framecache): phase 1 — framecache container + consumer library` --- ## Phase 2 — Deltacast Bridge writes to framecache **Goal:** deltacast-bridge writes frames to framecache shm instead of named FIFOs. Legacy FIFO path kept as compile-time fallback (`-DLEGACY_FIFO=ON`) until Phase 5. On signal lock: 1. POST /slots to framecache HTTP API 2. shm_open + mmap the slot 3. Video thread writes frame into ring, advances write_cursor atomically, sem_post 4. Audio: keeps writing to audio FIFO (unchanged) 5. On shutdown: DELETE /slots/:id **Commit:** `feat(framecache): phase 2 — deltacast-bridge writes to shm` --- ## Phase 3 — Blackmagic DeckLink Bridge **Goal:** New decklink-bridge C program mirrors deltacast-bridge, replaces ffmpeg -f decklink direct path. - Uses IDeckLinkIterator to enumerate devices - VideoInputFrameArrived callback calls fc_write_frame - Registers slot on signal lock, deregisters on shutdown - Audio stays in FIFO (same as deltacast) **Commit:** `feat(framecache): phase 3 — decklink-bridge writes to shm` --- ## Phase 4 — capture-manager reads from framecache **Goal:** Enables simultaneous growing + proxy + HLS from one SDI input. - Node.js N-API addon wrapping fc_open/fc_read_frame/fc_close - capture-manager opens THREE fc_client handles per slot (own cursor each): 1. Growing/master ffmpeg feed 2. Proxy ffmpeg feed 3. HLS preview ffmpeg feed - Each gets a separate rawvideo pipe to ffmpeg - Growing MXF workflow (raw2bmx orchestrator) completely unchanged **Commit:** `feat(framecache): phase 4 — capture-manager reads from framecache` --- ## Phase 5 — Network ingest (RTMP/SRT) into framecache **Goal:** RTMP and SRT sources decoded to raw UYVY422, written into framecache slots. - net_ingest process per source: ffmpeg decodes to rawvideo, writes to slot - capture-manager waits for slot, same fc_client consumer pattern - Remove legacy FIFO code once all paths go through framecache **Commit:** `feat(framecache): phase 5 — network ingest via framecache` --- ## Hardware / Deployment | Node | RAM | /dev/shm | FC_SHM_SIZE | |------|-----|----------|-------------| | Baratheon | 251GB | 126GB | 60GB | | zampp1 | 93GB | 47GB | 40GB | | zampp2 | 18GB (upgrade) | 9.4GB | 8GB | Ring buffer per 1080p59.94 source: ~494MB (120 frames × 4.1MB) All recorder sidecars require `ipc: host`. --- ## Roadmap (not in this branch) - Audio in framecache shm - RDMA cross-node slot replication - AJA hardware support - Growing-file-while-recording browser HLS playback - Mastercontrol/playout consumer