dragonflight/docs/design/framecache/PLAN.md

Unified Framecache — Implementation Plan

Context

Replace the current named-FIFO-per-source architecture with a shared-memory ring buffer (framecache) that fans raw video frames from any ingest source to unlimited concurrent consumers with zero-copy reads.

Approved design: docs/design/framecache/DESIGN.md
Branch: feat/unified-framecache
Roadmap (out of scope here): RDMA cross-node, AJA, growing-file-while-recording browser playback

---

Migration Strategy

Ship in 5 phases. Each phase is independently deployable and leaves the system in a working state. Existing recording workflows are unaffected until Phase 5 cuts over.

---

Phase 1 — Framecache Container (foundation)

Goal: Running framecache service with slot registry. No ingest writers yet.

1.1 — Create services/framecache/ directory structure

services/framecache/
  src/
    framecache.c        # main — slot manager + HTTP API
    slot.c / slot.h     # shm ring buffer lifecycle
    registry.c          # /dev/shm/framecache/registry.json writer
    http.c              # lightweight HTTP server (libmicrohttpd)
  client/
    fc_client.c / fc_client.h   # consumer library
    fc_client_node/
      binding.cc        # Node.js N-API addon
      binding.gyp
  Dockerfile
  CMakeLists.txt

1.2 — Shared memory layout (slot.h)

Each slot lives at /dev/shm/framecache/<slot_id>:

#define FC_MAGIC        0x46524D43   // "FRMC"
#define FC_RING_DEPTH   120          // ~2s at 59.94fps
#define FC_HEADER_SIZE  4096         // 4KB header block

typedef struct {
  uint32_t magic;
  uint32_t version;          // = 1
  uint32_t width;
  uint32_t height;
  uint32_t fps_num;
  uint32_t fps_den;
  uint32_t pixel_format;     // FC_PIX_UYVY422 = 0
  uint32_t frame_size;       // width * height * 2
  uint32_t ring_depth;       // = FC_RING_DEPTH
  _Atomic uint64_t write_cursor;    // monotonically increasing frame index
  _Atomic uint64_t dropped_frames;
  uint8_t  _pad[FC_HEADER_SIZE - 48];
} fc_header_t;

typedef struct {
  uint64_t pts_us;
  uint64_t wall_us;
  uint32_t size;
  uint8_t  data[];   // frame_size bytes
} fc_frame_t;

Semaphore: sem_open("/framecache-<slot_id>-write", ...) — posted by writer on each new frame, consumers sem_timedwait on it.

1.3 — HTTP API (port 7435)

POST   /slots          body: {slot_id, width, height, fps_num, fps_den, source_type}
                       creates shm region, writes registry entry
                       201 {slot_id, shm_path, sem_name}

GET    /slots          200 [{slot_id, width, height, fps_num, fps_den,
                              source_type, write_cursor, dropped_frames,
                              current_fps}]

GET    /slots/:id      200 slot detail
DELETE /slots/:id      destroys shm + semaphore, removes registry entry, 204
GET    /health         200 {status: "ok"}

1.4 — Registry file

Written to /dev/shm/framecache/registry.json on every slot create/delete.

1.5 — Dockerfile

FROM debian:bookworm
RUN apt-get update && apt-get install -y \
    build-essential cmake libmicrohttpd-dev \
  && rm -rf /var/lib/apt/lists/*
COPY . /src
RUN cmake -S /src -B /build -DCMAKE_BUILD_TYPE=Release \
  && cmake --build /build -j$(nproc)
EXPOSE 7435
CMD ["/build/framecache"]

1.6 — docker-compose.worker.yml addition

framecache:
  build: ./services/framecache
  ipc: host
  shm_size: '60gb'
  environment:
    FC_SHM_SIZE: ${FC_SHM_SIZE:-64424509440}
    FC_PORT: 7435
  ports:
    - "7435:7435"
  volumes:
    - /dev/shm:/dev/shm
  restart: unless-stopped

1.7 — Consumer library (fc_client.c)

fc_slot_t *fc_open(const char *slot_id);
int fc_read_frame(fc_slot_t *slot, fc_frame_t **out, uint64_t timeout_ms);
void fc_close(fc_slot_t *slot);

Commit: feat(framecache): phase 1 — framecache container + consumer library

---

Phase 2 — Deltacast Bridge writes to framecache

Goal: deltacast-bridge writes frames to framecache shm instead of named FIFOs. Legacy FIFO path kept as compile-time fallback (-DLEGACY_FIFO=ON) until Phase 5.

On signal lock:

1. POST /slots to framecache HTTP API
2. shm_open + mmap the slot
3. Video thread writes frame into ring, advances write_cursor atomically, sem_post
4. Audio: keeps writing to audio FIFO (unchanged)
5. On shutdown: DELETE /slots/:id

Commit: feat(framecache): phase 2 — deltacast-bridge writes to shm

---

Phase 3 — Blackmagic DeckLink Bridge

Goal: New decklink-bridge C program mirrors deltacast-bridge, replaces ffmpeg -f decklink direct path.

• Uses IDeckLinkIterator to enumerate devices
• VideoInputFrameArrived callback calls fc_write_frame
• Registers slot on signal lock, deregisters on shutdown
• Audio stays in FIFO (same as deltacast)

Commit: feat(framecache): phase 3 — decklink-bridge writes to shm

---

Phase 4 — capture-manager reads from framecache

Goal: Enables simultaneous growing + proxy + HLS from one SDI input.

• Node.js N-API addon wrapping fc_open/fc_read_frame/fc_close
• capture-manager opens THREE fc_client handles per slot (own cursor each):
  1. Growing/master ffmpeg feed
  2. Proxy ffmpeg feed
  3. HLS preview ffmpeg feed
• Each gets a separate rawvideo pipe to ffmpeg
• Growing MXF workflow (raw2bmx orchestrator) completely unchanged

Commit: feat(framecache): phase 4 — capture-manager reads from framecache

---

Phase 5 — Network ingest (RTMP/SRT) into framecache

Goal: RTMP and SRT sources decoded to raw UYVY422, written into framecache slots.

• net_ingest process per source: ffmpeg decodes to rawvideo, writes to slot
• capture-manager waits for slot, same fc_client consumer pattern
• Remove legacy FIFO code once all paths go through framecache

Commit: feat(framecache): phase 5 — network ingest via framecache

---

Hardware / Deployment

Node	RAM	/dev/shm	FC_SHM_SIZE
Baratheon	251GB	126GB	60GB
zampp1	93GB	47GB	40GB
zampp2	18GB (upgrade)	9.4GB	8GB

Ring buffer per 1080p59.94 source: ~494MB (120 frames × 4.1MB)

All recorder sidecars require ipc: host.

---

Roadmap (not in this branch)

• Audio in framecache shm
• RDMA cross-node slot replication
• AJA hardware support
• Growing-file-while-recording browser HLS playback
• Mastercontrol/playout consumer