dragonflight/docs/design/framecache/PLAN.md

# Unified Framecache — Implementation Plan

## Context

Replace the current named-FIFO-per-source architecture with a shared-memory
ring buffer (framecache) that fans raw video frames from any ingest source to
unlimited concurrent consumers with zero-copy reads.

**Approved design:** docs/design/framecache/DESIGN.md  
**Branch:** feat/unified-framecache  
**Roadmap (out of scope here):** RDMA cross-node, AJA, growing-file-while-recording browser playback

---

## Migration Strategy

Ship in 5 phases. Each phase is independently deployable and leaves the system
in a working state. Existing recording workflows are unaffected until Phase 5
cuts over.

---

## Phase 1 — Framecache Container (foundation)

**Goal:** Running framecache service with slot registry. No ingest writers yet.

### 1.1 — Create `services/framecache/` directory structure

```
services/framecache/
  src/
    framecache.c        # main — slot manager + HTTP API
    slot.c / slot.h     # shm ring buffer lifecycle
    registry.c          # /dev/shm/framecache/registry.json writer
    http.c              # lightweight HTTP server (libmicrohttpd)
  client/
    fc_client.c / fc_client.h   # consumer library
    fc_client_node/
      binding.cc        # Node.js N-API addon
      binding.gyp
  Dockerfile
  CMakeLists.txt
```

### 1.2 — Shared memory layout (slot.h)

Each slot lives at `/dev/shm/framecache/<slot_id>`:

```c
#define FC_MAGIC        0x46524D43   // "FRMC"
#define FC_RING_DEPTH   120          // ~2s at 59.94fps
#define FC_HEADER_SIZE  4096         // 4KB header block

typedef struct {
  uint32_t magic;
  uint32_t version;          // = 1
  uint32_t width;
  uint32_t height;
  uint32_t fps_num;
  uint32_t fps_den;
  uint32_t pixel_format;     // FC_PIX_UYVY422 = 0
  uint32_t frame_size;       // width * height * 2
  uint32_t ring_depth;       // = FC_RING_DEPTH
  _Atomic uint64_t write_cursor;    // monotonically increasing frame index
  _Atomic uint64_t dropped_frames;
  uint8_t  _pad[FC_HEADER_SIZE - 48];
} fc_header_t;

typedef struct {
  uint64_t pts_us;
  uint64_t wall_us;
  uint32_t size;
  uint8_t  data[];   // frame_size bytes
} fc_frame_t;
```

Semaphore: `sem_open("/framecache-<slot_id>-write", ...)` — posted by writer
on each new frame, consumers `sem_timedwait` on it.

### 1.3 — HTTP API (port 7435)

```
POST   /slots          body: {slot_id, width, height, fps_num, fps_den, source_type}
                       creates shm region, writes registry entry
                       201 {slot_id, shm_path, sem_name}

GET    /slots          200 [{slot_id, width, height, fps_num, fps_den,
                              source_type, write_cursor, dropped_frames,
                              current_fps}]

GET    /slots/:id      200 slot detail
DELETE /slots/:id      destroys shm + semaphore, removes registry entry, 204
GET    /health         200 {status: "ok"}
```

### 1.4 — Registry file

Written to `/dev/shm/framecache/registry.json` on every slot create/delete.

### 1.5 — Dockerfile

```dockerfile
FROM debian:bookworm
RUN apt-get update && apt-get install -y \
    build-essential cmake libmicrohttpd-dev \
  && rm -rf /var/lib/apt/lists/*
COPY . /src
RUN cmake -S /src -B /build -DCMAKE_BUILD_TYPE=Release \
  && cmake --build /build -j$(nproc)
EXPOSE 7435
CMD ["/build/framecache"]
```

### 1.6 — docker-compose.worker.yml addition

```yaml
framecache:
  build: ./services/framecache
  ipc: host
  shm_size: '60gb'
  environment:
    FC_SHM_SIZE: ${FC_SHM_SIZE:-64424509440}
    FC_PORT: 7435
  ports:
    - "7435:7435"
  volumes:
    - /dev/shm:/dev/shm
  restart: unless-stopped
```

### 1.7 — Consumer library (fc_client.c)

```c
fc_slot_t *fc_open(const char *slot_id);
int fc_read_frame(fc_slot_t *slot, fc_frame_t **out, uint64_t timeout_ms);
void fc_close(fc_slot_t *slot);
```

**Commit:** `feat(framecache): phase 1 — framecache container + consumer library`

---

## Phase 2 — Deltacast Bridge writes to framecache

**Goal:** deltacast-bridge writes frames to framecache shm instead of named FIFOs.
Legacy FIFO path kept as compile-time fallback (`-DLEGACY_FIFO=ON`) until Phase 5.

On signal lock:
1. POST /slots to framecache HTTP API
2. shm_open + mmap the slot
3. Video thread writes frame into ring, advances write_cursor atomically, sem_post
4. Audio: keeps writing to audio FIFO (unchanged)
5. On shutdown: DELETE /slots/:id

**Commit:** `feat(framecache): phase 2 — deltacast-bridge writes to shm`

---

## Phase 3 — Blackmagic DeckLink Bridge

**Goal:** New decklink-bridge C program mirrors deltacast-bridge, replaces
ffmpeg -f decklink direct path.

- Uses IDeckLinkIterator to enumerate devices
- VideoInputFrameArrived callback calls fc_write_frame
- Registers slot on signal lock, deregisters on shutdown
- Audio stays in FIFO (same as deltacast)

**Commit:** `feat(framecache): phase 3 — decklink-bridge writes to shm`

---

## Phase 4 — capture-manager reads from framecache

**Goal:** Enables simultaneous growing + proxy + HLS from one SDI input.

- Node.js N-API addon wrapping fc_open/fc_read_frame/fc_close
- capture-manager opens THREE fc_client handles per slot (own cursor each):
  1. Growing/master ffmpeg feed
  2. Proxy ffmpeg feed
  3. HLS preview ffmpeg feed
- Each gets a separate rawvideo pipe to ffmpeg
- Growing MXF workflow (raw2bmx orchestrator) completely unchanged

**Commit:** `feat(framecache): phase 4 — capture-manager reads from framecache`

---

## Phase 5 — Network ingest (RTMP/SRT) into framecache

**Goal:** RTMP and SRT sources decoded to raw UYVY422, written into framecache slots.

- net_ingest process per source: ffmpeg decodes to rawvideo, writes to slot
- capture-manager waits for slot, same fc_client consumer pattern
- Remove legacy FIFO code once all paths go through framecache

**Commit:** `feat(framecache): phase 5 — network ingest via framecache`

---

## Hardware / Deployment

| Node | RAM | /dev/shm | FC_SHM_SIZE |
|------|-----|----------|-------------|
| Baratheon | 251GB | 126GB | 60GB |
| zampp1 | 93GB | 47GB | 40GB |
| zampp2 | 18GB (upgrade) | 9.4GB | 8GB |

Ring buffer per 1080p59.94 source: ~494MB (120 frames × 4.1MB)

All recorder sidecars require `ipc: host`.

---

## Roadmap (not in this branch)

- Audio in framecache shm
- RDMA cross-node slot replication
- AJA hardware support
- Growing-file-while-recording browser HLS playback
- Mastercontrol/playout consumer