feat(framecache): add implementation plan to docs
This commit is contained in:
parent
c269468014
commit
2f1697b77b
1 changed files with 221 additions and 0 deletions
221
docs/design/framecache/PLAN.md
Normal file
221
docs/design/framecache/PLAN.md
Normal file
|
|
@ -0,0 +1,221 @@
|
|||
# Unified Framecache — Implementation Plan
|
||||
|
||||
## Context
|
||||
|
||||
Replace the current named-FIFO-per-source architecture with a shared-memory
|
||||
ring buffer (framecache) that fans raw video frames from any ingest source to
|
||||
unlimited concurrent consumers with zero-copy reads.
|
||||
|
||||
**Approved design:** docs/design/framecache/DESIGN.md
|
||||
**Branch:** feat/unified-framecache
|
||||
**Roadmap (out of scope here):** RDMA cross-node, AJA, growing-file-while-recording browser playback
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
Ship in 5 phases. Each phase is independently deployable and leaves the system
|
||||
in a working state. Existing recording workflows are unaffected until Phase 5
|
||||
cuts over.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Framecache Container (foundation)
|
||||
|
||||
**Goal:** Running framecache service with slot registry. No ingest writers yet.
|
||||
|
||||
### 1.1 — Create `services/framecache/` directory structure
|
||||
|
||||
```
|
||||
services/framecache/
|
||||
src/
|
||||
framecache.c # main — slot manager + HTTP API
|
||||
slot.c / slot.h # shm ring buffer lifecycle
|
||||
registry.c # /dev/shm/framecache/registry.json writer
|
||||
http.c # lightweight HTTP server (libmicrohttpd)
|
||||
client/
|
||||
fc_client.c / fc_client.h # consumer library
|
||||
fc_client_node/
|
||||
binding.cc # Node.js N-API addon
|
||||
binding.gyp
|
||||
Dockerfile
|
||||
CMakeLists.txt
|
||||
```
|
||||
|
||||
### 1.2 — Shared memory layout (slot.h)
|
||||
|
||||
Each slot lives at `/dev/shm/framecache/<slot_id>`:
|
||||
|
||||
```c
|
||||
#define FC_MAGIC 0x46524D43 // "FRMC"
|
||||
#define FC_RING_DEPTH 120 // ~2s at 59.94fps
|
||||
#define FC_HEADER_SIZE 4096 // 4KB header block
|
||||
|
||||
typedef struct {
|
||||
uint32_t magic;
|
||||
uint32_t version; // = 1
|
||||
uint32_t width;
|
||||
uint32_t height;
|
||||
uint32_t fps_num;
|
||||
uint32_t fps_den;
|
||||
uint32_t pixel_format; // FC_PIX_UYVY422 = 0
|
||||
uint32_t frame_size; // width * height * 2
|
||||
uint32_t ring_depth; // = FC_RING_DEPTH
|
||||
_Atomic uint64_t write_cursor; // monotonically increasing frame index
|
||||
_Atomic uint64_t dropped_frames;
|
||||
uint8_t _pad[FC_HEADER_SIZE - 48];
|
||||
} fc_header_t;
|
||||
|
||||
typedef struct {
|
||||
uint64_t pts_us;
|
||||
uint64_t wall_us;
|
||||
uint32_t size;
|
||||
uint8_t data[]; // frame_size bytes
|
||||
} fc_frame_t;
|
||||
```
|
||||
|
||||
Semaphore: `sem_open("/framecache-<slot_id>-write", ...)` — posted by writer
|
||||
on each new frame, consumers `sem_timedwait` on it.
|
||||
|
||||
### 1.3 — HTTP API (port 7435)
|
||||
|
||||
```
|
||||
POST /slots body: {slot_id, width, height, fps_num, fps_den, source_type}
|
||||
creates shm region, writes registry entry
|
||||
201 {slot_id, shm_path, sem_name}
|
||||
|
||||
GET /slots 200 [{slot_id, width, height, fps_num, fps_den,
|
||||
source_type, write_cursor, dropped_frames,
|
||||
current_fps}]
|
||||
|
||||
GET /slots/:id 200 slot detail
|
||||
DELETE /slots/:id destroys shm + semaphore, removes registry entry, 204
|
||||
GET /health 200 {status: "ok"}
|
||||
```
|
||||
|
||||
### 1.4 — Registry file
|
||||
|
||||
Written to `/dev/shm/framecache/registry.json` on every slot create/delete.
|
||||
|
||||
### 1.5 — Dockerfile
|
||||
|
||||
```dockerfile
|
||||
FROM debian:bookworm
|
||||
RUN apt-get update && apt-get install -y \
|
||||
build-essential cmake libmicrohttpd-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY . /src
|
||||
RUN cmake -S /src -B /build -DCMAKE_BUILD_TYPE=Release \
|
||||
&& cmake --build /build -j$(nproc)
|
||||
EXPOSE 7435
|
||||
CMD ["/build/framecache"]
|
||||
```
|
||||
|
||||
### 1.6 — docker-compose.worker.yml addition
|
||||
|
||||
```yaml
|
||||
framecache:
|
||||
build: ./services/framecache
|
||||
ipc: host
|
||||
shm_size: '60gb'
|
||||
environment:
|
||||
FC_SHM_SIZE: ${FC_SHM_SIZE:-64424509440}
|
||||
FC_PORT: 7435
|
||||
ports:
|
||||
- "7435:7435"
|
||||
volumes:
|
||||
- /dev/shm:/dev/shm
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
### 1.7 — Consumer library (fc_client.c)
|
||||
|
||||
```c
|
||||
fc_slot_t *fc_open(const char *slot_id);
|
||||
int fc_read_frame(fc_slot_t *slot, fc_frame_t **out, uint64_t timeout_ms);
|
||||
void fc_close(fc_slot_t *slot);
|
||||
```
|
||||
|
||||
**Commit:** `feat(framecache): phase 1 — framecache container + consumer library`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Deltacast Bridge writes to framecache
|
||||
|
||||
**Goal:** deltacast-bridge writes frames to framecache shm instead of named FIFOs.
|
||||
Legacy FIFO path kept as compile-time fallback (`-DLEGACY_FIFO=ON`) until Phase 5.
|
||||
|
||||
On signal lock:
|
||||
1. POST /slots to framecache HTTP API
|
||||
2. shm_open + mmap the slot
|
||||
3. Video thread writes frame into ring, advances write_cursor atomically, sem_post
|
||||
4. Audio: keeps writing to audio FIFO (unchanged)
|
||||
5. On shutdown: DELETE /slots/:id
|
||||
|
||||
**Commit:** `feat(framecache): phase 2 — deltacast-bridge writes to shm`
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Blackmagic DeckLink Bridge
|
||||
|
||||
**Goal:** New decklink-bridge C program mirrors deltacast-bridge, replaces
|
||||
ffmpeg -f decklink direct path.
|
||||
|
||||
- Uses IDeckLinkIterator to enumerate devices
|
||||
- VideoInputFrameArrived callback calls fc_write_frame
|
||||
- Registers slot on signal lock, deregisters on shutdown
|
||||
- Audio stays in FIFO (same as deltacast)
|
||||
|
||||
**Commit:** `feat(framecache): phase 3 — decklink-bridge writes to shm`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — capture-manager reads from framecache
|
||||
|
||||
**Goal:** Enables simultaneous growing + proxy + HLS from one SDI input.
|
||||
|
||||
- Node.js N-API addon wrapping fc_open/fc_read_frame/fc_close
|
||||
- capture-manager opens THREE fc_client handles per slot (own cursor each):
|
||||
1. Growing/master ffmpeg feed
|
||||
2. Proxy ffmpeg feed
|
||||
3. HLS preview ffmpeg feed
|
||||
- Each gets a separate rawvideo pipe to ffmpeg
|
||||
- Growing MXF workflow (raw2bmx orchestrator) completely unchanged
|
||||
|
||||
**Commit:** `feat(framecache): phase 4 — capture-manager reads from framecache`
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Network ingest (RTMP/SRT) into framecache
|
||||
|
||||
**Goal:** RTMP and SRT sources decoded to raw UYVY422, written into framecache slots.
|
||||
|
||||
- net_ingest process per source: ffmpeg decodes to rawvideo, writes to slot
|
||||
- capture-manager waits for slot, same fc_client consumer pattern
|
||||
- Remove legacy FIFO code once all paths go through framecache
|
||||
|
||||
**Commit:** `feat(framecache): phase 5 — network ingest via framecache`
|
||||
|
||||
---
|
||||
|
||||
## Hardware / Deployment
|
||||
|
||||
| Node | RAM | /dev/shm | FC_SHM_SIZE |
|
||||
|------|-----|----------|-------------|
|
||||
| Baratheon | 251GB | 126GB | 60GB |
|
||||
| zampp1 | 93GB | 47GB | 40GB |
|
||||
| zampp2 | 18GB (upgrade) | 9.4GB | 8GB |
|
||||
|
||||
Ring buffer per 1080p59.94 source: ~494MB (120 frames × 4.1MB)
|
||||
|
||||
All recorder sidecars require `ipc: host`.
|
||||
|
||||
---
|
||||
|
||||
## Roadmap (not in this branch)
|
||||
|
||||
- Audio in framecache shm
|
||||
- RDMA cross-node slot replication
|
||||
- AJA hardware support
|
||||
- Growing-file-while-recording browser HLS playback
|
||||
- Mastercontrol/playout consumer
|
||||
Loading…
Reference in a new issue