datarhei-dragonfork-core/CHANGELOG.md

Datarhei — Dragon Fork

v0.4.0-dragonfork (2026-05-10)

WebRTC protocol compliance milestone. Focuses on correctness and operator experience: complete H.264 packetisation coverage, RFC-compliant signalling headers, proper multi-homed NAT support, and comprehensive test coverage for both WHEP and WHIP.

Resolves issues #18, #19, #20, #21.

Added

• STAP-A IDR detection (core/webrtc/keyframecache.go). The H.264 keyframe cache now handles all three RTP packetisation modes for IDR slices: single-NAL (type 5), FU-A start (type 28 with start bit), and STAP-A aggregation packets (type 24, first inner NAL type 5). Without this, an encoder that sends the SPS/PPS/IDR as a single STAP-A aggregate would never trigger a cache reset and late-joining subscribers would not receive a reference frame. Closes #18.

  • Four new tests in keyframecache_test.go exercise STAP-A leading IDR, non-IDR first NAL, and truncated (1-/2-/3-byte) payloads.

• WHEP Link headers — RFC 9429 §4.3 (app/webrtc/handler.go). The WHEP 201 Subscribe response now emits one Link: <uri>; rel="ice-server" header per configured ICE server URI. Browsers can discover STUN/TURN endpoints from the response without a separate signalling round-trip. addCORS() updated to expose Link in Access-Control-Expose-Headers. ICEServerURIs() []string added to Subsystem. Closes #19.

  • Tests: TestSubsystem_ICEServerURIs_ReturnsConfiguredURIs, TestAddCORS_ExposesLinkHeader in handler_test.go.

• Multi-IP NAT1To1 support (core/webrtc/config.go, core/webrtc/ice.go, app/webrtc/subsystem.go). The NAT1To1 config now accepts a list of IPs (NAT1To1IPs []string) instead of a single string. Pion's SetNAT1To1IPs receives the full list so dual-homed servers (e.g., LAN + public) advertise host candidates on all interfaces. BuildICEConfig falls back to the legacy PublicIP field as a single-element list for backward compatibility. Subsystem merges PublicIP + NAT1To1IPs with deduplication. Closes #20.

  • Five new tests in ice_test.go: multi-IP list, PublicIP fallback, both-set, neither-set, invalid config rejection.

• WHIP Link headers — RFC 9261 §5.2 (app/webrtc/whip_handler.go). Symmetric with the WHEP change: the WHIP 201 Publish response now emits Link headers for each ICE server, allowing OBS, GStreamer, and browser-based publishers to discover STUN/TURN from the offer response. Closes #21.

• WHIP handler test suite (app/webrtc/whip_handler_test.go, new). Nine tests covering Publish/Unpublish/TrickleIngest routes and CORS preflight. Verifies 404 on missing ingest, 400 on bad body, 204 on idempotent DELETE of unknown resource, 404 on PATCH to unknown peer, and Link/Location/ETag present in CORS expose-headers.

Upgrade (from v0.3)

No config changes required. The new NAT1To1IPs field in DataWebRTC defaults to empty, preserving the existing single-IP behaviour via PublicIP.

---

v0.3.0-dragonfork (2026-05-10)

WebRTC ingest (WHIP) milestone. Browsers and OBS can now push a WebRTC stream into a channel, and the first-frame experience for WHEP viewers is dramatically improved by the in-memory keyframe cache.

Resolves issues #15, #16, #17.

Added

• WHIP ingest path — browsers and OBS Studio can push a WebRTC stream (H.264 + Opus) into any Dragon Fork channel via POST /api/v3/whip/{id}. The publisher sends an SDP offer; Core answers, allocates a loopback UDP pair, and injects RTP input legs into the FFmpeg command line — the exact mirror of the WHEP egress path. DELETE /api/v3/whip/{id}/{resource} tears down the publisher cleanly. Closes #16.

• ProcessConfigWHIPIngest API struct in http/api/process.go mapping whip_ingest.{enabled,video_pt,audio_pt} between the JSON API and app.ConfigWHIPIngest. Without this struct, WHIPIngest.Enabled was always false and WHIP could never activate via the API.

• WHIP ingest lifecycle hooks — onWHIPProcessStart / onWHIPProcessStop in app/webrtc/whip_lifecycle.go allocate and teardown the ingest UDP port pair, controlled by the per-process whip_ingest.enabled flag. Merged via MergedHooks() alongside the existing WHEP egress hooks.

• Wild Dragon UI — WHIP toggle control (overlay/src/misc/controls/WHIP.js in the wilddragon-restreamer-ui overlay). Mirrors WHEP.js exactly. Renders an Enable checkbox with caption in the channel edit view.

• Wild Dragon UI — Edit/index.js wiring — renders the WHIP control in the Edit view and patches props.restreamer._upsertProcess in the save() handler to inject whip_ingest.enabled into the process config before the SDK PUT reaches Core. The patch is required because the Restreamer SDK's UpsertIngest does not forward webrtc or whip_ingest fields (SDK gap).

• In-memory H.264 keyframe cache in core/webrtc/keyframecache.go. Retains the most recent IDR burst (all RTP packets from the first IDR NAL fragment until the next one) per video Source. Bounded at 512 packets / 2 MiB. Detects single-NAL IDR (type 5) and FU-A start fragments (type 28, start bit set, inner type 5). Closes #17.

• Subscribe pre-fill — Source.Subscribe() snapshots the keyframe cache before registering the new subscriber, then drains the burst into the channel immediately. New WHEP peers receive a complete reference frame on join instead of waiting up to one GOP (≈ 2 s at 30 fps / GOP=60).

• Source.EnableKeyFrameCache() — opt-in method; called only on video sources in allocAdjacentPair(). Audio sources are intentionally uncached (Opus payloads would accumulate without ever triggering a reset).

• Test suite for core/webrtc — keyframecache_test.go (18 functions) and source_test.go (5 functions). Covers IDR detection in all packetisation modes, cache reset, burst accumulation, capacity caps, snapshot independence, concurrent read/write under -race, and Subscribe pre-fill behaviour. All 34 tests in core/webrtc green under go test -race.

Fixed

• deploy/truenas/core/seed-data.sh — the old no-clobber-only approach kept stale JS bundles alive on the data volume after image rebuilds (static/ was never refreshed because it already existed). Fixed by splitting into two phases: always-overwrite for index.html, asset-manifest.json, and static/; no-clobber for everything else (channel data, player bundles, operator content). Prevents a class of "new code never runs" deployment bugs.

Upgrade (from v0.2)

cd deploy/truenas/core
git pull
docker compose build --no-cache core
docker compose up -d core

The seed-data.sh fix means there is no longer a need to manually docker exec a static-bundle copy after rebuilds — it happens automatically on container start.

---

v0.2 backlog (2026-05-06)

Completes the open v0.2 issues from the post-GUI-ship backlog. Resolves issues #11, #12, #13, #14.

Added

• WebRTC Prometheus metrics — eleven metrics in the dragonfork_webrtc_* namespace using RED-method principles. Hybrid instrumentation: direct client_golang counters/histograms for hot-path WHEP routes and ICE establishment in app/webrtc/metrics.go, plus a snapshot collector for gauges in prometheus/webrtc.go. Metrics: whep_requests_total, whep_request_duration_seconds, ice_establishment_duration_seconds, ice_failures_total, codec_mismatches_total, cap_rejections_total, ffmpeg_leg_failures_total, active_streams, active_peers, udp_ports_in_use. Closes #11.

• Grafana observability stack in deploy/truenas/core/:\n Prometheus v2.55 and Grafana OSS 11.3 containers on a dragonfork-mon bridge network reaching Core via host.docker.internal. Pre-loaded WebRTC Health dashboard (5 rows: WHEP API, ICE, streams/peers, capacity, silent-degradation canary). Four pre-loaded Prometheus alert rules. Deploy upgrade: add GRAFANA_ADMIN_PASSWORD to .env, docker compose pull && docker compose up -d. Closes #11.

• Docker image CI publish workflow at .forgejo/workflows/publish.yml. Triggers on semver tags. Builds multi-arch (linux/amd64 + linux/arm64) and pushes to the configured registry (REGISTRY repo variable, defaults to ghcr.io). Requires REGISTRY_TOKEN secret and optional REGISTRY_USER / IMAGE_NAME variables. Layer cache via GitHub Actions cache. Closes #12.

• Upstream rebase policy at docs/REBASE.md. Documents monthly cadence, rebase-not-merge strategy, Dragon Fork divergence boundaries, pre/post-rebase checklist, vendored-dependency procedure, first-rebase runbook, and record-keeping table. First rebase against upstream is pending (to be run locally per the procedure in docs/REBASE.md). Closes #13.

• WHEP sustained load test at test/load/sustained.go. Headless Go program (//go:build ignore, run with go run) that drives N concurrent WHEP subscribers against a single stream for a configurable duration. Measures: ICE establishment (p50/p95), jitter (RFC 3550 running average), packet loss estimate (sequence-number gaps), packets received. Outputs a markdown report to test/load/results/. Staggered connection setup, trickle-ICE, and graceful DELETE on teardown. Closes #14.

• core/webrtc.Peer.Connected() channel — closed on first PeerConnectionStateConnected event. Required by the ICE establishment histogram (allows async measurement after the WHEP POST returns).

Changed

• deploy/truenas/core/docker-compose.yml: adds prom and grafana services + dragonfork-mon bridge network + named volumes. core service is unchanged (stays on network_mode: host).
• app/webrtc/handler.go: WHEP route handlers now record request duration, status code, codec mismatch, and cap rejection metrics. tearDownStreamPeers records FFmpeg leg failures when peers were active at stop time.
• app/webrtc/subsystem.go: adds StreamCount() accessor for the snapshot collector.

Upgrade (from v0.2.0-dragonfork)

cd deploy/truenas/core
git pull
# Add new lines to .env:
#   GRAFANA_ADMIN_PASSWORD=$(openssl rand -base64 24)
#   GRAFANA_PORT=3000
#   PROM_PORT=9090
docker compose pull   # pulls prom + grafana images
docker compose up -d  # core unchanged, prom + grafana start fresh

---

v0.2.0-dragonfork (2026-05-03)

The "GUI ship" release. Everything from v0.1 is preserved; this round documents and ships a usable graphical surface for the WebRTC feature that v0.1 only exposed through the API.

Added

• Wild Dragon WebRTC admin page at /wilddragon-webrtc.html. Single-file HTML/JS; no build step. Sign in with the API_AUTH_USERNAME / PASSWORD creds, see every process, toggle webrtc.enabled per-process with one click, restart on change, copy the WHEP URL, jump straight to the smoke player. Closes the v0.1 GUI gap — the upstream Restreamer UI ships with v0.2 but doesn't know about Core's webrtc config block, so toggling WebRTC previously required direct API calls.

Documented (was present, just unannounced)

• Restreamer UI bundle in the TrueNAS deploy. The deploy/truenas/core/ Dockerfile builds the upstream datarhei/restreamer-ui v1.14.0 React bundle with the Wild Dragon overlay applied (logo / favicon / header title / welcome card), copies the result into Core's disk filesystem via seed-data.sh, and Core serves it at /. Was added during M2 but not called out in the v0.1 CHANGELOG.
• WHEP smoke player at /whep-player.html. Standalone WebRTC subscriber with ICE/codec/bitrate diagnostics. Was added during M4.

---

v0.1.0-dragonfork (2026-05-03)

The first tagged Dragon Fork release. Forked from upstream datarhei Core v16.16.0; everything upstream does is preserved unchanged. New: WebRTC (WHEP) egress, integrated with the existing process supervisor.

Added

• WebRTC subsystem under app/webrtc/, mirroring the shape of upstream's RTMP and SRT servers (Server interface, Echo handlers, process-graph hooks, admin endpoints).
• Per-process opt-in via config.webrtc.enabled on every restream process; resolver auto-injects two RTP output legs and allocates loopback UDP ports.
• POST /api/v3/whep/{id} — WebRTC-HTTP Egress Protocol subscribe. JWT-protected by the existing Core auth.
• DELETE /api/v3/whep/{id}/{resource} — idempotent teardown (returns 204 even on unknown resource per WHEP spec).
• PATCH /api/v3/whep/{id}/{resource} — trickle ICE.
• CORS preflight on every WHEP route + Access-Control-Expose-Headers for Location and ETag so browser-side WHEP players work cross-origin.
• Configurable stream maps via webrtc.video_map / webrtc.audio_map on the per-process config — defaults to 0:v:0 / 0:a:0 for RTMP/SRT publishers, overridable for multi-input pipelines.
• webrtc.* global config block with CORE_WEBRTC_* env-var bindings parallel to RTMP and SRT.
• Admin API: GET /api/v3/webrtc/streams + /streams/{id}/peers.
• Browser smoke player at test/whep-player.html with ICE / codec / bitrate diagnostics, JWT field, and ?url=&token= shareable URLs.
• Server-hop latency p95 gate in CI (-tags latency), enforced at 50ms on the runner; locally observed p95 ≈ 240µs.
• TrueNAS deploy bundle at deploy/truenas/core/ — host-networked Docker stack with bundled FFmpeg, env-driven config.
• Multi-viewer correctness: per-stream peer cap, ICE-failure auto-cleanup goroutines, process-stop broadcast tear-down.
• Error matrix: 406 codec mismatch, 504 ICE timeout, 503 cap reached (separate body for total vs per-stream), 204 DELETE idempotent.

Fixed

• Config.Clone() now preserves the WebRTC section.
• http/api.ProcessConfig Marshal/Unmarshal now carry the per-process webrtc block.

---

Core (upstream)

Core v16.15.0 > v16.16.0

• Add ConnectionIdleTimeout to RTMP server
• Add WithLevel() to Logger interface
• Fix datarhei/restreamer#759
• Fix various RTMP bugs
• Fix wrong log output when receiving a RTMP stream
• Fix skipping session handling if collectors are nil
• Update dependencies

Core v16.14.0 > v16.15.0

• Add migrating to ffmpeg 6
• Fix missing process data if process has been deleted meanwhile
• Fix maintaining the metadata on process config update (datarhei/restreamer#698)
• Fix placeholder parsing
• Fix concurrent memfs accesses
• Fix memfs concurrent read and write performance

Core v16.13.1 > v16.14.0

• Add support for SRTv4 clients
• Add support for Enhanced RTMP in internal RTMP server
• Fix require positive persist interval (session)
• Fix race condition (process)
• Update dependencies

Core v16.13.0 > v16.13.1

• Fix transfer of reports to updated process
• Fix calling Wait after process has been read
• Fix 509 return code if non-existing stream is requested
• Fix default search paths for config file
• Fix sized filesystem
• Update dependencies

Core v16.12.0 > v16.13.0

• Add updated_at field in process infos
• Add preserve process log history when updating a process
• Add support for input framerate data from jsonstats patch
• Add number of keyframes and extradata size to process progress data
• Mod bumps FFmpeg to v5.1.3 (datarhei/core:tag bundles)
• Fix better naming for storage endpoint documentation
• Fix freeing up S3 mounts
• Fix URL validation if the path contains FFmpeg specific placeholders
• Fix purging default file from HTTP cache
• Fix parsing S3 storage definition from environment variable
• Fix checking length of CPU time array (#10)
• Fix possible infinite loop with HLS session rewriter
• Fix not propagating process limits
• Fix URL validation if the path contains FFmpeg specific placeholders
• Fix RTMP DoS attack (thx Johannes Frank)
• Deprecate ENV names that do not correspond to JSON name