datarhei-dragonfork-core/CHANGELOG.md
ZGaetano 55b61dd0e5
Some checks failed
ci / race tests (push) Blocked by required conditions
ci / WebRTC smoke (5-viewer fanout) (push) Blocked by required conditions
ci / WebRTC latency p95 gate (push) Blocked by required conditions
ci / vet + build (push) Has been cancelled
docs: update CHANGELOG for v0.2 backlog work (closes #11, #12, #13, #14)
2026-05-06 16:03:09 -04:00

9.7 KiB

Datarhei — Dragon Fork

v0.2 backlog (2026-05-06)

Completes the open v0.2 issues from the post-GUI-ship backlog. Resolves issues #11, #12, #13, #14.

Added

  • WebRTC Prometheus metrics — eleven metrics in the dragonfork_webrtc_* namespace using RED-method principles. Hybrid instrumentation: direct client_golang counters/histograms for hot-path WHEP routes and ICE establishment in app/webrtc/metrics.go, plus a snapshot collector for gauges in prometheus/webrtc.go. Metrics: whep_requests_total, whep_request_duration_seconds, ice_establishment_duration_seconds, ice_failures_total, codec_mismatches_total, cap_rejections_total, ffmpeg_leg_failures_total, active_streams, active_peers, udp_ports_in_use. Closes #11.

  • Grafana observability stack in deploy/truenas/core/: Prometheus v2.55 and Grafana OSS 11.3 containers on a dragonfork-mon bridge network reaching Core via host.docker.internal. Pre-loaded WebRTC Health dashboard (5 rows: WHEP API, ICE, streams/peers, capacity, silent-degradation canary). Four pre-loaded Prometheus alert rules. Deploy upgrade: add GRAFANA_ADMIN_PASSWORD to .env, docker compose pull && docker compose up -d. Closes #11.

  • Docker image CI publish workflow at .forgejo/workflows/publish.yml. Triggers on semver tags. Builds multi-arch (linux/amd64 + linux/arm64) and pushes to the configured registry (REGISTRY repo variable, defaults to ghcr.io). Requires REGISTRY_TOKEN secret and optional REGISTRY_USER / IMAGE_NAME variables. Layer cache via GitHub Actions cache. Closes #12.

  • Upstream rebase policy at docs/REBASE.md. Documents monthly cadence, rebase-not-merge strategy, Dragon Fork divergence boundaries, pre/post-rebase checklist, vendored-dependency procedure, first-rebase runbook, and record-keeping table. First rebase against upstream is pending (to be run locally per the procedure in docs/REBASE.md). Closes #13.

  • WHEP sustained load test at test/load/sustained.go. Headless Go program (//go:build ignore, run with go run) that drives N concurrent WHEP subscribers against a single stream for a configurable duration. Measures: ICE establishment (p50/p95), jitter (RFC 3550 running average), packet loss estimate (sequence-number gaps), packets received. Outputs a markdown report to test/load/results/. Staggered connection setup, trickle-ICE, and graceful DELETE on teardown. Closes #14.

  • core/webrtc.Peer.Connected() channel — closed on first PeerConnectionStateConnected event. Required by the ICE establishment histogram (allows async measurement after the WHEP POST returns).

Changed

  • deploy/truenas/core/docker-compose.yml: adds prom and grafana services + dragonfork-mon bridge network + named volumes. core service is unchanged (stays on network_mode: host).
  • app/webrtc/handler.go: WHEP route handlers now record request duration, status code, codec mismatch, and cap rejection metrics. tearDownStreamPeers records FFmpeg leg failures when peers were active at stop time.
  • app/webrtc/subsystem.go: adds StreamCount() accessor for the snapshot collector.

Known limitations (remaining v0.2 open items)

  • Restreamer UI fork (#15): separate repo, not started. The upstream Restreamer UI does not yet have a WebRTC toggle; use /wilddragon-webrtc.html in the meantime.
  • First upstream rebase (#13, partially done): docs/REBASE.md is committed; the actual git rebase upstream/main must be run locally per the procedure. Record the result in the REBASE.md table.

Upgrade (from v0.2.0-dragonfork)

cd deploy/truenas/core
git pull
# Add new lines to .env:
#   GRAFANA_ADMIN_PASSWORD=$(openssl rand -base64 24)
#   GRAFANA_PORT=3000
#   PROM_PORT=9090
docker compose pull   # pulls prom + grafana images
docker compose up -d  # core unchanged, prom + grafana start fresh

To publish an image for the first time, set REGISTRY, REGISTRY_USER, IMAGE_NAME, and REGISTRY_TOKEN in repo settings, then tag:

git tag v0.2.1-dragonfork && git push origin v0.2.1-dragonfork

v0.2.0-dragonfork (2026-05-03)

The "GUI ship" release. Everything from v0.1 is preserved; this round documents and ships a usable graphical surface for the WebRTC feature that v0.1 only exposed through the API.

Added

  • Wild Dragon WebRTC admin page at /wilddragon-webrtc.html. Single-file HTML/JS; no build step. Sign in with the API_AUTH_USERNAME / PASSWORD creds, see every process, toggle webrtc.enabled per-process with one click, restart on change, copy the WHEP URL, jump straight to the smoke player. Closes the v0.1 GUI gap — the upstream Restreamer UI ships with v0.2 but doesn't know about Core's webrtc config block, so toggling WebRTC previously required direct API calls.

Documented (was present, just unannounced)

  • Restreamer UI bundle in the TrueNAS deploy. The deploy/truenas/core/ Dockerfile builds the upstream datarhei/restreamer-ui v1.14.0 React bundle with the Wild Dragon overlay applied (logo / favicon / header title / welcome card), copies the result into Core's disk filesystem via seed-data.sh, and Core serves it at /. Was added during M2 but not called out in the v0.1 CHANGELOG.
  • WHEP smoke player at /whep-player.html. Standalone WebRTC subscriber with ICE/codec/bitrate diagnostics. Was added during M4.

v0.1.0-dragonfork (2026-05-03)

The first tagged Dragon Fork release. Forked from upstream datarhei Core v16.16.0; everything upstream does is preserved unchanged. New: WebRTC (WHEP) egress, integrated with the existing process supervisor.

Added

  • WebRTC subsystem under app/webrtc/, mirroring the shape of upstream's RTMP and SRT servers (Server interface, Echo handlers, process-graph hooks, admin endpoints).
  • Per-process opt-in via config.webrtc.enabled on every restream process; resolver auto-injects two RTP output legs and allocates loopback UDP ports.
  • POST /api/v3/whep/{id} — WebRTC-HTTP Egress Protocol subscribe. JWT-protected by the existing Core auth.
  • DELETE /api/v3/whep/{id}/{resource} — idempotent teardown (returns 204 even on unknown resource per WHEP spec).
  • PATCH /api/v3/whep/{id}/{resource} — trickle ICE.
  • CORS preflight on every WHEP route + Access-Control-Expose-Headers for Location and ETag so browser-side WHEP players work cross-origin.
  • Configurable stream maps via webrtc.video_map / webrtc.audio_map on the per-process config — defaults to 0:v:0 / 0:a:0 for RTMP/SRT publishers, overridable for multi-input pipelines.
  • webrtc.* global config block with CORE_WEBRTC_* env-var bindings parallel to RTMP and SRT.
  • Admin API: GET /api/v3/webrtc/streams + /streams/{id}/peers.
  • Browser smoke player at test/whep-player.html with ICE / codec / bitrate diagnostics, JWT field, and ?url=&token= shareable URLs.
  • Server-hop latency p95 gate in CI (-tags latency), enforced at 50ms on the runner; locally observed p95 ≈ 240µs.
  • TrueNAS deploy bundle at deploy/truenas/core/ — host-networked Docker stack with bundled FFmpeg, env-driven config.
  • Multi-viewer correctness: per-stream peer cap, ICE-failure auto-cleanup goroutines, process-stop broadcast tear-down.
  • Error matrix: 406 codec mismatch, 504 ICE timeout, 503 cap reached (separate body for total vs per-stream), 204 DELETE idempotent.

Fixed

  • Config.Clone() now preserves the WebRTC section.
  • http/api.ProcessConfig Marshal/Unmarshal now carry the per-process webrtc block.

Core (upstream)

Core v16.15.0 > v16.16.0

  • Add ConnectionIdleTimeout to RTMP server
  • Add WithLevel() to Logger interface
  • Fix datarhei/restreamer#759
  • Fix various RTMP bugs
  • Fix wrong log output when receiving a RTMP stream
  • Fix skipping session handling if collectors are nil
  • Update dependencies

Core v16.14.0 > v16.15.0

  • Add migrating to ffmpeg 6
  • Fix missing process data if process has been deleted meanwhile
  • Fix maintaining the metadata on process config update (datarhei/restreamer#698)
  • Fix placeholder parsing
  • Fix concurrent memfs accesses
  • Fix memfs concurrent read and write performance

Core v16.13.1 > v16.14.0

  • Add support for SRTv4 clients
  • Add support for Enhanced RTMP in internal RTMP server
  • Fix require positive persist interval (session)
  • Fix race condition (process)
  • Update dependencies

Core v16.13.0 > v16.13.1

  • Fix transfer of reports to updated process
  • Fix calling Wait after process has been read
  • Fix 509 return code if non-existing stream is requested
  • Fix default search paths for config file
  • Fix sized filesystem
  • Update dependencies

Core v16.12.0 > v16.13.0

  • Add updated_at field in process infos
  • Add preserve process log history when updating a process
  • Add support for input framerate data from jsonstats patch
  • Add number of keyframes and extradata size to process progress data
  • Mod bumps FFmpeg to v5.1.3 (datarhei/core:tag bundles)
  • Fix better naming for storage endpoint documentation
  • Fix freeing up S3 mounts
  • Fix URL validation if the path contains FFmpeg specific placeholders
  • Fix purging default file from HTTP cache
  • Fix parsing S3 storage definition from environment variable
  • Fix checking length of CPU time array (#10)
  • Fix possible infinite loop with HLS session rewriter
  • Fix not propagating process limits
  • Fix URL validation if the path contains FFmpeg specific placeholders
  • Fix RTMP DoS attack (thx Johannes Frank)
  • Deprecate ENV names that do not correspond to JSON name