M2: WebRTC into datarhei Core proper #4
Loading…
Reference in a new issue
No description provided.
Delete branch "m2-webrtc-core-integration"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Promotes the M1 standalone
cmd/webrtc-pocPoC into a first-class output of the datarhei Core binary. After this PR a process whoseconfig.webrtc.enabled = trueautomatically gets two RTP output legs (video + audio) and a JWT-protected WHEP endpoint atPOST /api/v3/whep/{processID}.Design:
docs/design/2026-04-17-datarhei-dragon-fork-m2-webrtc-core-integration.mdPlan:
docs/superpowers/plans/2026-04-17-m2-webrtc-core-integration.mdArchitecture (recap)
app/webrtc/subsystem — owns thecore/webrtc.Registry,PeerFactory, port allocator, and Echo-mounted WHEP handler.restream:ProcessHooks(OnStart/OnStop) andAppendOutputfor injecting the RTP legs at exec time.ConfigWebRTConrestream/app.Config(jsonwebrtc).WebRTCblock onconfig.DatawithCORE_WEBRTC_*env bindings.core/webrtcfrom M1 is unchanged.What's in this PR
docs/design/+docs/superpowers/plans/— full design + step-by-step planrestream/app/process.go—ConfigWebRTCtype,Config.Clone()carries itconfig/data.go+config/config.go— global block + env varsrestream/restream.go—ProcessHook,ProcessHooks,AppendOutput, OnStart/OnStop wired into the task lifecycleapp/webrtc/{subsystem,lifecycle,portalloc,ffmpeg_args,handler}.go+ their testsapp/api/api.go— subsystem instantiation, hook registrationhttp/server.go— WHEP routes mounted under/api/v3(inherits JWT)deploy/truenas/core/— Dockerfile + compose for the production deployapp/webrtc/integration_test.go— synthetic-RTP smoke testBug fixes already in this PR
2d29dc9Config.Clone()was dropping the WebRTC sectionenable=trueenv, but WHEP returned 404 anywayf6d36bfProcessConfigwas droppingwebrtc.enabledonMarshal()POST /api/v3/processsucceeded butwebrtcblock silently zero'd before reaching restream0417afftest/whep-clientcouldn't talk to a JWT-gated endpoint-tokenfor/api/v3/whep/...Acceptance criteria from the design (§ 8)
webrtc.enable=trueglobal block acceptedconfig.webrtc.enabled=truepersists and startsPOST /api/v3/process/{id}/whepreturns 201 with valid JWT, ICE reaches connectedcore-uirepo-racego test -race ./core/webrtc/... ./app/webrtc/... ./config/... ./restream/...cleanLive deploy proof
The TrueNAS deploy at
dragonfork-corehas been running build2d29dc9since 2026-04-17, surviving daily traffic. After commitf6d36bfit has been rebuilt and verified:POST /api/v3/processwithwebrtc.enabled=true→ 200, persisted with WebRTC block intactWebRTC egress registered for process audio_port=49878 audio_pt=111 id="smoke" video_port=49877 video_pt=102... -f rtp udp://127.0.0.1:49877 ... -f rtp udp://127.0.0.1:49878POST /api/v3/whep/<id>returns 201 (verified with empty SDP → expectedset remote: failed to unmarshal SDP: EOF, route+auth+lookup all working)Known gaps tracked as follow-ups
BuildArgshardcodes-map 0:v:0/-map 0:a:0. Fine for RTMP/SRT (single combined input); breaks multi-input lavfi test pipelines. M3./api/v3/whep/...(docs.go pre-dates these routes). M4.Tests
Commits
(Co-authored with Claude Opus 4.7.)
Pion webrtc/v4 (v4.2.11) requires Go 1.24+. Upstream datarhei was at go 1.21.0. Bumping to go 1.24.0 pulls minor bumps across testify, golang.org/x/{crypto,net,sync,sys,text,time,tools,mod}; vendor/ is regenerated via 'go mod vendor' to reflect the new versions. No application code changes; pure dep bump to unblock M1.M2 promotes the M1 standalone PoC into the datarhei Core binary so WebRTC becomes a first-class output alongside RTMP/SRT/HLS, surfaced in the core-ui dashboard. Architecture: new app/webrtc sibling subsystem + two small hooks on restream (ProcessHooks + AppendOutput), reusing the untouched M1 core/webrtc package. WHEP served under /api/v3/process/{id}/whep, inheriting JWT auth. A new "Live (WebRTC)" tab on the process detail view provides the embedded browser player. Covers: purpose, architecture diagram, decision table, components, data flow (enable/subscribe/stop/disable/restart), error handling, testing strategy (unit/integration/e2e), acceptance criteria, rollback, and a seven-milestone sanity breakdown. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>Introduces the subsystem layer that sits alongside api.API and wires the M1 core/webrtc primitives into the per-process restream lifecycle. app/webrtc/subsystem.go: - Subsystem struct holding the global WebRTC config, core PeerFactory, per-process stream map, and logger - New(config.DataWebRTC, logger) constructor - Enabled(), Hooks(), Close(), lookup() methods app/webrtc/lifecycle.go: - onProcessStart: allocates an adjacent UDP port pair, binds two Pion Sources (video on V, audio on V+1), registers them under the process id, and returns the two RTP output legs to append to the FFmpeg command. - onProcessStop: tears down the pair. - allocAdjacentPair: retries up to 10 times to find a free (V, V+1) pair since the kernel's ephemeral picker can hand us an odd port. - splitRTPLegs: converts BuildArgs' flat []string into two ConfigIO entries by splitting on the second -map token. core/webrtc/peer.go + forward.go: - Adds PeerFactory.CreatePeerFromSources for the M2 two-source forwarding mode (video and audio on separate UDP ports, no payload-type sniffing). Leaves CreatePeer intact for the M1 PoC. - Adds forwardRTPSplit companion goroutine. config/data.go: - Promote anonymous WebRTC struct to named type DataWebRTC so app/webrtc can accept it by value.ProcessConfig in http/api/process.go shipped without a WebRTC field, so JSON arriving at POST /api/v3/process was silently stripped of "webrtc":{"enabled":true}. Marshal() handed restream a zero ConfigWebRTC, the OnProcessStart hook no-op'd, and every WHEP request returned 404 — even with a running webrtc-enabled process. Caught on the M2 TrueNAS deploy at acceptance time: GET /process/{id}/config came back without the webrtc block, despite the inbound JSON having it. This is the API-layer twin of the earlier 'fix(config): preserve WebRTC section in Config.Clone()' — same class of bug (drop-on-copy), different struct. - Add ProcessConfigWebRTC mirroring app.ConfigWebRTC. - Marshal: copy DTO -> app.Config.WebRTC. - Unmarshal: copy app.Config.WebRTC -> DTO. - Regression tests cover both the JSON->DTO->Config path and the default (no webrtc block) case. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>Major Handler rewrite implementing the design's M3 acceptance criteria ('5 concurrent viewers, all error paths correct, clean teardown'): Multi-viewer correctness: - streamID -> resourceID -> Peer two-level index (was flat) - per-stream peer cap alongside total cap, defaults match the design's '5–8 viewer' target (8/stream, total from corewebrtc) - per-peer awaitPeerClose goroutine watches Peer.Done() so ICE failures yank the index entry + decrement the counter (no leaks) - tearDownStreamPeers callback (registered with Subsystem in NewHandler) drives all peer closes when the source process stops Error matrix from design §6: - 406 on codec mismatch (offer missing H264 or Opus rtpmap) - 504 on ICE gathering timeout (passthrough from CreatePeerFromSources) - 204 on DELETE unknown resource (idempotent per WHEP spec; was 404) - 503 on per-stream cap reached (separate body from total-cap 503) - 400 on missing/empty body (unchanged) - 404 on unknown stream (unchanged) WHEP spec compatibility: - PATCH /whep/:id/:resource for trickle-ICE - OPTIONS preflight on every WHEP path - CORS Allow-Origin/Methods/Headers + Expose-Headers (Location, ETag) - ETag header on Subscribe response Defensive nil-peer guards in tearDown / Close paths so a partial state doesn't panic. Refactor: 134 -> 341 lines on handler.go but the surface is the same (NewHandler/Register/Subscribe/Unsubscribe/Close); existing callers continue to work. Pre-M3 test 'Unsubscribe_404WhenUnknown' renamed and updated to the new 204 expectation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>The WHEP routes were mounted by http/server.go via the app/webrtc Handler.Register(), but Subscribe and Unsubscribe carried no swag annotations. The Swagger UI at /api/swagger/index.html therefore didn't list /api/v3/whep/* — programmatic API consumers and humans browsing the docs couldn't discover the endpoints. Adds the standard upstream-shaped @Summary / @Tags / @ID / @Router annotations on Subscribe and Unsubscribe (matching the rtmp.go and srt.go pattern) and regenerates docs/{docs.go,swagger.json,swagger.yaml} via 'make swagger'. Verified: swagger.json now contains both paths, swagger UI renders them under the v16.16.0 tag. Closes #3. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>Three artifacts that close out the easier half of the M4 milestone: 1. .forgejo/workflows/test.yml — CI on every push and PR. Three jobs: - lint-and-vet: go vet + go build (~30s) - test: go test -race -short ./... + a no-race coverage pass that uploads coverage.out as an artifact - webrtc-smoke: TestIntegration_FiveViewerFanout and the rest of the WebRTC subsystem tests in isolation, so a failure on the egress path stays readable in the log. Pinned to Go 1.24 to match go.mod. The forge has a forgejo-runner sibling container; this YAML uses GitHub Actions syntax which Forgejo Actions accepts unchanged. 2. test/whep-player.html — self-contained browser WHEP subscriber for manual smoke testing. RTCPeerConnection (recvonly V+A) + fetch() POST/DELETE/PATCH against /api/v3/whep/:id, ICE/PC state pills, inbound-bitrate sampling at 1 Hz, codec hint pulled from the answer SDP, JWT token field, ?url=&token= shareable query string. No external deps; works from file:// or any static host. 3. test/TESTING.md — short doc that ties together the in-process race tests, the browser player, and the existing Pion CLI helper at test/whep-client/. Notes the latency p95 gate as a follow-up. Latency gate (FFmpeg drawtext frame counter + decode-side pixel sampling, p95 < 300ms RTMP / < 200ms SRT) is queued for a separate PR — it's a several-hundred-line addition in its own right and shouldn't block CI from landing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>M5 / final M2-stack work. The fork now identifies itself unambiguously in logs, the API, and the README without changing the Go module path (internal imports stay at github.com/datarhei/core/v16 — see NOTES.md for the rationale). Identity surfaces: - app/version.go gains Variant ('dragonfork') and Fork ('Datarhei — Dragon Fork') as vars (overridable via -ldflags for downstream re-packagers). - api.About + the /api endpoint expose 'variant' and 'fork' fields; Swagger docs regenerated. - Startup banner logs 'variant' + 'fork' alongside the existing application + version fields, so a TrueNAS sysadmin tail-following /var/log can tell at a glance which fork is running. Documentation: - README.md rewritten with a Dragon Fork header and Quick start; the upstream feature surface is summarised in 'From upstream Datarhei' with a clear additivity statement. Sample process JSON, multi-input pipeline guidance, link to the design + testing docs. - NOTICE: Apache 2.0 §4(d) attribution to upstream datarhei Core, Pion, Echo, FFmpeg. - CREDITS: enumerated dependency list with licenses. - CHANGELOG.md prepended with a 'Datarhei — Dragon Fork' section starting at v0.1.0-dragonfork; upstream's '# Core' history preserved below. Module path stays github.com/datarhei/core/v16 by design — the fork is distinguished by repo location and branch history, not import path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>Merged into
mainvia direct push as part of the v0.1.0-dragonfork release. Branch commits are reachable from main; closing this PR. Release: https://forge.wilddragon.net/zgaetano/datarhei-dragonfork-core/releases/tag/v0.1.0-dragonforkPull request closed