Conflict resolution: keep M3's full handler.go rewrite (per-stream
index, error matrix, PATCH, CORS, auto-cleanup) and re-apply the
swagger annotations from #7 onto the new function declarations,
including a fresh annotation for the M3-introduced Trickle endpoint.
Swagger docs regenerated to pick up all three.
Race-clean: go test -race ./app/webrtc/... green.
M5 / final M2-stack work. The fork now identifies itself unambiguously
in logs, the API, and the README without changing the Go module path
(internal imports stay at github.com/datarhei/core/v16 — see NOTES.md
for the rationale).
Identity surfaces:
- app/version.go gains Variant ('dragonfork') and Fork ('Datarhei —
Dragon Fork') as vars (overridable via -ldflags for downstream
re-packagers).
- api.About + the /api endpoint expose 'variant' and 'fork' fields;
Swagger docs regenerated.
- Startup banner logs 'variant' + 'fork' alongside the existing
application + version fields, so a TrueNAS sysadmin tail-following
/var/log can tell at a glance which fork is running.
Documentation:
- README.md rewritten with a Dragon Fork header and Quick start; the
upstream feature surface is summarised in 'From upstream Datarhei'
with a clear additivity statement. Sample process JSON, multi-input
pipeline guidance, link to the design + testing docs.
- NOTICE: Apache 2.0 §4(d) attribution to upstream datarhei Core,
Pion, Echo, FFmpeg.
- CREDITS: enumerated dependency list with licenses.
- CHANGELOG.md prepended with a 'Datarhei — Dragon Fork' section
starting at v0.1.0-dragonfork; upstream's '# Core' history preserved
below.
Module path stays github.com/datarhei/core/v16 by design — the fork is
distinguished by repo location and branch history, not import path.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds an end-to-end RTP-arrival latency probe that runs as a dedicated
CI job and asserts p95 < 50ms.
Implementation
--------------
A build-tagged test (-tags latency, off by default) sends 1000
synthetic RTP packets at 60Hz into corewebrtc.Source and reads them
back via a Pion subscriber's track.ReadRTP(). Each packet's payload
starts with the publisher's UnixNano send time; the subscriber diffs
against time.Now() at arrival and accumulates p50/p95/p99.
This exercises every link of the egress hop: Source UDP read,
subscriber fan-out, forwardRTPSplit, Pion's TrackLocalStaticRTP
write, DTLS-SRTP encrypt, ICE socket write, decrypt at the
subscriber, RTP unmarshal at ReadRTP. Pure server-side; no FFmpeg
or codecs involved.
Why not glass-to-glass
----------------------
The design's §7 calls for FFmpeg drawtext frame counters + decode-
side pixel sampling, p95<300ms RTMP / <200ms SRT. Implementing that
in pure Go needs a cgo H.264 decoder or an FFmpeg sidecar pipe — a
significantly bigger lift for a marginal regression-detection win
(encode/decode latency is roughly fixed by the codec stack and
isn't moved by Core code changes). The server-hop measurement
captures everything Core code can actually regress.
Threshold
---------
50ms p95. Locally observed on a quiet host:
p50=110µs, p95=237µs, p99=318µs.
The 50ms gate is ~200x headroom — generous enough to absorb CI
runner noise without false alarms, tight enough to catch a real
slowdown.
Race-clean: latencySamples uses a sync.Mutex around the slice append
(initial draft had a slice racing with the receive goroutine; vet
caught it).
Documented in test/TESTING.md and wired to .forgejo/workflows/test.yml
as the latency-gate job (depends on lint-and-vet, parallel with test
and webrtc-smoke).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The WHEP routes were mounted by http/server.go via the app/webrtc
Handler.Register(), but Subscribe and Unsubscribe carried no swag
annotations. The Swagger UI at /api/swagger/index.html therefore
didn't list /api/v3/whep/* — programmatic API consumers and humans
browsing the docs couldn't discover the endpoints.
Adds the standard upstream-shaped @Summary / @Tags / @ID / @Router
annotations on Subscribe and Unsubscribe (matching the rtmp.go and
srt.go pattern) and regenerates docs/{docs.go,swagger.json,swagger.yaml}
via 'make swagger'. Verified: swagger.json now contains both paths,
swagger UI renders them under the v16.16.0 tag.
Closes#3.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
BuildArgs hardcoded -map 0✌️0 / -map 0🅰️0 for the two RTP legs.
Correct for production RTMP/SRT publishers (single combined input),
but breaks any process whose audio lives on a different input index
— multi-input lavfi test scaffolds, multi-camera pipelines, SDI +
file-audio mixes, etc.
Adds VideoMap and AudioMap fields to ConfigWebRTC (and the API DTO),
defaulting to the prior literals so existing deployments are
unaffected. BuildArgs reads them.
Tests:
- TestBuildArgs_DefaultMaps locks the empty-string default behavior
- TestBuildArgs_CustomMaps drives the multi-input override path
- TestProcessConfigWebRTCMapsRoundtrip extends the DTO roundtrip
Closes#2.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
TestIntegration_FiveViewerFanout drives the M3 acceptance criterion
in the wide direction: spin up the subsystem, register one process,
attach 5 Pion subscribers in parallel via the real Echo handler,
spray synthetic RTP at the allocated UDP ports, and assert each
subscriber's video + audio track receive at least one packet inside
a 15s window. After onProcessStop, the per-stream peer index must
drain to zero within 3s.
TestSubsystem_TeardownHookFiresOnProcessStop is the unit-level
counterpart — confirms the callback registered via
SetTeardownHook actually fires when a process is torn down, even
without a full Pion handshake.
Together these cover the acceptance language: '5 concurrent viewers,
all error paths correct, clean teardown'.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Covers each new code path that the design's §6 table requires:
- Subscribe -> 406 on non-H264 / non-Opus offer (TestHandler_Subscribe_406OnCodecMismatch)
- Subscribe -> 503 when total cap exhausted (TestHandler_Subscribe_503OnTotalCap)
- Subscribe -> 503 when per-stream cap exhausted (TestHandler_Subscribe_503OnPerStreamCap)
- Trickle -> 404 on unknown resource (TestHandler_Trickle_404WhenUnknown)
- preflight -> 204 + CORS headers (TestHandler_PreflightCORS)
- Register installs all 5 routes (TestHandler_RegisterMountsAllRoutes)
- Close drains the index without panicking (TestHandler_Close_DrainsPeers)
- requireH264AndOpus table-driven (TestRequireH264AndOpus)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Major Handler rewrite implementing the design's M3 acceptance
criteria ('5 concurrent viewers, all error paths correct, clean
teardown'):
Multi-viewer correctness:
- streamID -> resourceID -> Peer two-level index (was flat)
- per-stream peer cap alongside total cap, defaults match the
design's '5–8 viewer' target (8/stream, total from corewebrtc)
- per-peer awaitPeerClose goroutine watches Peer.Done() so ICE
failures yank the index entry + decrement the counter (no leaks)
- tearDownStreamPeers callback (registered with Subsystem in
NewHandler) drives all peer closes when the source process stops
Error matrix from design §6:
- 406 on codec mismatch (offer missing H264 or Opus rtpmap)
- 504 on ICE gathering timeout (passthrough from CreatePeerFromSources)
- 204 on DELETE unknown resource (idempotent per WHEP spec; was 404)
- 503 on per-stream cap reached (separate body from total-cap 503)
- 400 on missing/empty body (unchanged)
- 404 on unknown stream (unchanged)
WHEP spec compatibility:
- PATCH /whep/:id/:resource for trickle-ICE
- OPTIONS preflight on every WHEP path
- CORS Allow-Origin/Methods/Headers + Expose-Headers (Location, ETag)
- ETag header on Subscribe response
Defensive nil-peer guards in tearDown / Close paths so a partial
state doesn't panic.
Refactor: 134 -> 341 lines on handler.go but the surface is the
same (NewHandler/Register/Subscribe/Unsubscribe/Close); existing
callers continue to work. Pre-M3 test 'Unsubscribe_404WhenUnknown'
renamed and updated to the new 204 expectation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Subsystem.SetTeardownHook installs a callback the subsystem invokes
just before closing per-stream Sources in onProcessStop. Used by the
WHEP Handler in M3 to drain its per-stream peer index before the
underlying Sources go away — closes the 'subscribers fan out into a
closed channel' race the design's §6 error matrix calls out as
'Publisher disconnects / FFmpeg exits'.
Single consumer by design (one subsystem, one handler). Calling
SetTeardownHook again replaces the previous callback; nil detaches.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
End-to-end exercise of the M2 pipeline — subsystem hook, port
allocation, two-track forwarding, WHEP handshake — without
spinning up a full Core HTTP server:
- Fire onProcessStart directly to get the two RTP legs back
- Parse video + audio UDP ports out of the leg addresses,
assert adjacency
- Mount the Handler on an Echo httptest server
- Build a Pion PeerConnection (recvonly video + audio), POST
its offer, feed the answer back in
- Spray synthetic RTP packets at both loopback sockets
- Assert both OnTrack callbacks fire and each delivers at least
one RTP packet within 10s
- DELETE via the returned Location header to confirm teardown
Passes cleanly under -race in ~1s. Catches regressions across
the whole M2 wiring from a single fixture.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Installs the WebRTC egress subsystem at Core boot when
cfg.WebRTC.Enable is true and the subsystem constructs cleanly:
- http.Config gains an optional WebRTC *appwebrtc.Handler field;
server.setRoutesV3 mounts its WHEP routes on the JWT-protected
/api/v3 group.
- api.start() constructs the Subsystem, registers its ProcessHooks
with the restreamer, and builds a Handler. A construction failure
is logged and Core continues without WebRTC — consistent with
disabling the subsystem outright.
- api.stop() closes the Handler (tearing down active peers) before
closing the Subsystem (releasing per-process UDP sockets), mirroring
the RTMP/SRT teardown pattern.
Verified: go build ./... clean; go test ./app/webrtc/...
./core/webrtc/... ./restream/... ./http/... all pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Introduces the HTTP surface the browser (or OBS WebRTC clients)
target when subscribing to a process's egress:
POST /whep/:id -> answer SDP + Location header
DELETE /whep/:id/:resource -> tear down a specific peer
The handler looks up the per-process stream pair via the Subsystem,
validates SDP offer shape, and delegates peer creation to the core
PeerFactory's CreatePeerFromSources (two-source forwarding).
WHEP routes are left unauthenticated in M2 — browsers and OBS don't
carry the Core JWT, and per-process signed-URL tokens are an M3
enhancement. Deployments should place the endpoint behind an
authenticated reverse-proxy for now.
Tests cover:
- 404 for POSTs against unregistered streams
- 400 for empty/invalid SDP offers once a stream is registered
- 404 for DELETE against unknown resource ids
Introduces the subsystem layer that sits alongside api.API and wires
the M1 core/webrtc primitives into the per-process restream lifecycle.
app/webrtc/subsystem.go:
- Subsystem struct holding the global WebRTC config, core PeerFactory,
per-process stream map, and logger
- New(config.DataWebRTC, logger) constructor
- Enabled(), Hooks(), Close(), lookup() methods
app/webrtc/lifecycle.go:
- onProcessStart: allocates an adjacent UDP port pair, binds two
Pion Sources (video on V, audio on V+1), registers them under the
process id, and returns the two RTP output legs to append to the
FFmpeg command.
- onProcessStop: tears down the pair.
- allocAdjacentPair: retries up to 10 times to find a free (V, V+1)
pair since the kernel's ephemeral picker can hand us an odd port.
- splitRTPLegs: converts BuildArgs' flat []string into two ConfigIO
entries by splitting on the second -map token.
core/webrtc/peer.go + forward.go:
- Adds PeerFactory.CreatePeerFromSources for the M2 two-source
forwarding mode (video and audio on separate UDP ports, no
payload-type sniffing). Leaves CreatePeer intact for the M1 PoC.
- Adds forwardRTPSplit companion goroutine.
config/data.go:
- Promote anonymous WebRTC struct to named type DataWebRTC so
app/webrtc can accept it by value.
Adds Alloc(), the ephemeral loopback UDP port grabber the subsystem
uses to pick the RTP port it will hand to FFmpeg and then re-bind with
core/webrtc.NewSourceOn. Covered by a 100x rebind test.
Adds BuildArgs(), which emits the -f rtp output fragments (video on
the passed port, audio on port+1) with copy codecs by default and an
H.264 baseline / libopus re-encode leg when ForceTranscode is set.
Covered by three unit tests.
If purging is enabled, overwriting a file with a file of the same
or smaller size will not result in an error.
It is now possible to change the purging mode on an existing sized
filesystem.
1. Allow variables in placeholders for parameter values, e.g.
{rtmp,name=$processid}. The variable starts with a $ letter.
The recognized variables are provided with the Replace func.
2. The template func recieves the process config and the name of
the section where this placeholder is located, i.e. "global",
"input", or "output".
created_at represents the time when the configuration has been persisted to disk.
loaded_at represents the time when the configuration has actually been used.
If created_at is larger than loaded_at, then the Core needs a reload in order
to apply the latest configuration.
if created_at is lower than laoded_at, then the Core applied the latest
configuration.
The value of updated_at is irrelevant and shouldn't be used.
If no path is given in the environment variable CORE_CONFIGFILE, different
standard locations will be probed:
- os.UserConfigDir() + /datarhei-core/config.js
- os.UserHomeDir() + /.config/datarhei-core/config.js
- ./config/config.js
If the config.js doesn't exist in any of these locations, it will be
assumed at ./config/config.js
