Datarhei - Dragon Fork: fork of datarhei/core with native WebRTC (WHEP) egress for low-latency streaming. Built with Pion.
Zac Gaetano
b7afd0f08a
Some checks failed
ci / vet + build (push) Successful in 9m54s
ci / vet + build (pull_request) Successful in 9m49s
ci / race tests (push) Failing after 8m1s
ci / WebRTC smoke (5-viewer fanout) (push) Successful in 9m45s
ci / WebRTC latency p95 gate (push) Successful in 10m3s
ci / race tests (pull_request) Failing after 7m59s
ci / WebRTC smoke (5-viewer fanout) (pull_request) Successful in 9m45s
ci / WebRTC latency p95 gate (pull_request) Successful in 10m4s
Adds an end-to-end RTP-arrival latency probe that runs as a dedicated CI job and asserts p95 < 50ms. Implementation -------------- A build-tagged test (-tags latency, off by default) sends 1000 synthetic RTP packets at 60Hz into corewebrtc.Source and reads them back via a Pion subscriber's track.ReadRTP(). Each packet's payload starts with the publisher's UnixNano send time; the subscriber diffs against time.Now() at arrival and accumulates p50/p95/p99. This exercises every link of the egress hop: Source UDP read, subscriber fan-out, forwardRTPSplit, Pion's TrackLocalStaticRTP write, DTLS-SRTP encrypt, ICE socket write, decrypt at the subscriber, RTP unmarshal at ReadRTP. Pure server-side; no FFmpeg or codecs involved. Why not glass-to-glass ---------------------- The design's §7 calls for FFmpeg drawtext frame counters + decode- side pixel sampling, p95<300ms RTMP / <200ms SRT. Implementing that in pure Go needs a cgo H.264 decoder or an FFmpeg sidecar pipe — a significantly bigger lift for a marginal regression-detection win (encode/decode latency is roughly fixed by the codec stack and isn't moved by Core code changes). The server-hop measurement captures everything Core code can actually regress. Threshold --------- 50ms p95. Locally observed on a quiet host: p50=110µs, p95=237µs, p99=318µs. The 50ms gate is ~200x headroom — generous enough to absorb CI runner noise without false alarms, tight enough to catch a real slowdown. Race-clean: latencySamples uses a sync.Mutex around the slice append (initial draft had a slice racing with the receive goroutine; vet caught it). Documented in test/TESTING.md and wired to .forgejo/workflows/test.yml as the latency-gate job (depends on lint-and-vet, parallel with test and webrtc-smoke). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|---|---|---|
| .forgejo/workflows | ||
| .github | ||
| app | ||
| cmd/webrtc-poc | ||
| config | ||
| core/webrtc | ||
| deploy | ||
| docs | ||
| encoding/json | ||
| ffmpeg | ||
| glob | ||
| http | ||
| internal | ||
| io | ||
| log | ||
| math/rand | ||
| monitor | ||
| net | ||
| playout | ||
| process | ||
| prometheus | ||
| psutil | ||
| restream | ||
| rtmp | ||
| service | ||
| session | ||
| srt | ||
| test | ||
| update | ||
| vendor | ||
| .dockerignore | ||
| .editorconfig | ||
| .gitignore | ||
| build.sh | ||
| CHANGELOG.md | ||
| Dockerfile | ||
| Dockerfile.bundle | ||
| Dockerfile.test | ||
| go.mod | ||
| go.sum | ||
| LICENSE | ||
| main.go | ||
| Makefile | ||
| mime.types | ||
| NOTES.md | ||
| README.md | ||
| run.sh | ||
| SECURITY.md | ||
Core
The datarhei Core is a process management solution for FFmpeg that offers a range of interfaces for media content, including HTTP, RTMP, SRT, and storage options. It is optimized for use in virtual environments such as Docker. It has been implemented in various contexts, from small-scale applications like Restreamer to large-scale, multi-instance frameworks spanning multiple locations, such as dedicated servers, cloud instances, and single-board computers. The datarhei Core stands out from traditional media servers by emphasizing FFmpeg and its capabilities rather than focusing on media conversion.
Objectives of development
The objectives of development are:
- Unhindered use of FFmpeg processes
- Portability of FFmpeg, including management across development and production environments
- Scalability of FFmpeg-based applications through the ability to offload processes to additional instances
- Streamlining of media product development by focusing on features and design.
What issues have been resolved thus far?
Process management
- Run multiple processes via API
- Unrestricted FFmpeg commands in process configuration.
- Error detection and recovery (e.g., FFmpeg stalls, dumps)
- Referencing for process chaining (pipelines)
- Placeholders for storage, RTMP, and SRT usage (automatic credentials management and URL resolution)
- Logs (access to current stdout/stderr)
- Log history (configurable log history, e.g., for error analysis)
- Resource limitation (max. CPU and MEMORY usage per process)
- Statistics (like FFmpeg progress per input and output, CPU and MEMORY, state, uptime)
- Input verification (like FFprobe)
- Metadata (option to store additional information like a title)
Media delivery
- Configurable file systems (in-memory, disk-mount, S3)
- HTTP/S, RTMP/S, and SRT services, including Let's Encrypt
- Bandwidth and session limiting for HLS/MPEG DASH sessions (protects restreams from congestion)
- Viewer session API and logging
Misc
- HTTP REST and GraphQL API
- Swagger documentation
- Metrics incl. Prometheus support (also detects POSIX and cgroups resources)
- Docker images for fast setup of development environments up to the integration of cloud resources
Docker images
- datarhei/core:latest (AMD64, ARM64, ARMv7)
- datarhei/core:cuda-latest (Nvidia CUDA 11.7.1, AMD64)
- datarhei/core:rpi-latest (Raspberry Pi / OMX/V4L2-M2M, AMD64/ARMv7)
- datarhei/core:vaapi-latest (Intel VAAPI, AMD64)
Quick start
- Run the Docker image
docker run --name core -d \
-e CORE_API_AUTH_USERNAME=admin \
-e CORE_API_AUTH_PASSWORD=secret \
-p 8080:8080 \
-v ${HOME}/core/config:/core/config \
-v ${HOME}/core/data:/core/data \
datarhei/core:latest
-
Open Swagger http://host-ip:8080/api/swagger/index.html
-
Log in with Swagger Authorize > Basic authorization > Username: admin, Password: secret
Documentation
Documentation is available on docs.datarhei.com/core.
License
datarhei/core is licensed under the Apache License 2.0