Zac Gaetano
8e08d7dc6a
Some checks failed
CI / build-and-test (push) Failing after 32s
Added Vortice.MediaFoundation 3.6.2 NuGet package to TeamsISO.Engine so the scaffold compiles when MF_AVAILABLE is defined. However: the scaffold (May 9) was written against an older Vortice surface and the 3.6.2 API has materially changed: - MFVersion not on MediaFactory, MF_LOW_LATENCY moved - IMFAttributes.SetUINT32 replaced with generic Set - IMFMediaType.MajorType / SubType / AvgBitrate property setters → SetGUID(MFAttributeKeys.MajorType, ...) etc. - VideoFormatGuids.RGB32 renamed (likely Rgb32) - IMFMediaBuffer.Lock signature changed (explicit out IntPtr / out int / out int) - IMFSinkWriter.Finalize_ renamed Leaving MF_AVAILABLE undefined for now so the build stays clean. NuGet ref stays so a porter doesn't need to re-add. docs/REAL-TIME-RECORDING.md updated with the deferred status + the specific API gaps to port.
95 lines
4.3 KiB
Markdown
95 lines
4.3 KiB
Markdown
# Real-time H.264 recording
|
||
|
||
The default recorder (`RawBgraRecorderSink`) writes uncompressed BGRA to disk
|
||
and ships a `convert.cmd` for post-recording FFmpeg encoding. That's safe
|
||
(no extra dependencies, works without an encoder installed) but disk-heavy:
|
||
1080p60 = ~500 MB/s, 720p30 = ~88 MB/s.
|
||
|
||
For long shows or operators on slower disks, the engine ships a
|
||
**`MediaFoundationRecorderSink`** that encodes to H.264 in real time using
|
||
Windows Media Foundation. Inline encoding cuts disk pressure ~10× and
|
||
produces a finished `.mp4` without the convert step.
|
||
|
||
It's behind a build flag because activating it requires adding a NuGet
|
||
dependency. The structural code is already in
|
||
`src/TeamsISO.Engine/Pipeline/MediaFoundationRecorderSink.cs`.
|
||
|
||
## Status — May 2026
|
||
|
||
**Activation deferred.** The Vortice.MediaFoundation 3.6.2 NuGet package
|
||
is referenced from `TeamsISO.Engine.csproj`, but the `MF_AVAILABLE` symbol
|
||
is *not* defined. The scaffold in
|
||
`src/TeamsISO.Engine/Pipeline/MediaFoundationRecorderSink.cs` was written
|
||
against an older Vortice API and needs a port pass before activation:
|
||
|
||
- `MFVersion` → not on `MediaFactory` in 3.6.2; pass the SDK version
|
||
directly to `MFStartup`.
|
||
- `MediaFactory.MF_LOW_LATENCY` → relocated to a different attribute
|
||
constants class.
|
||
- `IMFAttributes.SetUINT32` → replaced with a generic `Set` overload.
|
||
- `IMFMediaType.MajorType` / `.SubType` / `.AvgBitrate` properties
|
||
→ now use `SetGUID(MFAttributeKeys.MajorType, ...)` etc.
|
||
- `VideoFormatGuids.RGB32` → renamed (likely `Rgb32`).
|
||
- `IMFMediaBuffer.Lock(out IntPtr, out int, out int)` → explicit out-param
|
||
signature, no longer returns a locked-buffer wrapper.
|
||
- `IMFSinkWriter.Finalize_` → renamed (likely `Finalize`).
|
||
|
||
Until the port lands, the `RawBgraRecorderSink` is the only IRecorderSink
|
||
production uses. The raw recorder is reliable and FFmpeg post-processing
|
||
via the emitted `convert.cmd` produces equivalent .mp4s; you just pay the
|
||
disk pressure during the show.
|
||
|
||
## Activating it (after the port)
|
||
|
||
1. **Update the scaffold** to match Vortice 3.6.2's API surface. A clean
|
||
reference implementation lives in the Vortice samples repo under
|
||
`samples/MediaFoundationSamples`.
|
||
|
||
2. **Define the `MF_AVAILABLE` build symbol** in `TeamsISO.Engine.csproj`:
|
||
|
||
```xml
|
||
<PropertyGroup>
|
||
<DefineConstants>$(DefineConstants);MF_AVAILABLE</DefineConstants>
|
||
</PropertyGroup>
|
||
```
|
||
|
||
3. **Swap the recorder factory** in `IsoController.EnableIsoAsync`:
|
||
|
||
```csharp
|
||
// Old:
|
||
recorder = new RawBgraRecorderSink(_loggerFactory.CreateLogger<RawBgraRecorderSink>());
|
||
// New:
|
||
recorder = new MediaFoundationRecorderSink(_loggerFactory.CreateLogger<MediaFoundationRecorderSink>());
|
||
```
|
||
|
||
Both classes implement `IRecorderSink` so the rest of the pipeline is
|
||
unchanged.
|
||
|
||
4. **Build and smoke-test.** Existing unit tests don't touch the recorder;
|
||
the integration tier covers it once you've enabled MF.
|
||
|
||
## What the MF recorder produces
|
||
|
||
For each enabled ISO with recording on:
|
||
- `<recordings>/<participant>/output.mp4` — H.264 video at the engine's
|
||
configured resolution / framerate, target bitrate ~0.07 bits/pixel
|
||
(~7 Mbps for 1080p30, ~3 Mbps for 720p30).
|
||
- `<recordings>/<participant>/markers.txt` — tab-separated marker offsets
|
||
from `IIsoController.AddRecordingMarker`. Manually chapter the .mp4 with
|
||
`mp4chaps -c -i markers.txt output.mp4` (mp4chaps from the `mp4v2` tools).
|
||
|
||
## Trade-offs vs. RawBgraRecorderSink
|
||
|
||
| | Raw BGRA | Media Foundation H.264 |
|
||
| --------------------- | --------------- | ---------------------- |
|
||
| Dependencies | None | Vortice.MediaFoundation NuGet |
|
||
| Disk @ 1080p60 | ~500 MB/s | ~50 MB/s |
|
||
| Disk @ 720p30 | ~88 MB/s | ~9 MB/s |
|
||
| CPU | Negligible | Moderate (inline encode) |
|
||
| Output | `.bgra` + `convert.cmd` for FFmpeg post-pass | Finished `.mp4` |
|
||
| Markers in container | No (sidecar JSON) | Sidecar `.txt`, chapter via mp4chaps |
|
||
| Reliable on legacy GPUs | Yes | Yes (MF falls back to software encoder if no hw H.264) |
|
||
|
||
If your target machines have NVIDIA NVENC / Intel QuickSync, MF will use
|
||
the hardware encoder transparently — that's the path that gives you
|
||
multi-stream realtime H.264 with low CPU.
|