From dd7827de8207b3690ff3c565a134ccfe3c5f0451 Mon Sep 17 00:00:00 2001 From: Zac Gaetano Date: Sun, 10 May 2026 09:41:34 -0400 Subject: [PATCH] docs: refresh _NEXT.md after recording + control surface --- docs/superpowers/plans/_NEXT.md | 66 +++++++++++++++++++++++++-------- 1 file changed, 51 insertions(+), 15 deletions(-) diff --git a/docs/superpowers/plans/_NEXT.md b/docs/superpowers/plans/_NEXT.md index 4a800a6..ac6b802 100644 --- a/docs/superpowers/plans/_NEXT.md +++ b/docs/superpowers/plans/_NEXT.md @@ -33,13 +33,55 @@ - Hide-(Local) toggle so the user's own self-preview is filtered from the participants list. - Window position / size / state persisted to `%LOCALAPPDATA%\TeamsISO\window.json`, multi-monitor safe. - Tooltips on every interactive control in the settings panel + per-row textbox + ISO toggle. +- Toast feedback for settings actions (Apply / Apply Transcoder Topology / Stop All / Auto-disable). +- **Auto-disable on participant departure** (configurable, off by default): when a participant's NDI source disappears the engine tears down their pipeline; the toggle lives in `DISPLAY` settings. +- **Operator presets**: chromeless `Presets…` dialog from the participants header. Saves the current per-participant `IsEnabled` + `CustomName` set keyed by display name to `%LOCALAPPDATA%\TeamsISO\presets.json` (atomic write, schema-versioned). Apply walks the live participants and reconciles via `EnableIsoAsync` / `DisableIsoAsync`; participants in the preset who aren't in the current meeting are reported in the toast. +- **Auto-apply last preset on launch**: opt-in checkbox in `DISPLAY` settings. After the operator's first manual Apply, every subsequent TeamsISO launch silently re-applies the same preset once participants populate (30-second grace window before applying with whoever's online). State lives in `presets.json` next to the preset list. +- **Refresh discovery** affordance: header pill that rebuilds the underlying NDI finder on the next poll tick. `IIsoController.RefreshDiscovery` flips a flag the discovery loop honors before the next tick — old finder disposed, new finder created, seen-set cleared so all currently-visible sources re-fire as Added. `ParticipantTracker.HandleAdded` is idempotent: re-emitting the same FullName refreshes LastSeen rather than minting a duplicate row. +- **Settings tabs**: the settings sidebar is now a TabControl with `OUTPUT` / `NETWORK` / `DISPLAY` tabs and a single Apply Changes button below. Underline-on-active tab style lives in `WildDragonTheme.xaml` (`Wd.TabControl` + `Wd.TabItem`). +- **Crash diagnostics**: `App.OnStartup` wires `AppDomain.UnhandledException`, `Application.DispatcherUnhandledException`, and `TaskScheduler.UnobservedTaskException` into a unified Serilog.Critical log line + user-facing dialog that points at the log directory. Dispatcher exceptions are marked `Handled = true` so a single bad UI thunk doesn't take the app down; AppDomain crashes are terminal but at least the user gets the log path before exit. +- **First-launch onboarding**: chromeless welcome dialog walks users through the once-per-machine setup (NDI runtime, Teams admin permission, transcoder topology, presets, log location). Suppressed after dismissal via marker file at `%LOCALAPPDATA%\TeamsISO\onboarding.flag`. Re-openable from the About dialog via "Show welcome" button. +- **Reset output to defaults**: ghost button at the bottom of the OUTPUT settings tab restores framerate / resolution / aspect / audio to `FrameProcessingSettings.Default` after confirmation. Doesn't touch NDI groups (sticky per-machine) or display toggles. +- **Per-output recording**: `IRecorderSink` interface + `RawBgraRecorderSink` implementation. When the operator enables "Record ISOs to disk" in the DISPLAY tab, each newly-enabled ISO writes its normalized output to `//video.bgra` plus a sidecar `manifest.json` (width / height / fps / frame counts) and a `convert.cmd` one-liner that pipes the raw stream into FFmpeg to produce a final H.264 `output.mkv`. Recorder runs on its own bounded queue (240-frame `DropOldest` buffer) so disk pressure never blocks the live ISO; recorder failures are caught and ignored at the channel-write layer for the same reason. Already-running ISOs are not retroactively captured — operator disables + re-enables to start recording. Recording can be wired to a real-time H.264 encoder later via Vortice.MediaFoundation; the `IRecorderSink` interface is designed to swap implementations without touching the pipeline. +- **REST control surface**: `ControlSurfaceServer` — `System.Net.HttpListener` on `127.0.0.1:9755` (configurable). Endpoints for participant ISO toggle (by Id or display name), refresh discovery, stop-all, recording on/off, preset apply, and Teams in-call commands (mute / camera / share / leave / raise-hand). Off by default; toggle in the DISPLAY tab. Bitfocus Companion / Stream Deck plugins / OSC bridges drive it. Documented at `docs/CONTROL-SURFACE.md`. +- **PresetApplier**: extracted from `PresetsDialog.OnApply`. Single source of truth for "apply this preset to live participants" — used by the dialog, by `MainViewModel.TryAutoApplyPendingPreset` (auto-apply on launch), and by the REST `POST /presets/{name}/apply` endpoint. Marshals UI-bound writes (CustomName / IsEnabled) through an optional Dispatcher so off-thread callers don't crash WPF. +- **In-app preview thumbnails**: 160×90 WriteableBitmap per participant, fed from the engine's most recent `ProcessedFrame` at the existing 1Hz stats tick. Inline nearest-neighbor scaler in `ParticipantViewModel.UpdateThumbnail` writes directly into the bitmap's pinned BackBuffer (unsafe block, `true` in the .csproj) for ~10× perf vs. going through Span. Falls back to a `—` placeholder card when no pipeline is running. New "Preview" column in the participants DataGrid. +- **WebSocket live state push**: `ws://127.0.0.1:9755/ws` — clients connect, receive a participants snapshot immediately, and get fresh snapshots within 250ms whenever state changes. Snapshot diffing on JSON string keeps the wire quiet during steady-state. Used by Stream Deck / Companion buttons that want to light up when an ISO goes LIVE without polling. +- **OSC bridge over UDP**: `OscBridge` listens on `127.0.0.1:9000` (TouchOSC's default). Same command vocabulary as the REST endpoints — `/teamsiso/iso "Jane" 1`, `/teamsiso/preset "Friday Show"`, `/teamsiso/teams/mute`, etc. Minimal OSC 1.0 parser (int / float / string / T / F type tags; no bundles). TouchOSC layouts and Companion's Generic OSC surface can both drive it directly. +- **Manual update check**: "Check for updates" button in the About dialog. Asks `forge.wilddragon.net/api/v1/repos/zgaetano/teamsiso/releases?limit=1`, compares the newest tag's SemVer to the running version, prompts to open the releases page if newer. Manual only — no background polling for v1 so a long-running show doesn't get interrupted by a surprise installer. +- **Auto-update banner on launch**: opt-in (default on) silent check throttled to once per 24h via `%LOCALAPPDATA%\TeamsISO\last-update-check.txt`. When a newer release is found, a non-modal banner appears above the body with "Get update" / "Dismiss" buttons. Suppression via flag file at `no-update-check.flag` for fleets that prefer central rollout. New `UpdateBannerViewModel` distinct from the engine alert banner. +- **Preset import / export**: Export / Import buttons in the Presets dialog footer, backed by `OperatorPresetStore.ExportAllAsJson` / `ImportBundle`. Bundle format is `teamsiso-presets-bundle/v1` JSON. On name collision the importer asks once (Overwrite/Keep/Cancel) rather than per-preset; deliberately doesn't include the operator's `LastAppliedName` / `AutoApplyOnStartup` since those are machine-local. +- **Recording markers**: `IRecorderSink.AddMarker(label)` plus `IIsoController.AddRecordingMarker(label)` fan-out to every active recorder. Surfaced via "Marker" button in the IN-CALL bar (auto-labels with timestamp), `POST /recording/marker` in the REST surface, and `/teamsiso/recording/marker "Label"` in OSC. Markers land in `manifest.json` under `markers[]` with `offsetMs` + `label` fields for post-production chaptering. +- **Custom NDI output name template**: `OutputNameTemplate` static helper persisted to `output-name-template.txt` with `{name}` / `{guid}` / `{machine}` / `{timestamp}` tokens. Default `TEAMSISO_{guid}` preserves the engine's hard-coded behavior; operator can switch to `TEAMSISO_{name}` for human-readable downstream switcher names. UI editor in the NETWORK settings tab. +- **Enriched footer status bar**: rec badge (coral dot + count) when at least one ISO is being recorded; control-surface badge (cyan dot + "REST :9755 + OSC :9000") when those services are running. Computed at the existing 1Hz stats tick from `IIsoController.RecordingEnabled` × running pipeline count and `App.ControlSurface.IsRunning` / `App.OscBridge.IsRunning`. +- **Disk space watcher**: `DiskSpaceWatcher` polls the recording drive every 5s while recording is on. Coral toast at <10GB free; auto-disables recording at <1GB so an unattended long show doesn't crash the host on disk-full. +- **Diagnostic bundle export**: "Export diagnostics" button in About zips logs + config + presets + window state + version metadata into a `teamsiso-diagnostics-.zip` in `~/Downloads`. Excludes screenshots / memory dumps; only files the user already wrote. +- **Per-participant recording opt-out**: new `Rec` column in the DataGrid lets the operator choose which ISOs get recorded when global recording is on. `IIsoController.EnableIsoAsync` gained an optional `bool? recordOverride` parameter — null = follow global flag, true = force on, false = force off. +- **Window-scoped keyboard shortcuts**: F1 (help), Ctrl+M (drop marker), Ctrl+Shift+S (stop all), Ctrl+R (refresh discovery). InputBindings on MainWindow → MainViewModel commands; F1 opens the new `HelpWindow` cheat sheet. +- **Help cheat sheet**: chromeless `HelpWindow` lists keyboard shortcuts, file locations (`%LOCALAPPDATA%\TeamsISO\Logs\`, `%APPDATA%\TeamsISO\config.json`, etc.), and links to the public docs. Reduces support friction. +- **Bulk enable**: header `Enable all` button (green dot) enables ISOs for every online + non-enabled participant. Per-participant best-effort with a count toast. +- **Live participant filter**: textbox above the DataGrid filters by display-name substring as you type. Backed by an `ICollectionView` Filter callback so the underlying `ObservableCollection` isn't mutated (preserving identity-tracking). +- **Right-click context menu** on participant rows: Toggle ISO, toggle Record-this-participant, Copy NDI source name to clipboard. Uses the existing per-row commands so the menu is just another binding surface. +- **CLI: `--apply-preset NAME`**: launch-time flag that auto-applies the named preset once participants populate. Same code path as the persisted auto-apply preference. Useful for `Friday Show.lnk` desktop shortcuts that drive recurring routings. +- **Dynamic status text**: footer's center text now reads "3/5 ISOs live · 2 recording" once routing starts, instead of the static "Engine running at X fps target." Composed in `OnStatsTick` from running participant + recording counts. +- **Embedded HTML control panel** at `GET /ui`: self-contained ~6KB page with WebSocket-driven live state and buttons for the common control actions. Open in a phone or second-monitor browser to drive TeamsISO without context-switching from the show. No external dependencies, no build step. +- **Session timer** in footer: shows `MM:SS` (or `HH:MM:SS` past an hour) elapsed since the first ISO went live this session. Resets when all ISOs go offline. Green dot indicator for at-a-glance status. +- **Show notes service**: `POST /notes` and `/teamsiso/notes "..."` (OSC) append timestamped lines to `%LOCALAPPDATA%\TeamsISO\Notes\.md`. Operators wire a Stream Deck button to drop notes during a live show without leaving the production app. Markdown format renders cleanly in any editor. +- **NotesWindow inline viewer**: chromeless dialog that displays today's notes file with 2s polling so REST/OSC-driven appends surface live. "Notes" button in the IN-CALL bar. +- **Duplicate-preset action**: "Duplicate" footer button in the Presets dialog. Custom inline prompt suggests ` (copy)` / `(copy 2)` / etc. names. +- **CHANGELOG.md**: project-wide changelog following keep-a-changelog format. Captures the full May 2026 batch under `[Unreleased]`. +- **README rewrite**: top-level README now lists what TeamsISO does, build instructions, doc links, keyboard shortcuts table, file-locations table. +- **Confirm-before-Stop-All**: stop-all button now requires Yes confirmation, preventing accidental mid-show clicks. Default-No so Enter cancels. ### Networking automation - One-click **transcoder topology** button in Settings: writes `%APPDATA%\NDI\ndi-config.v1.json` so all local senders broadcast on `teamsiso-input` and local receivers see both `public` + `teamsiso-input`. Engine settings auto-flip to receive-from `teamsiso-input` and emit-on `public`. Atomic write with timestamped backup of the prior config. -### Phase E.1 starter (embedded Teams) -- Spec at `docs/superpowers/specs/2026-05-08-embedded-teams-orchestration.md` (three-phase rollout: launcher → window orchestration → in-app meeting controls). -- Rail "Launch / Stop Teams" toggle: launches via `ms-teams:` URI / `ms-teams.exe` / classic `Update.exe --processStart`, asks to confirm `WM_CLOSE` of all running Teams windows when toggled while Teams is up. +### Phase E — embedded Teams orchestration +Spec at `docs/superpowers/specs/2026-05-08-embedded-teams-orchestration.md`. All three sub-phases shipped in May 2026: +- **E.1 — Launcher.** Rail "Launch / Stop Teams" toggle: launches via `ms-teams:` URI → `ms-teams.exe` → classic `Update.exe --processStart`, asks to confirm `WM_CLOSE` of all running Teams windows when toggled while Teams is up. +- **E.2 — Window orchestration.** Rail eye-icon button hides every visible top-level Teams window via `EnumWindows` + `ShowWindow(SW_HIDE)`. Click again to restore + foreground. Lets the operator drive Teams from TeamsISO without ever seeing the Teams UI. +- **E.3 — In-call controls.** UIAutomation-driven Mute / Camera / Share / Leave buttons in a new `IN-CALL` card at the top of the participants area. `TeamsControlBridge` walks Teams' automation tree by candidate Name list (`Mute`, `Unmute`, `Microphone`, `Toggle mute` …) and tries Invoke or Toggle pattern. Tolerant lookup: when a Teams update renames a button we extend the candidate list, no crash. Toasts reflect the four outcomes (Invoked / TeamsNotRunning / ControlNotFound / InvokeFailed). Bridge also exposes (UI-not-yet-wired) `ToggleRaiseHand`, `ToggleChat`, `OpenBackgroundEffects`. Candidate name lists localized for English/German/Spanish/French/Portuguese/Japanese — all locales matched in a single pass; the first match wins. +- **PostMessage shortcut forwarding fallback.** `TeamsLauncher.SendShortcut(modifiers, vk)` posts WM_KEYDOWN/UP to the most-recently-used hidden Teams HWND. Best-effort — modern WebView2-hosted Teams sometimes ignores synthesized key messages at the app-shortcut layer; UIA is preferred when a button exists for the action. ### Diagnostics - `TeamsISO.Console --list-sources` enumerates raw NDI source names visible to the local finder for ~5 seconds; debugging tool for setup issues. @@ -49,25 +91,19 @@ ### CI / Release / Docs - Forgejo CI is green: `actions/upload-artifact@v3` (Forgejo doesn't support v4 yet). - `.forgejo/workflows/release.yml`: tag-push (`v*.*.*`) builds + tests + publishes + builds the MSI on a Windows runner and attaches it to the auto-created Forgejo release via the REST API. -- `docs/RELEASING.md` walks through cutting a release and flags the code-signing TODO. +- **Optional code-signing** wired into `release.yml`: when `SIGN_CERT_PFX_BASE64` + `SIGN_CERT_PASSWORD` Forgejo secrets are set, the workflow signs both `TeamsISO.exe` (before MSI build) and the MSI (after) with SHA-256 + RFC 3161 timestamp. Skipped silently when the cert isn't configured. `docs/RELEASING.md` documents the OV vs EV trade-offs and the Azure Trusted Signing migration path. ### Tests - 78 unit tests passing; 9 NDI integration tests gated behind `--filter requires=ndi` (runtime probe, finder + sender lifecycle on default and custom groups, loopback discovery, full pipeline frame round-trip asserting 1080p normalization). ## Next -1. **Phase E — Embedded Teams orchestration (continued).** E.2 (window orchestration: hide Teams' main window once launched, forward keyboard shortcuts via SendInput) and E.3 (Microsoft Graph or UIAutomation in-call controls) per the spec at `docs/superpowers/specs/2026-05-08-embedded-teams-orchestration.md`. +1. **Smoke-test on real Teams.** Most of May's work hasn't run against a live meeting yet: the UIA in-call commands (mute / camera / share / leave) need their candidate-Name lists validated against the current Teams build, and the auto-apply-on-launch flow needs a real recurring meeting to confirm the 30-second grace window is right. Pin the AutomationIds for buttons we find — Name-based lookup is a starting point, AutomationId is what survives Teams UI updates. -2. **Code-signing the MSI.** `installer/TeamsISO.Installer.wixproj` has a `SignOutput` property hook but no cert; for v1.0 wire a `signtool` invocation from a CI secret. SmartScreen will warn on first launch until that lands. +2. **Acquire a code-signing cert.** Pipeline is wired (see "CI / Release / Docs" above); just needs `SIGN_CERT_PFX_BASE64` + `SIGN_CERT_PASSWORD` set in Forgejo Secrets. OV cert (~$200/yr) gets us signed but SmartScreen builds reputation slowly; EV cert (~$300/yr, hardware token) is SmartScreen-trusted immediately. Azure Trusted Signing is the cloud-native path if a token-on-runner is fiddly. -3. **Toast / inline feedback** for "Apply Changes" and other settings actions (currently silent unless an error occurs). +3. **Activate `MediaFoundationRecorderSink`.** Scaffold + activation docs at `docs/REAL-TIME-RECORDING.md` ship in this batch (gated behind `MF_AVAILABLE` build symbol). To enable: `dotnet add Vortice.MediaFoundation`, define `MF_AVAILABLE`, swap one line in `IsoController.EnableIsoAsync`. Rough ~10× disk-pressure reduction. -4. **Rail "Refresh sources"** affordance — force a discovery rescan, useful right after applying a new transcoder topology when the operator wants Teams to reconnect. +4. **Wire engine audio capture.** The UI's audio level VU bar (in the Live column) is in place but inert — `IsoHealthStats.PeakAudioLevel` always reads 0.0. Engine work needed: extend `INdiInterop.CaptureFrame` to also surface audio frames, parse based on FourCC (FLTp, PCMs16, etc), compute peak per pipeline tick, publish through `IsoPipeline.GetStats`. Once that's done the UI bar starts animating with no further changes. -5. **Output thumbnail previews** in the participant DataGrid — small live-frame previews of each ISO output. Complex (needs WPF WriteableBitmap from the Pipeline's last ProcessedFrame); deferred until v1.5. - -6. **Settings panel UX tightening.** It's getting long; consider an Accordion or tabs for OUTPUT FORMAT / NDI NETWORK / DISPLAY rather than one scrolling stack. - -7. **Auto-disable on participant departure (configurable).** Today an ISO stays "enabled" if its participant leaves; the source goes null. Optional toggle: tear down the pipeline automatically when a participant has been gone past the rename window. - -8. **Operator presets.** "Save current ISO assignments to a named preset" + "Load preset on next launch" so an operator with a recurring show doesn't have to re-toggle every meeting. +5. **Forward Teams keyboard shortcuts via SendInput.** Phase E.2 hides the Teams window but doesn't forward Ctrl+Shift+M / Ctrl+Shift+O / Ctrl+Shift+H to it. UIA covers mute/camera/share/leave/raise-hand/chat/background already; SendInput would let us pass arbitrary global hotkeys through to a hidden Teams for actions UIA can't reach. Lower priority now that UIA covers the core actions.