WildDragonLLC/dragon-iso
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 1 Packages Wiki Activity Actions
dragon-iso/docs/superpowers/plans/_NEXT.md
Zac Gaetano f12cbe7517
Some checks failed
CI / build-and-test (push) Failing after 49s
Details
docs: _NEXT.md captures the full E.4 + autorec + UX-pass batch
2026-05-10 21:29:27 -04:00

25 KiB
Raw Blame History

Plan Backlog

Completed phases

  • Phase A — Engine Foundation (tag: phase-a-complete) — domain model, parsers, participant tracker, frame processor, config, fakes, CI gate.
  • Phase B-1 — Pipeline Orchestration (tag: phase-b-1-complete) — NdiReceiver, NdiSender, ExponentialBackoff, NdiRuntimeProbe, IsoPipeline supervisor, IsoController.
  • Phase B-2 — Real NDI Interop (tag: phase-b-2-complete) — NdiInteropPInvoke against NDI 6 SDK, managed BGRA scaler, TeamsISO.Console headless smoke runner, NdiVersion constants.
  • Phase C — WPF UI (tag: phase-c-complete) — MVVM helpers, ParticipantViewModel, GlobalSettingsViewModel, AlertBannerViewModel, MainViewModel, MainWindow XAML with participants DataGrid + settings sidebar + alert banner, App.xaml DI bootstrap.
  • Hardening + brand pass — May 2026 — see "Done since the May 2026 hand-off" below.
  • Phase D — WiX Installer & Forgejo release — WiX v5 MSI scaffold, ARP icon wired, tag-push release workflow that builds + uploads MSI as a release asset.

Done since the May 2026 hand-off

Engine

  • Forward-slash project paths in TeamsISO.sln so .slnf filters work on Windows MSBuild.
  • NdiNativeLibraryResolver resolves Processing.NDI.Lib.x64.dll via NDI_RUNTIME_DIR_V6 (with V5 / V4 fallbacks), so the engine starts on installs where the NDI dir isn't on PATH.
  • NdiVersion.ExpectedRuntimeVersionPrefix updated to match the shipping NDI 6 banner format (NDI SDK WIN64 …).
  • NdiSourceParser accepts current Teams desktop's MS Teams - <name> brand format (plus legacy Teams and defensive Microsoft Teams); reserved suffixes (Active Speaker, Audio, Audio Mix, Screen Share) are recognized in both legacy and dash-prefixed forms.
  • NDI groups end-to-end (discovery + output): INdiInterop.CreateFinder(string?) and CreateSender(string, string?) populate p_groups; IsoController threads them through from EngineConfig.NdiGroups.
  • ParticipantTracker surfaces NdiSourceKind.ActiveSpeaker as a synthetic routable row named "Active Speaker" with a deterministic v5-GUID Id derived from auto-mix:<machine>.
  • IsoHealthStats wired end-to-end: live receiver/sender/processor refs published from the inner pipeline, frame counters / source resolution / running FPS (30-frame moving window) / drops + duplicates / pipeline state surfaced via IsoController.GetStats.
  • Rolling daily file logging at %LOCALAPPDATA%\TeamsISO\Logs\ via Serilog.Sinks.File.

UI

  • WPF rebuilt around Wild Dragon brand × Microsoft Teams flush layout — left rail with real dragon-mark logo (clickable → About dialog), chromeless title bar with custom min/max/close caption controls, cyan accent.
  • Inter Variable + JetBrains Mono Variable bundled as <Resource> so typography matches wilddragon.net regardless of system fonts.
  • App icon teamsiso.ico (7 sizes) on taskbar / window / About / WiX MSI ARP.
  • Single-instance enforcement via per-user named Mutex with broadcast bring-to-front.
  • Empty-state placeholder when no Teams sources are visible (faded dragon + checklist).
  • Live frame counters in the Source / Live columns (in/out/drops, source resolution, running FPS).
  • Per-pipeline state surfaced in the ISO toggle: ● LIVE (cyan), ● ERROR (coral), ● NO SIGNAL (amber), … (processing).
  • "Stop all ISOs" emergency button at the participants header.
  • Hide-(Local) toggle so the user's own self-preview is filtered from the participants list.
  • Window position / size / state persisted to %LOCALAPPDATA%\TeamsISO\window.json, multi-monitor safe.
  • Tooltips on every interactive control in the settings panel + per-row textbox + ISO toggle.
  • Toast feedback for settings actions (Apply / Apply Transcoder Topology / Stop All / Auto-disable).
  • Auto-disable on participant departure (configurable, off by default): when a participant's NDI source disappears the engine tears down their pipeline; the toggle lives in DISPLAY settings.
  • Operator presets: chromeless Presets… dialog from the participants header. Saves the current per-participant IsEnabled + CustomName set keyed by display name to %LOCALAPPDATA%\TeamsISO\presets.json (atomic write, schema-versioned). Apply walks the live participants and reconciles via EnableIsoAsync / DisableIsoAsync; participants in the preset who aren't in the current meeting are reported in the toast.
  • Auto-apply last preset on launch: opt-in checkbox in DISPLAY settings. After the operator's first manual Apply, every subsequent TeamsISO launch silently re-applies the same preset once participants populate (30-second grace window before applying with whoever's online). State lives in presets.json next to the preset list.
  • Refresh discovery affordance: header pill that rebuilds the underlying NDI finder on the next poll tick. IIsoController.RefreshDiscovery flips a flag the discovery loop honors before the next tick — old finder disposed, new finder created, seen-set cleared so all currently-visible sources re-fire as Added. ParticipantTracker.HandleAdded is idempotent: re-emitting the same FullName refreshes LastSeen rather than minting a duplicate row.
  • Settings tabs: the settings sidebar is now a TabControl with OUTPUT / NETWORK / DISPLAY tabs and a single Apply Changes button below. Underline-on-active tab style lives in WildDragonTheme.xaml (Wd.TabControl + Wd.TabItem).
  • Crash diagnostics: App.OnStartup wires AppDomain.UnhandledException, Application.DispatcherUnhandledException, and TaskScheduler.UnobservedTaskException into a unified Serilog.Critical log line + user-facing dialog that points at the log directory. Dispatcher exceptions are marked Handled = true so a single bad UI thunk doesn't take the app down; AppDomain crashes are terminal but at least the user gets the log path before exit.
  • First-launch onboarding: chromeless welcome dialog walks users through the once-per-machine setup (NDI runtime, Teams admin permission, transcoder topology, presets, log location). Suppressed after dismissal via marker file at %LOCALAPPDATA%\TeamsISO\onboarding.flag. Re-openable from the About dialog via "Show welcome" button.
  • Reset output to defaults: ghost button at the bottom of the OUTPUT settings tab restores framerate / resolution / aspect / audio to FrameProcessingSettings.Default after confirmation. Doesn't touch NDI groups (sticky per-machine) or display toggles.
  • Per-output recording: IRecorderSink interface + RawBgraRecorderSink implementation. When the operator enables "Record ISOs to disk" in the DISPLAY tab, each newly-enabled ISO writes its normalized output to <chosen-dir>/<participant>/video.bgra plus a sidecar manifest.json (width / height / fps / frame counts) and a convert.cmd one-liner that pipes the raw stream into FFmpeg to produce a final H.264 output.mkv. Recorder runs on its own bounded queue (240-frame DropOldest buffer) so disk pressure never blocks the live ISO; recorder failures are caught and ignored at the channel-write layer for the same reason. Already-running ISOs are not retroactively captured — operator disables + re-enables to start recording. Recording can be wired to a real-time H.264 encoder later via Vortice.MediaFoundation; the IRecorderSink interface is designed to swap implementations without touching the pipeline.
  • REST control surface: ControlSurfaceServer — System.Net.HttpListener on 127.0.0.1:9755 (configurable). Endpoints for participant ISO toggle (by Id or display name), refresh discovery, stop-all, recording on/off, preset apply, and Teams in-call commands (mute / camera / share / leave / raise-hand). Off by default; toggle in the DISPLAY tab. Bitfocus Companion / Stream Deck plugins / OSC bridges drive it. Documented at docs/CONTROL-SURFACE.md.
  • PresetApplier: extracted from PresetsDialog.OnApply. Single source of truth for "apply this preset to live participants" — used by the dialog, by MainViewModel.TryAutoApplyPendingPreset (auto-apply on launch), and by the REST POST /presets/{name}/apply endpoint. Marshals UI-bound writes (CustomName / IsEnabled) through an optional Dispatcher so off-thread callers don't crash WPF.
  • In-app preview thumbnails: 160×90 WriteableBitmap per participant, fed from the engine's most recent ProcessedFrame at the existing 1Hz stats tick. Inline nearest-neighbor scaler in ParticipantViewModel.UpdateThumbnail writes directly into the bitmap's pinned BackBuffer (unsafe block, <AllowUnsafeBlocks>true</AllowUnsafeBlocks> in the .csproj) for ~10× perf vs. going through Span. Falls back to a — placeholder card when no pipeline is running. New "Preview" column in the participants DataGrid.
  • WebSocket live state push: ws://127.0.0.1:9755/ws — clients connect, receive a participants snapshot immediately, and get fresh snapshots within 250ms whenever state changes. Snapshot diffing on JSON string keeps the wire quiet during steady-state. Used by Stream Deck / Companion buttons that want to light up when an ISO goes LIVE without polling.
  • OSC bridge over UDP: OscBridge listens on 127.0.0.1:9000 (TouchOSC's default). Same command vocabulary as the REST endpoints — /teamsiso/iso "Jane" 1, /teamsiso/preset "Friday Show", /teamsiso/teams/mute, etc. Minimal OSC 1.0 parser (int / float / string / T / F type tags; no bundles). TouchOSC layouts and Companion's Generic OSC surface can both drive it directly.
  • Manual update check: "Check for updates" button in the About dialog. Asks forge.wilddragon.net/api/v1/repos/zgaetano/teamsiso/releases?limit=1, compares the newest tag's SemVer to the running version, prompts to open the releases page if newer. Manual only — no background polling for v1 so a long-running show doesn't get interrupted by a surprise installer.
  • Auto-update banner on launch: opt-in (default on) silent check throttled to once per 24h via %LOCALAPPDATA%\TeamsISO\last-update-check.txt. When a newer release is found, a non-modal banner appears above the body with "Get update" / "Dismiss" buttons. Suppression via flag file at no-update-check.flag for fleets that prefer central rollout. New UpdateBannerViewModel distinct from the engine alert banner.
  • Preset import / export: Export / Import buttons in the Presets dialog footer, backed by OperatorPresetStore.ExportAllAsJson / ImportBundle. Bundle format is teamsiso-presets-bundle/v1 JSON. On name collision the importer asks once (Overwrite/Keep/Cancel) rather than per-preset; deliberately doesn't include the operator's LastAppliedName / AutoApplyOnStartup since those are machine-local.
  • Recording markers: IRecorderSink.AddMarker(label) plus IIsoController.AddRecordingMarker(label) fan-out to every active recorder. Surfaced via "Marker" button in the IN-CALL bar (auto-labels with timestamp), POST /recording/marker in the REST surface, and /teamsiso/recording/marker "Label" in OSC. Markers land in manifest.json under markers[] with offsetMs + label fields for post-production chaptering.
  • Custom NDI output name template: OutputNameTemplate static helper persisted to output-name-template.txt with {name} / {guid} / {machine} / {timestamp} tokens. Default TEAMSISO_{guid} preserves the engine's hard-coded behavior; operator can switch to TEAMSISO_{name} for human-readable downstream switcher names. UI editor in the NETWORK settings tab.
  • Enriched footer status bar: rec badge (coral dot + count) when at least one ISO is being recorded; control-surface badge (cyan dot + "REST :9755 + OSC :9000") when those services are running. Computed at the existing 1Hz stats tick from IIsoController.RecordingEnabled × running pipeline count and App.ControlSurface.IsRunning / App.OscBridge.IsRunning.
  • Disk space watcher: DiskSpaceWatcher polls the recording drive every 5s while recording is on. Coral toast at <10GB free; auto-disables recording at <1GB so an unattended long show doesn't crash the host on disk-full.
  • Diagnostic bundle export: "Export diagnostics" button in About zips logs + config + presets + window state + version metadata into a teamsiso-diagnostics-<ts>.zip in ~/Downloads. Excludes screenshots / memory dumps; only files the user already wrote.
  • Per-participant recording opt-out: new Rec column in the DataGrid lets the operator choose which ISOs get recorded when global recording is on. IIsoController.EnableIsoAsync gained an optional bool? recordOverride parameter — null = follow global flag, true = force on, false = force off.
  • Window-scoped keyboard shortcuts: F1 (help), Ctrl+M (drop marker), Ctrl+Shift+S (stop all), Ctrl+R (refresh discovery). InputBindings on MainWindow → MainViewModel commands; F1 opens the new HelpWindow cheat sheet.
  • Help cheat sheet: chromeless HelpWindow lists keyboard shortcuts, file locations (%LOCALAPPDATA%\TeamsISO\Logs\, %APPDATA%\TeamsISO\config.json, etc.), and links to the public docs. Reduces support friction.
  • Bulk enable: header Enable all button (green dot) enables ISOs for every online + non-enabled participant. Per-participant best-effort with a count toast.
  • Live participant filter: textbox above the DataGrid filters by display-name substring as you type. Backed by an ICollectionView Filter callback so the underlying ObservableCollection isn't mutated (preserving identity-tracking).
  • Right-click context menu on participant rows: Toggle ISO, toggle Record-this-participant, Copy NDI source name to clipboard. Uses the existing per-row commands so the menu is just another binding surface.
  • CLI: --apply-preset NAME: launch-time flag that auto-applies the named preset once participants populate. Same code path as the persisted auto-apply preference. Useful for Friday Show.lnk desktop shortcuts that drive recurring routings.
  • Dynamic status text: footer's center text now reads "3/5 ISOs live · 2 recording" once routing starts, instead of the static "Engine running at X fps target." Composed in OnStatsTick from running participant + recording counts.
  • Embedded HTML control panel at GET /ui: self-contained ~6KB page with WebSocket-driven live state and buttons for the common control actions. Open in a phone or second-monitor browser to drive TeamsISO without context-switching from the show. No external dependencies, no build step.
  • Session timer in footer: shows MM:SS (or HH:MM:SS past an hour) elapsed since the first ISO went live this session. Resets when all ISOs go offline. Green dot indicator for at-a-glance status.
  • Show notes service: POST /notes and /teamsiso/notes "..." (OSC) append timestamped lines to %LOCALAPPDATA%\TeamsISO\Notes\<YYYY-MM-DD>.md. Operators wire a Stream Deck button to drop notes during a live show without leaving the production app. Markdown format renders cleanly in any editor.
  • NotesWindow inline viewer: chromeless dialog that displays today's notes file with 2s polling so REST/OSC-driven appends surface live. "Notes" button in the IN-CALL bar.
  • Duplicate-preset action: "Duplicate" footer button in the Presets dialog. Custom inline prompt suggests <original> (copy) / (copy 2) / etc. names.
  • CHANGELOG.md: project-wide changelog following keep-a-changelog format. Captures the full May 2026 batch under [Unreleased].
  • README rewrite: top-level README now lists what TeamsISO does, build instructions, doc links, keyboard shortcuts table, file-locations table.
  • Confirm-before-Stop-All: stop-all button now requires Yes confirmation, preventing accidental mid-show clicks. Default-No so Enter cancels.

Networking automation

  • One-click transcoder topology button in Settings: writes %APPDATA%\NDI\ndi-config.v1.json so all local senders broadcast on teamsiso-input and local receivers see both public + teamsiso-input. Engine settings auto-flip to receive-from teamsiso-input and emit-on public. Atomic write with timestamped backup of the prior config.

Phase E — embedded Teams orchestration

Spec at docs/superpowers/specs/2026-05-08-embedded-teams-orchestration.md. All three sub-phases shipped in May 2026:

  • E.1 — Launcher. Rail "Launch / Stop Teams" toggle: launches via ms-teams: URI → ms-teams.exe → classic Update.exe --processStart, asks to confirm WM_CLOSE of all running Teams windows when toggled while Teams is up.
  • E.2 — Window orchestration. Rail eye-icon button hides every visible top-level Teams window via EnumWindows + ShowWindow(SW_HIDE). Click again to restore + foreground. Lets the operator drive Teams from TeamsISO without ever seeing the Teams UI.
  • E.3 — In-call controls. UIAutomation-driven Mute / Camera / Share / Leave buttons in a new IN-CALL card at the top of the participants area. TeamsControlBridge walks Teams' automation tree by candidate Name list (Mute, Unmute, Microphone, Toggle mute …) and tries Invoke or Toggle pattern. Tolerant lookup: when a Teams update renames a button we extend the candidate list, no crash. Toasts reflect the four outcomes (Invoked / TeamsNotRunning / ControlNotFound / InvokeFailed). Bridge also exposes (UI-not-yet-wired) ToggleRaiseHand, ToggleChat, OpenBackgroundEffects. Candidate name lists localized for English/German/Spanish/French/Portuguese/Japanese — all locales matched in a single pass; the first match wins.
  • PostMessage shortcut forwarding fallback. TeamsLauncher.SendShortcut(modifiers, vk) posts WM_KEYDOWN/UP to the most-recently-used hidden Teams HWND. Best-effort — modern WebView2-hosted Teams sometimes ignores synthesized key messages at the app-shortcut layer; UIA is preferred when a button exists for the action.

Diagnostics

  • TeamsISO.Console --list-sources enumerates raw NDI source names visible to the local finder for ~5 seconds; debugging tool for setup issues.
  • TeamsISO.Console --version prints engine version + build SHA + .NET + OS + NDI runtime banner + exit-code legend, for support tickets.
  • About dialog inside the WPF host with the same info.

CI / Release / Docs

  • Forgejo CI is green: actions/upload-artifact@v3 (Forgejo doesn't support v4 yet).
  • .forgejo/workflows/release.yml: tag-push (v*.*.*) builds + tests + publishes + builds the MSI on a Windows runner and attaches it to the auto-created Forgejo release via the REST API.
  • Optional code-signing wired into release.yml: when SIGN_CERT_PFX_BASE64 + SIGN_CERT_PASSWORD Forgejo secrets are set, the workflow signs both TeamsISO.exe (before MSI build) and the MSI (after) with SHA-256 + RFC 3161 timestamp. Skipped silently when the cert isn't configured. docs/RELEASING.md documents the OV vs EV trade-offs and the Azure Trusted Signing migration path.

Tests

  • 78 unit tests passing; 9 NDI integration tests gated behind --filter requires=ndi (runtime probe, finder + sender lifecycle on default and custom groups, loopback discovery, full pipeline frame round-trip asserting 1080p normalization).

Done since May 10 hand-off

Engine

  • Audio peak metering wired end-to-end. IsoHealthStats.PeakAudioLevel now reports real values from a sibling NDI audio capture loop in NdiReceiver. New INdiInterop.CaptureAudioPeak method (default- implemented for FakeNdiInterop, overridden in NdiInteropPInvoke). AudioPeakComputer handles FLTP / FLT / PCM s16 with 14 unit tests covering edge cases. UI VU bars in the participants DataGrid now animate; the existing decay logic in ParticipantViewModel was already in place waiting for real values.

Control surface

  • LAN-reachable mode. New checkbox in DISPLAY tab toggles whether the REST/WebSocket surface and OSC bridge bind to 127.0.0.1 only or to all interfaces (http://+:port/, IPAddress.Any). Settings panel surfaces the routable URL with a Copy button (picker prefers physical NICs and skips Tailscale / VPN tunnels / APIPA addresses). Use case: headless host PC + thin client on the same LAN — operator runs Teams + TeamsISO on a quiet machine, drives it from anywhere on the production network. No auth — documented as a trusted-LAN-only mode. First-time bind requires a one-shot netsh http add urlacl; the diagnostic warning fires the exact remediation command if the bind fails.

"I only see TeamsISO" — Phase E.1+E.2 follow-ups

  • Launch + auto-hide Teams preferences in DISPLAY tab. Teams runs in the background; window appears briefly then hides automatically; operator drives everything from the IN-CALL bar + participants DataGrid.
  • Quick-join from URL in the IN-CALL bar. Paste a Teams meeting link, click Join, Teams launches into the meeting. Eliminates the open-Teams → Calendar → find → click join dance.
  • Teams meeting state pill — IN CALL · <meeting title> / READY / empty. UIA probe at 1Hz for the Leave button; meeting title from Teams' window title with the brand suffix stripped.
  • Launch Teams click semantics — left-click = launch / surface / restore; right-click = stop. Was previously ambushing operators with a stop-Teams dialog when Teams was hidden via the eye-toggle.
  • Auto-record on meeting start preference. Recording auto-flips ON when Teams transitions into a call (UIA Leave button appears) and OFF when the call ends — completes the unattended-show story.
  • MUTED / CAM OFF pills in the IN-CALL bar via UIA — local-user state visible at a glance without restoring Teams.
  • Phase E.4 (experimental) — Teams window embedding via SetParent. Reparents Teams' main window into a TeamsISO-owned host so Teams appears visually INSIDE TeamsISO. WebView2 in modern Teams may render glitches after reparent; if so operator unticks and falls back to auto-hide mode. Live in TeamsEmbedWindow + TeamsLauncher.EmbedTeamsInto / RestoreEmbed.
  • Loudest sort mode + active speaker row highlight (3px cyan left border) — operators react to who's talking without scanning every VU bar.
  • NumPad 1-9 hotkeys toggle Nth visible participant's ISO. Generic RelayCommand<T> added so XAML CommandParameter strings convert cleanly.
  • Snapshot frame to PNG (per-participant via right-click + bulk header action). Saves under %USERPROFILE%\Pictures\TeamsISO\.
  • Recording drive free space in the footer (· 245 GB free). Coral tint below 10GB; existing DiskSpaceWatcher still auto-disables at 1GB.
  • Recording elapsed duration in the footer next to the count (REC 3 · 12:45).
  • Quick-join meeting URL + IN-CALL pill with meeting title for the headless workflow — paste link, click Join, see what meeting you're in.

UI polish

  • Visible hover affordances on every themed button (Ghost / Caption / RailIcon / IsoToggle / Primary). Cyan accent borders + brighter fills so mouse-hover and tab-focus give an unmistakable affordance regardless of which dark surface the button sits on.
  • Keyboard focus rings (IsKeyboardFocused triggers) so tab-cycling through the UI gives visual feedback (was nothing — FocusVisualStyle was x:Null with no replacement).
  • ScrollBar restyled to slim transparent track + tinted thumb (Edge / VS Code pattern) in place of the chunky Win9x default.
  • ContextMenu / MenuItem styled to match the dark canvas — right-click on a participant row no longer shows the cream-colored Notepad popup.
  • ToolTip restyled: SurfaceElevated card with rounded corner + 320px text wrap, replacing the cream Win98 popup.
  • Wd.Button.Primary disabled state distinct (was identical to enabled).

Next

  1. Smoke-test on real Teams. Most of May's work hasn't run against a live meeting yet: the UIA in-call commands (mute / camera / share / leave) need their candidate-Name lists validated against the current Teams build, and the auto-apply-on-launch flow needs a real recurring meeting to confirm the 30-second grace window is right. Pin the AutomationIds for buttons we find — Name-based lookup is a starting point, AutomationId is what survives Teams UI updates. Now also includes: validate the audio peak metering against real Teams audio (check that FLTP decoding is correct for whatever sample rate Teams is broadcasting; the --filter requires=ndi integration tests don't exercise audio).

  2. Acquire a code-signing cert. Pipeline is wired (see "CI / Release / Docs" above); just needs SIGN_CERT_PFX_BASE64 + SIGN_CERT_PASSWORD set in Forgejo Secrets. OV cert ($200/yr) gets us signed but SmartScreen builds reputation slowly; EV cert ($300/yr, hardware token) is SmartScreen-trusted immediately. Azure Trusted Signing is the cloud-native path if a token-on-runner is fiddly.

  3. Port MediaFoundationRecorderSink to Vortice 3.6.2 API. NuGet package added but the May 9 scaffold targeted an older Vortice API. Port pass needed before MF_AVAILABLE can be defined; see docs/REAL-TIME-RECORDING.md "Status — May 2026" section for the specific API gaps (MFVersion / MF_LOW_LATENCY / IMFMediaType setters / IMFMediaBuffer.Lock signature / IMFSinkWriter.Finalize_ rename). Once ported, gives ~10× recording disk-pressure reduction.

  4. Forward Teams keyboard shortcuts via SendInput. Phase E.2 hides the Teams window but doesn't forward Ctrl+Shift+M / Ctrl+Shift+O / Ctrl+Shift+H to it. UIA covers mute/camera/share/leave/raise-hand/chat/background already; SendInput would let us pass arbitrary global hotkeys through to a hidden Teams for actions UIA can't reach. Lower priority now that UIA covers the core actions.