diff --git a/docs/engineering/windows-native-recorder-roadmap.md b/docs/engineering/windows-native-recorder-roadmap.md index 29986db..146582a 100644 --- a/docs/engineering/windows-native-recorder-roadmap.md +++ b/docs/engineering/windows-native-recorder-roadmap.md @@ -206,6 +206,39 @@ Acceptance: - `npm run test:wgc-mic:win`: validates default-microphone capture writes an AAC track when an input endpoint is available. - `npm run test:wgc-mixed-audio:win`: validates system loopback plus microphone writes one mixed AAC track when endpoint formats are compatible. +## Backlog + +### Native Cursor Click Bounce Is Not Visibly Applied + +Status: open. Do not treat Windows native cursor `Click Bounce` as shipped. + +Problem: + +- The cursor settings UI exposes `Size`, `Smoothing`, `Motion Blur`, and `Click Bounce`. +- On Windows native cursor recordings, `Size`, `Smoothing`, and `Motion Blur` are visibly applied in preview/export. +- `Click Bounce` still has no visible effect in manual packaged-app testing, even after adding click-related sample metadata. + +What has already been tried: + +- Added `interactionType: "click" | "mouseup" | "move"` to native cursor samples. +- Added polling-based left-button state through `GetAsyncKeyState`. +- Added the `GetAsyncKeyState` low-bit path to catch quick clicks between samples. +- Added a PowerShell/C# `WH_MOUSE_LL` mouse hook experiment and launched the sampler through a temporary `.ps1` file to avoid Windows command-line length limits. +- Updated `npm run test:cursor-native:win` so the diagnostic can observe a synthetic short click and emit `clickSampleCount`. + +Current diagnosis: + +- The diagnostic can observe synthetic click events, but this has not translated into a visible `Click Bounce` effect in the real packaged app. +- The test currently proves that some click metadata can be recorded, not that the full OpenScreen record -> preview -> export path displays a bounce at the expected time. +- The current native implementation may be animating from metadata that is not present in the real recording session, may be using the wrong timestamp origin, or may be applying a scale change too subtle to notice on the DOM/native cursor path. + +Next investigation when resumed: + +- Inspect the actual `.cursor.json`/session sidecar generated by a packaged-app manual recording and confirm whether real clicks produce `interactionType: "click"` at the right `timeMs`. +- Add a targeted end-to-end fixture that records a known click, loads the generated project, and asserts the preview/export cursor scale changes across adjacent frames. +- Compare the native DOM cursor path against the older `PixiCursorOverlay` click visual state and decide whether native cursor bounce should be a scale-only animation, an additional click ring, or a short explicit keyframe animation independent of sample cadence. +- If event capture remains unreliable in the PowerShell sampler, move click events into a small native cursor helper instead of PowerShell/C# script injection. + ## Ship Criteria - Windows display capture works with cursor, system audio, microphone, and webcam. diff --git a/docs/testing/windows-native-cursor.md b/docs/testing/windows-native-cursor.md index 4c7da94..f6d214b 100644 --- a/docs/testing/windows-native-cursor.md +++ b/docs/testing/windows-native-cursor.md @@ -84,6 +84,12 @@ Together, the scripts make it quick to inspect: They are not a full substitute for an end-to-end manual recording pass. Before shipping cursor changes, also test a real capture session and export from the packaged app. +## Known Gap + +Windows native cursor `Click Bounce` is currently backlogged. `Size`, `Smoothing`, and `Motion Blur` can be validated through preview/export, but `Click Bounce` has not shown a visible effect in packaged-app manual testing. The current diagnostic can observe synthetic click metadata, but that is not enough to validate the real OpenScreen record -> preview -> export path. + +Track the open item in `docs/engineering/windows-native-recorder-roadmap.md` under `Native Cursor Click Bounce Is Not Visibly Applied`. + ## Native Windows capture backend The app now routes Windows recordings through an external WGC helper instead of Electron `getDisplayMedia`. This is meant to remove the coordinate and clock split that made the reconstructed cursor drift in the preview/export path.