EtienneLescot
124 lines
4.7 KiB
Markdown
124 lines
4.7 KiB
Markdown
# Windows native cursor test pipeline
|
|
|
|
This branch includes two Windows-focused diagnostics for fast iteration on native cursor capture and rendering. They are intentionally local developer tools: they create short videos and JSON reports so cursor changes can be inspected without doing a full manual record/edit/export cycle.
|
|
|
|
## Native sampler diagnostic
|
|
|
|
```powershell
|
|
npm run test:cursor-native:win
|
|
```
|
|
|
|
This script does not launch OpenScreen. It:
|
|
|
|
- starts a Windows `GetCursorInfo` sampler
|
|
- moves the real OS pointer with `SetCursorPos`
|
|
- captures native cursor handles, hotspots, assets, and standard `IDC_*` cursor types
|
|
- writes normalized `CursorRecordingData`
|
|
- generates an abstract preview video
|
|
- generates a real-screen preview video using screenshots of the current desktop
|
|
|
|
The output directory is printed in the command result, for example:
|
|
|
|
```text
|
|
C:\Users\<user>\AppData\Local\Temp\openscreen-cursor-native-...
|
|
```
|
|
|
|
Useful files:
|
|
|
|
- `report.json`: sample counts, asset counts, cursor handles, and generated artifact paths
|
|
- `cursor-recording-data.json`: sidecar-compatible cursor data
|
|
- `preview.webm`: abstract path/asset/hotspot preview
|
|
- `real-capture-preview.webm`: real desktop screenshot background with reconstructed cursor overlay
|
|
- `assets/*.png`: raw cursor bitmaps captured from Windows
|
|
|
|
Environment overrides:
|
|
|
|
```powershell
|
|
$env:CURSOR_TEST_DURATION_MS = "3000"
|
|
$env:CURSOR_TEST_SAMPLE_INTERVAL_MS = "16"
|
|
$env:CURSOR_TEST_SCREEN_FRAME_INTERVAL_MS = "80"
|
|
$env:CURSOR_TEST_OUTPUT_DIR = "C:\temp\openscreen-cursor-test"
|
|
npm run test:cursor-native:win
|
|
```
|
|
|
|
## OpenScreen preview capture
|
|
|
|
```powershell
|
|
npm run capture:openscreen-preview
|
|
```
|
|
|
|
This script launches the real Electron app, injects a fixture video plus cursor sidecar data, opens the editor, captures frames from the actual OpenScreen preview UI, and encodes them into a WebM.
|
|
|
|
By default it uses the latest `cursor-recording-data.json` generated by `npm run test:cursor-native:win`. To force a specific sidecar:
|
|
|
|
```powershell
|
|
$env:CURSOR_RECORDING_DATA_PATH = "C:\path\to\cursor-recording-data.json"
|
|
npm run capture:openscreen-preview
|
|
```
|
|
|
|
Useful environment overrides:
|
|
|
|
```powershell
|
|
$env:OPENSCREEN_PREVIEW_SKIP_BUILD = "true"
|
|
$env:OPENSCREEN_PREVIEW_FRAME_COUNT = "120"
|
|
$env:OPENSCREEN_PREVIEW_FPS = "30"
|
|
$env:OPENSCREEN_PREVIEW_OUTPUT_DIR = "C:\temp\openscreen-preview"
|
|
npm run capture:openscreen-preview
|
|
```
|
|
|
|
Useful files:
|
|
|
|
- `openscreen-preview.webm`: video of the real OpenScreen editor preview
|
|
- `frames/*.png`: captured preview frames
|
|
- `report.json`: fixture paths, source sidecar, frame count, and output path
|
|
|
|
## What these tests validate
|
|
|
|
Together, the scripts make it quick to inspect:
|
|
|
|
- whether Windows cursor samples are visible and continuous
|
|
- whether native hotspots stay anchored when scaling to `3x`
|
|
- whether standard Windows cursors are recognized via `IDC_*`
|
|
- whether high-quality SVG cursor replacements follow the native hotspot
|
|
- whether the real OpenScreen preview renders the same cursor behavior as the diagnostic pipeline
|
|
|
|
They are not a full substitute for an end-to-end manual recording pass. Before shipping cursor changes, also test a real capture session and export from the packaged app.
|
|
|
|
## Native Windows capture backend
|
|
|
|
The app now routes Windows recordings through an external WGC helper instead of Electron `getDisplayMedia`. This is meant to remove the coordinate and clock split that made the reconstructed cursor drift in the preview/export path.
|
|
|
|
Current native availability rules:
|
|
|
|
- Windows 10 build 19041 or newer
|
|
- a helper executable is available
|
|
|
|
The helper currently implements display/window video capture, system audio loopback, default microphone capture, Media Foundation webcam capture, and DirectShow fallback for selected virtual cameras such as NVIDIA Broadcast. Webcam frames are composed into the primary MP4 as a bottom-right picture-in-picture overlay, and black webcam warmup frames are ignored until the first visible frame is available.
|
|
|
|
Build OpenScreen's helper locally:
|
|
|
|
```powershell
|
|
npm run build:native:win
|
|
```
|
|
|
|
Smoke-test the helper directly:
|
|
|
|
```powershell
|
|
npm run test:wgc-helper:win
|
|
npm run test:wgc-window:win
|
|
npm run test:wgc-audio:win
|
|
npm run test:wgc-mic:win
|
|
npm run test:wgc-mixed-audio:win
|
|
npm run test:wgc-webcam:win
|
|
```
|
|
|
|
For local diagnostics with another compatible helper, point OpenScreen at that executable:
|
|
|
|
```powershell
|
|
$env:OPENSCREEN_WGC_CAPTURE_EXE = "C:\path\to\wgc-capture.exe"
|
|
npm run build-vite
|
|
npm run dev
|
|
```
|
|
|
|
The helper receives one JSON config argument, emits JSON lifecycle events, prints the legacy `Recording started` marker, accepts `stop` on stdin, and prints `Recording stopped. Output path: <path>`. See `electron/native/README.md` for the exact contract and build output paths.
|