Commit Graph

195 Commits

Author SHA1 Message Date
Etienne Lescot e9650225ba feat: add cursor overlay pipeline for high-fidelity cursor recording and playback
- Implement native bridge for Windows cursor capture via PowerShell/C#
- Add cursor-free capture using getDisplayMedia with setDisplayMediaRequestHandler
- Update video player and exporters to support native cursor telemetry
- Enable system audio capture on Windows via WASAPI loopback
- Add interpolation for smoother cursor movement in playback and export
- Improve cursor scaling and visibility handling in editor and playback
2026-05-10 15:11:00 +02:00
Etienne Lescot 248ebabcf1 feat: add windows native cursor capture and rendering 2026-05-10 15:10:56 +02:00
Siddharth e3d4a330df ui revamp 2026-05-09 19:18:16 -07:00
Sid 38f2044967 Merge pull request #549 from Ayusman-Singhal/feat/no-webcam-layout-preset
feat: add 'No Webcam' layout preset to hide webcam in final recording
2026-05-09 10:03:45 -07:00
makaradam 37215531c2 feat: add custom zoom slider with continuous scale control (#513)
Adds a Radix UI slider below the zoom preset buttons allowing any scale
between 1.0x and 5.0x. When the slider value matches a preset exactly,
that preset button also shows as active.

- Add `customScale?: number` to `ZoomRegion` and `getZoomScale()` helper
  that returns customScale when set, falling back to ZOOM_DEPTH_SCALES[depth]
- Overlay indicator, playback renderer, and frame exporter all use
  getZoomScale() so preview, playback, and export are consistent
- Fix focus clamping in zoomRegionUtils and frameRenderer to use actual
  scale instead of depth-based preset scale, preventing zoom drift with
  custom values
- Fix drag boundary in VideoPlayback to use clampFocusToScale with the
  actual scale so the full canvas is clickable at high custom zoom levels
- Timeline item label shows custom scale value when set
- Slider styled dark with green thumb/fill when a custom (non-preset) value is active

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 11:23:36 +02:00
Sid 9af318561f Merge pull request #512 from AbhinRustagi/feature/remember-last-export-folder
feat: Add exportFolder to user preferences
2026-05-08 19:30:56 -07:00
Marc Diaz c9980b0dca fix: tests + how to write them 2026-05-07 23:22:32 -04:00
AbhinRustagi 1aac6eddb0 Merge branch 'main' of github.com:siddharthvaddem/openscreen into feature/remember-last-export-folder 2026-05-08 05:29:14 +05:30
AbhinRustagi 25cfd2777f fix: resolve comments 2026-05-08 05:24:40 +05:30
Ayusman Singhal ada1f434f7 feat: add 'No Webcam' layout preset to hide webcam in final recording
Adds a new 'No Webcam' option to the webcam layout preset dropdown in the editor. When selected, the webcam feed is completely hidden from both the preview and the exported video, allowing users who recorded with a webcam to exclude it from the final output.

- Add 'no-webcam' to WebcamLayoutPreset type union and preset map

- Handle 'no-webcam' in computeCompositeLayout (returns webcamRect: null)

- Add 'no-webcam' case in project persistence normalization

- Add 'No Webcam' option to the layout preset dropdown in SettingsPanel

- Add 'noWebcam' i18n translation key (en)
2026-05-07 12:19:48 +05:30
Siddharth 899504f8e2 fix export mouse overlay 2026-05-05 22:02:21 -07:00
Siddharth 190d5d8ecb 3d iso,tilt 2026-05-03 17:54:21 -07:00
Siddharth 8d79a14e3b cursor highlighting and clicks 2026-05-02 23:03:14 -07:00
AbhinRustagi b801c1ccea fix: resolve comments 2026-05-02 01:19:44 +05:30
AbhinRustagi c40727672f feat: implement handlers to store last export location 2026-05-02 01:05:17 +05:30
Sid 884021c7d6 Merge pull request #505 from marcgabe15/fix/decodeEarlyBug
Fix/decode early bug
2026-04-29 21:33:18 -07:00
Marc Diaz 93466fdda1 fix: add max duration 2026-04-29 22:52:15 -04:00
Marc Diaz 786165208f misc: remove misc changes 2026-04-29 22:45:41 -04:00
Marc Diaz 0768c449d7 feat: all changes 2026-04-29 22:36:49 -04:00
Sid a6fe33a0f6 Merge pull request #501 from FabLrc/fix/vp8-vp9-codec-normalization
Fix/vp8 vp9 codec normalization
2026-04-28 19:51:03 -07:00
Sid 608e0abe87 Merge pull request #457 from shaun0927/fix/cursor-telemetry-session-isolation
fix: isolate cursor telemetry samples per recording session
2026-04-28 08:08:08 -07:00
FabLrc f9401f051c fix(exporter): fall back to avc1.640033 for unsupported H.264 codec strings 2026-04-28 14:13:34 +02:00
FabLrc cae71ed49c fix(exporter): add codec normalization for bare avc1/h264 and logging 2026-04-28 14:08:01 +02:00
FabLrc 6577a54418 fix(exporter): normalize bare VP8/VP9 codec strings from web-demuxer 2026-04-28 13:59:10 +02:00
shaun0927 3b9b4192bf fix: key cursor telemetry batches by recordingId for safe discard
discardLatestPending() popped whichever batch happened to be at the
back of the queue. With a Stop → Record → Discard sequence, the
pending queue can have recording B's batch sitting in front of A's by
the time A's finalize callback resolves (because finalizeRecording
awaits fixWebmDuration), so the discard targets the wrong recording.

Tag each completed batch with the recording id supplied at
startSession() time and replace discardLatestPending() with
discardBatch(recordingId). takeNextBatch() now returns the full
{recordingId, samples} shape so prependBatch() can re-queue it on
write-failure without losing the id. The renderer already owns a
stable recordingId (Date.now() in useScreenRecorder) and the IPC
surface threads it through set-recording-state and
discard-cursor-telemetry.

Adds a regression test that mirrors FabLrc's scenario in PR #457:
two recordings finalize, A is discarded after B has already been
queued, and the buffer must drop A while keeping B intact.
2026-04-28 18:27:14 +09:00
Sid 13c982618a Merge pull request #491 from AmitwalaH/feature/video-playback-fix
Fix video playback initialization and zoom sync
2026-04-25 09:24:00 -07:00
Enriquefft e06e40dbc2 clean review nits: typed prefix sentinel, instanceof narrowing, drop dead re-export
- Replace anonymous Error in resolveImageWallpaperUrl with typed
  UnsafeImagePrefixError, mirroring UnsafeAssetPathError so cause
  chains stay discriminable.
- Replace `(err as BackgroundLoadError).cause` casts in wallpaper
  tests with instanceof narrowing (no `as` per project rules).
- Remove unused `WALLPAPER_PATHS` re-export from projectPersistence;
  consumers import directly from @/lib/wallpaper (SSOT).
2026-04-24 22:34:00 -05:00
Enriquefft af159e8a2b tighten legacy normalizer and guard against BackgroundLoadError double-wrap
Reviewer audit found two real risks in the prior amendment:

1. LEGACY_FILE_WALLPAPER_RE was too permissive. Any file:// URL
   containing /wallpapers/wallpaperN.jpg would match — including a user's
   own file at /home/me/wallpapers/wallpaper1.jpg that happened to share
   the name pattern. Silent data-loss potential: user's photo replaced
   with a bundled asset. In-app upload flow uses data: URIs today so it
   can't actually produce such a value, but the regex should be tight
   on intent. Now requires a known install-layout segment:
   resources/[assets/]wallpapers/ (packaged) or public/wallpapers/ (dev).

2. No upper bound on \d+. A corrupted or future-schema project with
   wallpaper99.jpg was silently rewritten to /wallpapers/wallpaper99.jpg
   which 404s. Now validates against WALLPAPER_PATHS; out-of-set
   bundled-looking values fall back to DEFAULT_WALLPAPER.

Also applied R2.2 defensive guard: resolveImageWallpaperUrl's catch
block now checks instanceof BackgroundLoadError and rethrows unchanged
instead of wrapping a second time. Current getAssetPath cannot throw
BackgroundLoadError so this is a future-proof against refactors.

Tests: 56 pass (up from 54). Added coverage for "user file outside
install dir stays untouched" and "bundled-looking but out-of-set falls
back to default".
2026-04-24 18:58:34 -05:00
Enriquefft f2ff7fb21c address review audit: persist canonical wallpaper, dedupe types, tighten edge cases
R1 — Persisted wallpaper is now always the canonical /wallpapers/wallpaperN.jpg
form, never the resolved file:// URL. Swatch clicks pass WALLPAPER_PATHS[i]
(the relative path) to onWallpaperChange; the resolved URL stays in
wallpaperPreviewUrls for rendering only. This prevents machine-specific paths
from being written into project JSON and avoids break-on-upgrade /
break-on-share regressions. Legacy projects carrying resolved file:// URLs are
rewritten by a new normalizer in normalizeProjectEditor:
file://…(/assets)?/wallpapers/wallpaperN.jpg → /wallpapers/wallpaperN.jpg.

R2 — resolveImageWallpaperUrl now catches anything getAssetPath throws
(UnsafeAssetPathError, AssetBaseUnavailableError) and rewraps as
BackgroundLoadError with the original as cause. Callers (videoExporter retry
loop, gifExporter catch, VideoEditor toast) only need one instanceof check and
users always see the translated errors.exportBackgroundLoadFailed toast.

R3 — src/vite-env.d.ts no longer duplicates Window.electronAPI. The interface
had drifted — renderer declaration was missing readBinaryFile, getPlatform,
revealInFolder, getShortcuts, saveShortcuts, hudOverlay*, countdown overlay
methods that electron-env.d.ts already declares. Removed the duplicate and
kept the triple-slash reference so the authoritative declaration is the one
in electron/electron-env.d.ts.

N1 — GRADIENT_RE accepts optional "repeating-" prefix so
repeating-linear/radial/conic-gradient values classify as gradients instead
of falling through to color.

N2 — displayBasename returns "(unknown)" sentinel for URLs without a
meaningful basename (file:///, bare /) instead of leaking the original string.

N3 — electron-builder.json5 extraResources block gets an inline comment
pointing at preload.ts:assetBaseDir so the bidirectional coupling is
discoverable from either file.

Tests: 54 unit tests pass (up from 35). New coverage for repeating
gradients, displayBasename sentinels, BackgroundLoadError cause wrapping,
legacy file:// wallpaper normalization (5 cases).
2026-04-24 18:55:04 -05:00
Enriquefft 702b733074 resolve asset base path synchronously from preload
Every consumer of /wallpapers/*.jpg — SettingsPanel, VideoPlayback,
frameRenderer — was doing async IPC round trips, useEffect dances, and
Promise.all for a value that is a build-time constant per process. Each
consumer showed briefly-empty or briefly-404ing state on first paint
until the handler's reply resolved.

The asset base URL depends only on process.defaultApp and
process.resourcesPath / __dirname — all available in preload at
context-bridge time. Compute once there, expose as a sync string.

- preload.ts resolves baseDir (process.resourcesPath packaged,
  <appRoot>/public unpackaged) and emits assetBaseUrl synchronously.
- get-asset-base-path IPC handler + main-process branching deleted.
- getAssetPath() is now sync. Returns string, not Promise<string>.
  Throws AssetBaseUnavailableError (new) when electronAPI.assetBaseUrl
  is missing — catastrophic preload failure, not silent 404.
- resolveImageWallpaperUrl() sync; same sync throw semantics.
- SettingsPanel: Promise.all + useState + useEffect collapse to one
  useMemo. First paint has real URLs, no 18× ERR_FILE_NOT_FOUND, no
  flicker.
- VideoPlayback: wallpaper-resolve useEffect collapses to useMemo.
- frameRenderer.setupBackground: drops the await.
- electronAPI type decls updated in both .d.ts files.
- 35 unit tests updated to reflect sync signature + new
  AssetBaseUnavailableError contract.

Silent-fallback behavior from getAssetPath (returning /relative when
electronAPI failed) is gone. Renderers now surface preload failures
instead of rendering 404s.
2026-04-24 18:33:03 -05:00
Enriquefft adf3855ac8 harden wallpaper resolver against traversal, PII, and SSOT drift
Adversarial review surfaced four defects and four drive-bys. All applied:

B1 (security, MEDIUM) — Path traversal via encodeRelativeAssetPath.
encodeURIComponent passed "." and ".." through unchanged; percent-encoded
"%2e%2e" got decoded by the URL constructor. Either form escaped the
asset root: new URL("../../etc/passwd", "file:///opt/Openscreen/resources/")
→ file:///opt/etc/passwd. Reject both at src/lib/assetPath.ts via a new
UnsafeAssetPathError thrown when a decoded segment equals "." or "..".

B2 (correctness) — classifyWallpaper returned { kind: "image" } for
conic-gradient(...), rgb(...), hsl(...), oklch(...), empty string,
and named colors like "red". Old frameRenderer's bare fillStyle = value
handled these; new code would throw BackgroundLoadError with misleading
message. Classification now anchors on regexes, accepts all CSS color
functions and all three gradient types, treats unknown strings as
fallthrough color (old behavior), and normalizes "" to "#000000".

B3 (SSOT) — DEFAULT_WALLPAPER, projectPersistence.WALLPAPER_PATHS, and
SettingsPanel.WALLPAPER_RELATIVE independently hardcoded the same
/wallpapers/wallpaperN.jpg pattern. Three drift sites collapse into one:
WALLPAPER_PATHS lives in src/lib/wallpaper.ts, DEFAULT_WALLPAPER derives
from WALLPAPER_PATHS[0], projectPersistence re-exports from the canonical
module, SettingsPanel imports it directly.

B4 (privacy) — BackgroundLoadError.message and the translated toast
surfaced full file paths like file:///home/<user>/…/wallpaper.jpg —
leaks the user's home directory in copy-pasted bug reports. Added a
displayUrl getter that returns just the basename (or "data:…" for data
URIs), wired into the toast. Full URL remains in console.error and
error.url for debugging.

N1 — resolveImageWallpaperUrl now rejects image paths that don't live
under /wallpapers/ (throws BackgroundLoadError). Narrows the blast
radius of the returned <resourcesPath>/ base so the renderer can only
request files within the wallpapers directory, regardless of what the
project JSON claims.

N2 — videoExporter retry loop no longer calls cleanup() twice in the
BackgroundLoadError branch; the finally handles it.

N3 — Browser tests assert BackgroundLoadError.url contains the failing
path. Guards the {{url}} i18n interpolation contract.

N4 — VideoPlayback wallpaper resolve effect now catches resolver
throws (UnsafeAssetPathError, BackgroundLoadError from /wallpapers/
prefix enforcement). Prevents the new strict-rejection logic from
silently leaving the preview without a background.

Tests: 35 unit tests pass (up from 20); new coverage for all color
functions, all gradient types, empty string, named color fallback,
whitespace trimming, /wallpapers/ prefix enforcement, traversal
rejection, percent-encoded traversal rejection, displayUrl basename
and data-URI abbreviation.
2026-04-24 18:16:57 -05:00
Enriquefft d145f80041 fix: wallpaper backgrounds black in exported video (#376)
Three independent defects plus one SSOT violation caused reported symptom
of image wallpapers rendering solid black in exported MP4/GIF while
appearing correctly in the editor preview.

Bug A — Dev-mode IPC handler returned <appPath>/public/assets/, but
wallpapers live at public/wallpapers/. No assets/ subdirectory exists in
source.

Bug B — FrameRenderer.setupBackground bypassed getAssetPath and did
window.location.origin + wallpaper, producing file:///wallpapers/*.jpg
404s in packaged Electron.

Bug C — setupBackground silently caught any background-load error and
filled black. Masked Bug B from the export pipeline; why the bug shipped.

Smell D — Asset layout asymmetric: public/wallpapers/ (dev) vs
resources/assets/wallpapers/ (packaged). assets/ subdirectory had no
other consumers.

Fixes:

- Unify asset layout. electron-builder extraResources now copies to
  resources/wallpapers/ (no assets/). Main handler returns
  <resourcesPath>/ packaged and <appPath>/public/ unpackaged. Same
  convention in both modes: /wallpapers/x.jpg maps to <base>/wallpapers/x.jpg.
  Nix package.nix mirror updated.

- New src/lib/wallpaper.ts module owns the wallpaper contract:
  DEFAULT_WALLPAPER, classifyWallpaper (color/gradient/image), and
  resolveImageWallpaperUrl (pure URL resolver, wraps getAssetPath).
  BackgroundLoadError typed error for short-circuit detection.

- FrameRenderer.setupBackground uses the new helpers. Silent black
  fallback removed; rethrows as BackgroundLoadError. Export pipeline
  (VideoExporter + GifExporter) short-circuits encoder-retry loop on
  BackgroundLoadError. VideoEditor catch site dispatches to translated
  exportBackgroundLoadFailed toast.

- VideoPlayback editor preview consolidated onto the same helpers.
  Three default-wallpaper path literals (useEditorHistory,
  projectPersistence, VideoPlayback) collapsed onto DEFAULT_WALLPAPER.

- i18n: new errors.exportBackgroundLoadFailed key added to all seven
  locales (en, zh-CN, zh-TW, es, fr, tr, ko-KR).

- Tests: 20 unit tests for wallpaper module (classifyWallpaper +
  resolveImageWallpaperUrl branches + BackgroundLoadError).
  videoExporter.browser.test.ts and gifExporter.browser.test.ts extended
  with image-wallpaper happy path and BackgroundLoadError failure path.

Migration note: packaged users upgrading in place may retain an empty
resources/assets/ directory from the prior layout. Unreferenced at
runtime; cosmetic only. DMG/AppImage fresh installs get the new layout
directly.
2026-04-24 17:59:21 -05:00
AmitwalaH 1673daabe4 merge: resolve conflicts and update video playback system 2026-04-24 15:07:05 +05:30
Marc Diaz cffca5f2ff fix: just use one test 2026-04-23 17:37:08 -04:00
Marc Diaz d1087af63c fix: lint 2026-04-23 15:46:35 -04:00
AmitwalaH 9a361a9f2e fix(video-playback): resolve initialization timing issues and ensure smooth zoom & layout rendering 2026-04-23 15:10:59 +05:30
shaun0927 96765e483d docs: correct cx/cy units and sanitize buffer option limits
Two follow-up fixes for CodeRabbit feedback on the docs commit:

- CursorTelemetryPoint JSDoc previously described cx/cy as 'device-pixel
  positions'. The producer sampleCursorPoint() in electron/ipc/handlers.ts
  clamps them to the [0, 1] range after dividing by the source display's
  width/height, so they are normalised ratios, not pixel values. Correct
  the doc comment accordingly.
- createCursorTelemetryBuffer now sanitizes maxActiveSamples and
  maxPendingBatches: non-finite, zero, or negative values fall back to
  safe positive-integer defaults. Without this, a caller passing Infinity
  or NaN would hang the trim loops.

New test covers the sanitisation path for both options.
2026-04-21 18:12:28 +09:00
shaun0927 adc610544c docs: document cursor telemetry buffer API and surface drop events
Add JSDoc to every public export in cursorTelemetryBuffer so the module
meets the 80% docstring-coverage threshold, and make two silent-drop
paths observable:

- endSession() now returns the number of pending batches evicted by the
  maxPendingBatches cap and emits console.warn when any are dropped.
- prependBatch() defensively trims and warns if an unusual retry pattern
  would push the queue past the cap (normal retry after takeNextBatch()
  stays a no-op).

Tests cover both drop paths.
2026-04-21 17:07:19 +09:00
Marc Diaz 95c7b7fc2b fix: add webm inflated duration and fix 2026-04-20 23:11:58 -04:00
Sid 1f99fcb4ad Merge pull request #325 from dheerajmr01/fix/camera-bugs
fix: camera light flashes and turns off when clicking webcam button (…
2026-04-20 08:10:37 -07:00
Trevin Chow dd622f83c1 fix(annotations): use Unicode script properties for CJK detection
Address review feedback on #471 from @coderabbitai. The BMP-only
codepoint ranges missed two classes of characters:

- Non-BMP Han extensions (CJK Unified Ideographs Extension B, C, D, E, F)
  such as 𠀀. A long string of Extension-B characters would still be
  tokenized as a single unbreakable unit and overflow the box.
- Halfwidth Katakana (U+FF65-U+FF9F) such as カ. Same failure mode.

Switch to Unicode script property escapes (\\p{Script=Han},
\\p{Script=Hiragana}, \\p{Script=Katakana}, \\p{Script=Hangul}) which
cover these cases without enumerating ranges. tsconfig target is ES2020;
property escapes require ES2018+ so this is safe.

Verified coverage: 漢 あ ア 가 𠀀 カ all match; A and digits do not.
2026-04-19 10:05:48 -07:00
Trevin Chow f04c2b7c14 fix(annotations): wrap CJK text at character boundaries in export renderer
renderText split each line on whitespace, which works for Latin text
but leaves CJK strings as a single unbreakable token because CJK
scripts have no word-separating whitespace. Result: CJK annotation
text overflows the clipped annotation box even though the editor's
HTML preview wraps it correctly via CSS word-break: break-word.

Replace the ad-hoc whitespace split with a tokenizeForWrap helper
that emits each CJK character (Hiragana, Katakana, Hangul Syllables,
CJK Unified Ideographs + Extension A, and CJK Compatibility
Ideographs) as its own token, while keeping Latin words + whitespace
intact. The existing width-measurement wrap loop then handles CJK
per-character, matching the editor's behavior.

Closes #449
2026-04-19 02:49:17 -07:00
Sid 57c6a590a9 Merge pull request #423 from org-cyber/fix/windows-export-clean
fix(windows): Fixed windows Export Issue and early decode Crash
2026-04-18 10:54:13 -07:00
Sid a20a31f27d Merge branch 'main' into fix/bug-fixes-security-and-reliability 2026-04-18 10:50:05 -07:00
Sid 9ef1f756b4 Merge pull request #448 from theopfr/fix/cpu-readback-only-for-linux
fix: improve performance on windows and macos by passing canvas direclty to `VideoFrame()`
2026-04-18 10:49:09 -07:00
Theodor Peifer 2f24038cb5 fix: use existing getPlatform() so the OS based CPU readback check also works in the browser 2026-04-18 11:31:09 +02:00
Theodor Peifer 934f05cc80 fix: pass platform from video/gifExporter to FrameRenderer, skip readback also for canvas composition for non-linux 2026-04-18 11:31:09 +02:00
Theodor Peifer d12f3980f9 fix: only read back frames from canvas if the OS is linux, work around not necessary for other OS' line win or darwin 2026-04-18 11:31:09 +02:00
Enriquefft dd8c001f6d refactor: require validatedDurationSec in AudioProcessor, drop fallbacks
AudioProcessor.process and renderPitchPreservedTimelineAudio accepted
validatedDurationSec as optional, so the speed-aware path fell back to
media.duration when it was absent. HTMLMediaElement.duration can be
Infinity for the same MediaRecorder/Chromium Linux containers this PR
targets, which would make effectiveEnd and the playback stop checks
unreliable.

The only caller (VideoExporter.process) already threads
streamingDecoder's validatedDuration through, so make the parameter
required. Drop the media.duration fallback, the Number.isFinite guard
on readEndSec, and the two `!== undefined` checks in the tick loop.

While here:
- Document that +0.5 on readEndSec mirrors streamingDecoder.decodeAll's
  read window so trim-only and speed-aware paths stay in sync.
- Replace the unreachable silent-blob fallback at the end of
  renderPitchPreservedTimelineAudio with a loud invariant throw, so a
  broken recorder contract surfaces instead of yielding empty audio.
2026-04-16 14:49:27 -05:00
Enriquefft 0c01db7afa fix: fall back to unbounded packet scan when duration hints missing
The earlier NaN/Infinity guard collapsed both duration hints to 0 when
the container reported invalid values, which turned scanEndSec into
0.5s. The packet scan then read only the first half-second, scannedDuration
capped there, and validateDuration fell back to that wrong value for the
entire export — exactly the Chromium Linux case this PR is meant to fix.

Use a 24h sentinel as the read endpoint when no hint is usable. An
explicit end is still required (some containers are truncated without
one, per prior comment), but the sentinel is large enough to exceed any
realistic recording so the scan reaches real EOF.
2026-04-16 14:33:27 -05:00