Commit Graph

14 Commits

Author SHA1 Message Date
Adam Hathcock 812dac9920 OL2(fix) Use BatchingQueue instead of BatchedPool as parsing download happens on main thread asap anyway (#5165)
* Use BatchingQueue instead of BatchedPool as parsing download happens on main thread anyway

* add tests for keyedqueue

* clean up

* more prettier fix
2025-08-26 12:07:02 +01:00
Adam Hathcock cab2a401db OL2 (fix) Simplify idb (#5174)
* Simplify idb usage and collapse the class

* fix tests

* fmt
2025-08-26 12:06:47 +01:00
Adam Hathcock 9220a3ee4b ol2(feat) remove dexie dependency (#5148)
* add new indexed db wrapper

* remove dexie and use new idb

* change storage durability

* allow undefined gets

* some clean up

* linting fixes

* add db close

* cleaner upgrade DB and more clean up

* fix database issues by deleting things instead of upgrading them
2025-07-28 15:57:07 +01:00
Adam Hathcock 7c1b9fa035 OL2(fix) Make a returned RawEncoding value return an empty Item instead of a undefined item (#5145)
* Make a returned RawEncoding value return an empty Item instead of a undefined item

* add generated tests and also cover RawEncoding case
2025-07-28 11:31:41 +01:00
Adam Hathcock 40e2b8148d OL2 (feat): dont return or save rawencoding type (#5143)
* Skip Objects.Other.RawEncoding when found

* ensure raw encoding is skipped

* Update packages/objectloader2/src/core/stages/serverDownloader.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Don't encode bytes if we don't have to

* oops, use const

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-25 14:37:55 +01:00
Adam Hathcock 877266bca7 OL2 (feat): useCache option and flag clean up (#5133)
* add addAll

* change to useCache query option which defaults to true

* add documentation

* called query params feature flags and fixed usage

* fixed debug logging

* eslint and prettier fixes

* eslint and prettier fixes

* revert

* Update packages/viewer-sandbox/src/Sandbox.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-23 15:13:24 +01:00
Adam Hathcock cc54b42a70 OL2 (fix): undefer on write (#5127)
* undefer only on write, not download

* fix test and format
2025-07-22 11:41:03 +01:00
Adam Hathcock 67bd1341aa dispose cache as it should (#5119) 2025-07-21 14:59:58 +01:00
Adam Hathcock 477db6ef02 All BatchedQueues should drain when disposed (also adds query string for output: "debug=true") (#5098)
* ensure disposal is correct

* add tests for disposal of batching queue

* fixes for draining disposal

* Update packages/objectloader2/src/queues/batchingQueue.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix comment

* fix tests and build

* add query string inspection of debug parameter

* Update packages/objectloader2/src/queues/batchingQueue.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update packages/objectloader2/src/core/objectLoader2Factory.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix test

* fix AI

* export getQueryParameter to avoid dup code.  Sandbox uses it too

* add tests for functions

* prettier fix

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-21 12:01:27 +01:00
Adam Hathcock 390e587709 Cache By Reference count (#5096)
* we don't want to break eviction checking for first item...continue the check

* always include size when adding

* scan for references and count them...request when found and don't clean up if referenced

* add display of closure calculation

* add always output to sandbox

* formatting fixes

* removed tracking of deferments, moved caching to separate concern and class

* fixed cache checking and simplifed deferment

* add tests for new deferment and cache

* formatting

* Update packages/objectloader2/src/deferment/BaseCache.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update packages/objectloader2/src/deferment/BaseCache.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update packages/objectloader2/src/core/objectLoader2.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update packages/objectloader2/src/core/objectLoader2.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update packages/objectloader2/src/core/objectLoader2.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix AI suggestions

* rename BaseCache to MemoryCache

* use private method

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-21 11:31:21 +01:00
Adam Hathcock 005b6224d0 adjust waits and actually save the root last (#5056)
* adjust waits and actually save the root last

* Update packages/objectloader2/src/core/objectLoader2.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-10 11:10:40 +01:00
Adam Hathcock f86893935f Change and fix BatchingQueue implementation (#5044)
* BatchingQueues allowed processing to happen more than one at a time.  Simplify and don't allow this.

* Use proper logging

* clean up and fix tests

* add more batchingqueue tests

* fixed according to AI

* fix: linting issues

* make deferment more intelligent and fix logging

* add to deferment tracking when bulk adding reads

* format

* fix tests
2025-07-10 10:13:58 +01:00
Adam Hathcock 1babe5859d safari fix: TextDecoder needs to be reset near 2 Gigs (#5048)
* safari fix: TextDecoder needs to be reset near 2 Gigs

* remove extra try/catch

* add back decoding for line error

* prettify
2025-07-10 10:13:24 +01:00
Adam Hathcock c5967a9616 (OL2) move files around to make more sense (#4950)
* Rename to saveBatch

* forgot a file

* first pass of cacheReader

* OL2 tests have infinite timeout

* OL2 refactor works

* fix for tests

* moved/removed types to make a more logical structure

* fixed imports

* rework loop to be in async generator for the expected count

* get rid of pumps and fix test

* lint fix

* redo mermaid diagrams

* add readme section on deferment

* always return root first

* fix linting

* revert the counting

* merge fixes

* remove unused var
2025-06-26 13:28:50 +01:00