* feat(viewer-sandbox): Made a sandbox function that will only invoke the object-loader loading objects
* first pass of creating an objectloader2
* updated build + added vitest
* try to get viewer sandbox to use new code
* sandbox type fix
* refactor a bit
* can download root
* intermediate commit for downloader/caching queue
* can download stuff!
* refactor files
* intro isBase and fix isString
* move single download to downloader
* fix download
* PR feedback
* some intermediate commit
* do clean up and download better
* clean up promises and linting
* can generate values while downloading and caching
* add a finish method
* remove unused functions
* remove asBase
* add temporary docs
* add more docs with mermaid
* add more test models
* add response validation
* add tests and redo options
* add test for download batch
* fix downloader tests and change Item to have clearer Base items
* add tests and refactor a little
* use fetch in downloader as an option
* use optional in-memory indexdb instead of monkey patching the global one
* more refactors for options for objectloader2
* add tests for objectloader2
* adjust single download
* benchmark loading and adjust ol2 batches
* download more!
* adjust to use hash privates
* refactored again with renaming
* cleanup
* make setupCacheDb throw instead
* use BatchedPool for downloads!
* fix tests
* adjust timings and add adaptive waiting
* Only wait if queue wasn't empty and queue size was full
* fix tests
* fix file names and some private usage
* fix interval and private usage
* rename vars
* use params for methods
* fix params for constructors and tests
* fix params for constructors and tests again
* using dexie
* faster settings but doesn't end well
* fixed end, optimized and removed logs
* fix tests
* fix types?
* update lock with WSL
* add e2e small model test
* fix/update yarn.lock
* Remove unused eslint ignore to fix pre-commit
* prettier fixes
* fix real DB usage
* rename methods to better match OL1
* rename methods to better match OL1 again
* add extra header collection
* add headers correctly
* test getTotalObjectCount
* feat(viewer-lib): Replaced old object loader with Adam's objectloder2
* fix(viewer-lib): Removed the old object loader. Removed unneeded pause time in speckle loader
* Testing
* only deferred if not downloaded....don't save everything
* Lockfile
* pool isn't adjustable, adjust download buckets, dexie read is faster
* chore
* fix(viewer-lib): Fixed compiler errors
* fix getObject access with real indexeddb...adjust buffer for deferred access
* Fix disposal and pausing
* don't index item!
* fix dockerfiles to use OL2
* fix Dockerfile
* Fix dockerfile
* defer correctly and use record to add/lookup/remove to
* delete stuff correctly
* chore(sandbox): Enabled viewer loading
* use objects instead of arrays to avoid findIndex
* remove extra count
* add a found cache to avoid some db hits
* order matters for deferment
* move found map to deferment
* change option numbers
* 2 level cache with expiry
* defer everything, use loader to track what is requested....expire only found items
* add deferment disposal
* oops mismerge
* chore(sandbox): Default stream
* Beta version of CachePump and CacheReader
* Clean up initialization
* More clean up
* chore(objectloader2): Fixed CI compiler error
* chore(objectloader2): Fixed prettier
* add cachePump tests
* add cacheReader tests
* fixed more tests
* fixed final tests
* moving stuff around and lock return value
* try to move stuff out of objectloader2
* use a factory
* rename factory
* formatting
* eslist fixes
* try allocating no strings
* add comments
* small refactor and add another test
* fix deferment expiration and have test
* use byte size for max memory cache size
* fix deferment manager tests
* saved comment
* fix(viewer-sandbox): Fixed compiler error
* ignore tshy
* chore(frontend): Attempt to make viewer loading sequential
---------
Co-authored-by: Adam Hathcock <adam@hathcock.uk>
Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
- debian has once again passively agressively removed older curl versions from their packages, forcing us to bump the version in order to build our image
* feat(preview-generator): add new preview generator webapp
* wip(preview-service): reworking the preview service backend
* feat(previews): logging
* feat(preview-service): streamline payloads
* fix(preview-service): do not log the full payload
* feat(preview-service): build new preview service
* feat(preview-service): add separate response queue
* feat(previews): integrate preview queues with the server
* feat(previews): use module alias
* chore(previews): remove old preview service code
* feat(previews): log stuff on job statuses
* fix(previews): add missing deps and scripts
* fix(previews): package deps fix
* fix(server): moar typing fixes
* Metrics related to jobs: total count, request failures, response errors & durations
* duration should include unit.
- histogram metric should be summary
- error responses include duration in seconds
- attempt to remove metric before adding it (prevent errors with duplicate metrics)
* fix(server, frontend): some ts fixes
* fixes
* fix(frontend): remove unneeded ts-expect-error
* chore(preview-service): eslint
* TS fix
* feat(previews): more smoal fixes
* fix(preview-service): alias loading
* feat(helm): updates for new preview service queue setup
* feat(preview-service): launch new browser for each job
* feat(preview-service): add timeout, fix liveliness
* fix(helm): add access to new secret in service accounts
* tidy metrics into a separate file
* Remove broken preview service acceptance test
* fix broken import
* Add metrics to test
* feat(preview-service): handle preview service shutdown properly
* fix(previews): merge bork
---------
Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
* chore(Dockerfile): pin images by digest this ensures consistent builds with images pinned by SHA
* introduces a GitHub Action will create a PR to update the images when there is a new version, and will pin to that newer version SHA.
* Bump debian to 12 'bookworm'
* Bump pypi libraries
* except frontend-1, as only debian 11 is currently supported by openresty
* Bump debian packages
Iain Sproat