Commit Graph

6 Commits

Author SHA1 Message Date
Adam Hathcock 69167dc206 update some deps and add locks to versions for known packages (#207) 2025-01-15 16:11:04 +00:00
Adam Hathcock 1fe1a54dcf Custom task scheduler for serialization, fix batch size calc (#194)
* Add a custom task scheduler

* Better usage, don't wait to enqueue to save to channels

* Completely pre-cal batch size to avoid spinning issues

* Try to fix cache counting

* properly dispose things

* format

* clean up

* adjust count and save on current thread

* move batch it's own file

* update a few packages

* fix build and add batch tests
2025-01-03 12:26:19 +00:00
Adam Hathcock 27d3028880 Revert to SQlite 7.0.5 to exactly match V2's usage (#153)
* Revert to SQlite 7.0.5 to exactly match V2's usage

* update comment to be exact
2024-10-31 14:58:52 +00:00
Adam Hathcock f5a8d1a738 Add cache download channels (#149)
* Use a stack channel for deserialization

* multi-threaded

* add object dictionary pool

* more pooling

* adjust sqlite transport

* format

* Optimize IsPropNameValid

* object loader first pass

* save test

* add cache pre check

* save better deserialize

* mostly works

* uses tasks but slower at end

* rework to make more sense

* add check to avoid multi-deserialize

* modify max parallelism

* async enqueuing of tasks

* switch to more asyncenumerable

* fmt

* fmt

* cleanup sqlite

* make ServerObjectManager

* revert change

* add ability to skip cache check

* cache json to know what is loaded

* testing

* clean up usage

* clean up and added new op

* Fix exception handling

* fixing progress

* remove codejam

* Hides ObjectPool dependency

* fmt

* Use the 1.0 BCL async to try to be more compatible

* rename to dependencies

* Move Polly to internal dependencies

* format

* remove more old references

* remove stackchannel

* fixes for registration

* remove console writeline

* add cache check shortcut for root object

* recevie2 benchmark

* add test for deserialize new

* Use same asyncinterfaces as Dynamo.  Merge fixes

* clean up

* fix download object progress

* add channels

* Use net48, netstandard2.1 and net8

* need AsyncInterfaces for net48 :(

* options changes

* revert to netstandard2.0 and net8.0

* fix totals

* revert httpcontext changes

* format

* add active tasks collection

* format

* remove more changes and remove unused method

* remove more comment changes

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2024-10-25 14:28:33 +01:00
Adam Hathcock b6d2d01068 Tasks hide new dependencies (#145)
* Use a stack channel for deserialization

* multi-threaded

* add object dictionary pool

* more pooling

* adjust sqlite transport

* format

* Optimize IsPropNameValid

* object loader first pass

* save test

* add cache pre check

* save better deserialize

* mostly works

* uses tasks but slower at end

* rework to make more sense

* add check to avoid multi-deserialize

* modify max parallelism

* async enqueuing of tasks

* switch to more asyncenumerable

* fmt

* fmt

* cleanup sqlite

* make ServerObjectManager

* revert change

* add ability to skip cache check

* cache json to know what is loaded

* testing

* clean up usage

* clean up and added new op

* Fix exception handling

* fixing progress

* remove codejam

* Hides ObjectPool dependency

* fmt

* Use the 1.0 BCL async to try to be more compatible

* rename to dependencies

* Move Polly to internal dependencies

* format

* remove more old references

* remove stackchannel

* fixes for registration

* remove console writeline

* add cache check shortcut for root object

* recevie2 benchmark

* add test for deserialize new

* Use same asyncinterfaces as Dynamo.  Merge fixes

* clean up

* fix download object progress

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2024-10-24 15:23:16 +02:00
Adam Hathcock cca8828565 Using Tasks for Deserialization (#143)
* Use a stack channel for deserialization

* multi-threaded

* add object dictionary pool

* more pooling

* adjust sqlite transport

* format

* Optimize IsPropNameValid

* object loader first pass

* save test

* add cache pre check

* save better deserialize

* mostly works

* uses tasks but slower at end

* rework to make more sense

* add check to avoid multi-deserialize

* modify max parallelism

* async enqueuing of tasks

* switch to more asyncenumerable

* fmt

* fmt

* cleanup sqlite

* make ServerObjectManager

* revert change

* add ability to skip cache check

* cache json to know what is loaded

* testing

* clean up usage

* clean up and added new op

* Fix exception handling

* fixing progress

* remove codejam

* remove stackchannel

* remove console writeline

* add cache check shortcut for root object

* recevie2 benchmark

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2024-10-22 14:14:39 +01:00