* First pass of ObjectSaver and better in-memory usage
* fix some tests
* add commit to match deserialize process
* correct more tests
* format
* make a deserialize factory
* fix tests? and format
* use distinct
* Fix mismerge
* Fix serialization issues with tests
* fix merges
* follow copilot suggestions
* remove disables
* change registration to take strings and TypeLoader isn't public
* remove unused transports
* more test fixes
* fmt
* add Application object back
* First pass of ObjectSaver and better in-memory usage
* fix some tests
* add commit to match deserialize process
* correct more tests
* format
* make a deserialize factory
* fix tests? and format
* use distinct
* Fix mismerge
* Fix serialization issues with tests
* fix merges
* follow copilot suggestions
* remove disables
* Serialize now waits for the scheduler to be completed before returning
* formatting
* wait in deserialize too
* Pass cancellation token to download, refactor how things are passed/created
* responses are now streamed and cancel them earlier
* format
* add manual empty message for sending that definitely knows when the channels are empty
* Fix configure awaits
* more configure await false
* Don't log cancelling
* redo exception handling for receive
* remove null test case
* clean up with Id/Json and more cancels
* Change the exception stacks
* fix serialization test
* make a custom scrubber for internalized exceptions
* clean up
* fix namespaces again :(
* adjust the scrubber
* try to make tests more predictable
* rework exceptions again
* strip out compile files used
* formatting
* custom exception validation
* fix init
* Move serialization to own class
* save serialize test
* add deep clean
* add cancellation test on save to cache
* cancellation tests
* format
* do DI correctly
* receive cancel works
* Add exception handling for SerializeProcess with CancellationTokenSource
* formatting
* add exception test to make sure we handle a server exception
* add extra exception and handling to stop
* add comment and another test
* one last chance for user to cancel
* formatting
* add ServerObjectManagerFactory
* add usage of a command pool
* add more disposal
* save saving increase
* fix tests
* fixes
* push out concurrency and disposablity
* Add a custom task scheduler
* Better usage, don't wait to enqueue to save to channels
* Completely pre-cal batch size to avoid spinning issues
* Try to fix cache counting
* properly dispose things
* format
* clean up
* adjust count and save on current thread
* move batch it's own file
* update a few packages
* fix build and add batch tests
* revert and format
* Revert "save saving increase"
This reverts commit 3b50c857fb.
* revert change
* adjust and add tests
* Dispose sqlite manager properly
* Make Batch a IMemoryOwner to allow for pooling
* Fix tests
* Upgrade some deps
* try to make tests more explicit
* remove return value
* Use named tuple for all objects
* Add a custom task scheduler
* Better usage, don't wait to enqueue to save to channels
* Completely pre-cal batch size to avoid spinning issues
* Try to fix cache counting
* properly dispose things
* format
* clean up
* adjust count and save on current thread
* move batch it's own file
* update a few packages
* fix build and add batch tests
* add list pool creation
* add back base cache for closures
* use bounded channels and tell them to wait
* format and add exception
* adding back json cache?
* nevermind
* format
* add option to skip total finding
* add some comments
* refactor options to use on factories, add dummies
* format
* reduce lists to yield return
* get primitives first to avoid to string
* fmt
* add cache base option
* Can debug dependencies
* Different exceptions
* Uses root id only after we found it to signal the end
* DataChunks are created later and need to be accounted for
* format
* use app ids in tests and references
* check sqlite cache after serialize
* use dummy to go through channels to end
* fmt
* Extend channel lib to batch by size
* fmt
* build fix
* adjust limits
* FIx sending
* Optimize reference generation
* more
* remove tolist
* rework closures to be constant and serializer only deals with current....references bases are cached
* fix chunk creation
* another bug fix
* clean up with factories
* add deserializer factory
* Needed to reference interface
* add Id and Json types
* Fix tests
* remove optional
* add another missing id write
* fix merge
* Fix new code
* Can debug dependencies
* Different exceptions
* Uses root id only after we found it to signal the end
* DataChunks are created later and need to be accounted for
* format
* use app ids in tests and references
* check sqlite cache after serialize
* use dummy to go through channels to end
* fmt
* Extend channel lib to batch by size
* fmt
* build fix
* adjust limits
* FIx sending
* Optimize reference generation
* more
* remove tolist
* rework closures to be constant and serializer only deals with current....references bases are cached
* fix chunk creation
* another bug fix
* clean up with factories
* add deserializer factory
* Needed to reference interface
* move around streamId
* some clean up
* Use StringBuilder pool on serialization to reduce memory pressure
* remove extra
* remove extra clears
* Fix a flaw in batchsize
* use default complete
* format
* loader should use 1 writer that is batched
* remove redundant ref gen
* Fix graphql commands by adding project id
* Use a stack channel for deserialization
* multi-threaded
* add object dictionary pool
* more pooling
* adjust sqlite transport
* format
* Optimize IsPropNameValid
* object loader first pass
* save test
* add cache pre check
* save better deserialize
* mostly works
* uses tasks but slower at end
* rework to make more sense
* add check to avoid multi-deserialize
* modify max parallelism
* async enqueuing of tasks
* switch to more asyncenumerable
* fmt
* fmt
* cleanup sqlite
* make ServerObjectManager
* revert change
* add ability to skip cache check
* cache json to know what is loaded
* testing
* clean up usage
* clean up and added new op
* Fix exception handling
* fixing progress
* remove codejam
* Hides ObjectPool dependency
* fmt
* Use the 1.0 BCL async to try to be more compatible
* rename to dependencies
* Move Polly to internal dependencies
* format
* remove more old references
* remove stackchannel
* fixes for registration
* remove console writeline
* add cache check shortcut for root object
* start refactoring send
* recevie2 benchmark
* add test for deserialize new
* use channels for sending
* test and fixes
* Use same asyncinterfaces as Dynamo. Merge fixes
* clean up
* fix download object progress
* put back from bad merge
* intermediate commit: separating get child function from serializer
* send didn't error
* add channels
* Use net48, netstandard2.1 and net8
* remove collection special case
* have to make a tree of tasks even though it may serialize things twice
* pre-id changing during serialize
* need AsyncInterfaces for net48 :(
* options changes
* revert to netstandard2.0 and net8.0
* fix totals
* revert httpcontext changes
* format
* clean up
* active tasks works when accounting for id not being stable
* add id tests
* more fixes
* works
* format
* Convert to BaseItem and use single SQLite checks to avoid locks
* use locks and batch sqlite operations
* hook up and handle null ids
* remove unused parameter
* remove progress from serializer itself
* invert has objects call
* readd object references
* format
* fix tests
* remove active tasks check
* bug fix for json cache
* remove locks from sqlite
* General Send test
* add childclosures
* redo extract all to be enumerable
* group tests in projects
* caching json does matter
* cache checking should be managed by channels
* format
* Merge pull request #152 from specklesystems/new-json-test
Uses a new objects test in Revit for serialization tests
* add skip
* add new roundtrip test
* fix finish
* clean up tests
* check happens in serialize...don't do it twice
* better progress reporting
* fix progress reporting
* only use detached properties when children gathering
* move detached tests
* add detached tests
* fix merge
* Fix progress change
* fix more tests
---------
Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
Co-authored-by: Claire Kuang <kuang.claire@gmail.com>
* Use a stack channel for deserialization
* multi-threaded
* add object dictionary pool
* more pooling
* adjust sqlite transport
* format
* Optimize IsPropNameValid
* object loader first pass
* save test
* add cache pre check
* save better deserialize
* mostly works
* uses tasks but slower at end
* rework to make more sense
* add check to avoid multi-deserialize
* modify max parallelism
* async enqueuing of tasks
* switch to more asyncenumerable
* fmt
* fmt
* cleanup sqlite
* make ServerObjectManager
* revert change
* add ability to skip cache check
* cache json to know what is loaded
* testing
* clean up usage
* clean up and added new op
* Fix exception handling
* fixing progress
* remove codejam
* remove stackchannel
* remove console writeline
* add cache check shortcut for root object
* recevie2 benchmark
---------
Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
Adam Hathcock