Adam Hathcock
975f0e1a51
* feat(objectloader2): Made objectloader2 to be faster! --------- Co-authored-by: AlexandruPopovici <alexandrupopoviciioan@gmail.com> Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com> Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
43 lines
1.3 KiB
Markdown
43 lines
1.3 KiB
Markdown
# objectloader2 for the Speckle viewer
|
|
|
|
This is an updated object loader for the Speckle viewer rewritten in Typescript.
|
|
|
|
The main aim for the objectloader is:
|
|
|
|
- download Speckle objects as JSON
|
|
- cache in IndexedDB so the same objects aren't downloaded twice
|
|
- give data to the viewer, as returned by the `getObjectIterator` generator, as soon as possible
|
|
- do the above as concurrently as a browser allows
|
|
|
|
## Architecture
|
|
|
|
To achieve increased concurrency, the different phases of the objectloader are divided into pools of workers with queues to feed them.
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
start(Root Commit)
|
|
getIds(Parse Root to get all IDs)
|
|
cached{Cached?}
|
|
download(Download IDs)
|
|
save(Write to Cache)
|
|
load(Load from Cache)
|
|
generate(Generate to Viewer!)
|
|
|
|
start --> getIds
|
|
getIds --> cached
|
|
cached -->|Yes| load
|
|
cached -->|No| download
|
|
load --> generate
|
|
download --> generate
|
|
download --> save
|
|
|
|
```
|
|
|
|
From the list of IDs, they are moved to a queue to be begin checking the cache from a pool of readers.
|
|
|
|
Results are then sent to the viewer, if found, else they're send to the download queue.
|
|
|
|
The download queue is a batching mechanism that gets what is available, up to a limit or a timeout. The results are parsed and given to the generator and written to another queue.
|
|
|
|
The write cache queue is processed with a single writer to the indexeddb.
|