Commit Graph

52 Commits

Author SHA1 Message Date
Iain Sproat 24d159d3cb fix(server/fileuploads): garbage collector should collect queued jobs which have reached max attempts (#5401) 2025-09-09 15:06:08 +01:00
Iain Sproat 79ef564e1b feat(server/fileuploads): background jobs draws down a compute budget (#5349) 2025-09-08 16:26:41 +01:00
Iain Sproat fdf3b93e95 fix(server/fileuploads): prevent file upload jobs silently failing (#5327) 2025-08-29 14:46:40 +01:00
Daniel Gak Anagrov b568e6d946 refactor(multiregion): reduce create stream usages in specs (#5319)
Co-authored-by: Charles Driesler <chuck@speckle.systems>
2025-08-28 14:21:26 +02:00
Chuck Driesler 8a9b4829d9 feat(multiregion): replace user replication (#5253) 2025-08-28 10:02:53 +02:00
Daniel Gak Anagrov 8fbe1322e1 chore(multiregion): drop db defaults for project (#5305)
* feat: added migration
* chore: move defaults and other functions outside of repository fn
2025-08-26 16:40:22 +02:00
Daniel Gak Anagrov f8c8170e48 feat(fileImport): gql mutation for nextGen (#5132)
* feat: drafted gql mutation

* fix: warning issue

* unified naming of action

* feat(file_uplaod): moved finishFileUplaod test to graphql

* fix: tests

* removed nextGen rest endpoint

* fix: Int type, adapted fileupload next gen call

* feat: udpated gqlgen

* improved fileimport errors

* fix: access control

* mr comments

* fix: tests

* fix: import
2025-07-24 08:00:52 +02:00
Iain Sproat d0e3377978 feat(file uploads): large file uploads API is always available (#5103) 2025-07-23 13:36:27 +01:00
Kristaps Fabians Geikins 4b06f42db7 chore(server): run TS files directly (no compilation) (#5134)
* sort of works

* type fixes

* added option to run old way too
2025-07-23 11:20:40 +02:00
Daniel Gak Anagrov 6d33b4989c feat(file_upload): update observability metrics (#5079)
* feat: added count of background jobs
* feat: added repository function to count background jobs
* feat: addapted metrics to new queueing system
* feat: added duration to fileuploads
2025-07-21 10:51:14 +02:00
Kristaps Fabians Geikins bde148f286 chore(server): migrating fully to ESM (#5042)
* wip

* some extra fixes

* stuff kinda works?

* need to figure out mocks

* need to figure out mocks

* fix db listener

* gqlgen fix

* minor gqlgen watch adjustment

* lint fixes

* delete old codegen file

* converting migrations to ESM

* getModuleDIrectory

* vitest sort of works

* added back ts-vitest

* resolve gql double load

* fixing test timeout configs

* TSC lint fix

* fix automate tests

* moar debugging

* debugging

* more debugging

* codegen update

* server works

* yargs migrated

* chore(server): getting rid of global mocks for Server ESM (#5046)

* got rid of email mock

* got rid of comment mocks

* got rid of multi region mocks

* got rid of stripe mock

* admin override mock updated

* removed final mock

* fixing import.meta.resolve calls

* another import.meta.resolve fix

* added requested test

* nyc ESM fix

* removed unneeded deps + linting

* yarn lock forgot to commit

* tryna fix flakyness

* email capture util fix

* sendEmail fix

* fix TSX check

* sender transporter fix + CR comments

* merge main fix

* test fixx

* circleci fix

* gqlgen bigint fix

* error formatter fix

* more error formatting improvements

* esmloader added to Dockerfile

* more dockerfile fixes

* bg jobs fix
2025-07-14 10:26:19 +03:00
Daniel Gak Anagrov ba3d9ade22 feat(activity): moved project roles to activity (#5019)
* feat: moved project roles to activity
* fix: added previous project role filed on activity
* fix: remvoke does not get logged as an actvity for every project in a workspace when removed from workspace
* fix: on delete user account, emit also seat and project event

Co-authored-by: Charles Driesler <chuck@speckle.systems>
2025-07-10 11:44:54 +02:00
Iain Sproat f769ff4572 fix(server/fileuploads): longer time out before expiry, from last converted time (#5039) 2025-07-08 09:42:36 +01:00
Iain Sproat 556c2791b3 feat(file imports): large file uploads now work on docker compose (#5037) 2025-07-07 11:00:55 +01:00
Iain Sproat d80fe58c1f feat(server/fileuploads): remove experimental next gen REST API (#5034) 2025-07-03 18:59:17 +01:00
Kristaps Fabians Geikins ed507c265b fix(server): not firing upload created/processed subs in next gen uploads (#4991)
* fix(server): not firing upload created subs in next gen uploads

* lint fixes

* upload finished fixes

* test fixes

* fixed file_import_started
2025-06-26 15:33:11 +03:00
Iain Sproat febe2a9e4e fix(file imports): ensure correct server url is used by parser (#4987) 2025-06-26 09:08:39 +01:00
Iain Sproat 4784fdec4f chore(file imports): metrics for next gen file importer (#4977) 2025-06-25 09:36:28 +01:00
Iain Sproat d7d564d0d4 feat(file imports): next gen allows multiple file types (#4953) 2025-06-19 08:50:42 +01:00
Kristaps Fabians Geikins 2be1592341 feat: show historical model uploads (#4954)
* ensure modelId is always filled

* BE seems fine

* frontendd

* pagination fix

* table max height

* lint fix

* unset tiny limit

* test fix
2025-06-18 10:56:33 +03:00
Iain Sproat 4738b97091 feat(server/fileuploads): use a presigned url to upload large files (#4901) 2025-06-17 17:38:47 +01:00
Daniel Gak Anagrov 460245a71b fix(versions): object history hidden when quering by project (#4884) 2025-06-04 10:44:08 +02:00
Daniel Gak Anagrov 23b61769b7 feat(fileupload): added eventBus and mixpanel monitoring 2025-06-02 08:22:20 +02:00
Gergő Jedlicska 2e86a723c6 feat(fileimport-service): add next gen file importer (#4697)
* feat(fileimport-service): add next gen file importer

* feat(fileimports): integrate server and fileimporter

* chore(dui3): remove leftover artifacts

* fix(server): test typing fixes

* fix(fileimports): test and pr comment fixes

* feat(fileimports: moare test fixes

* fix(fileimports): tests and yarn dedupe
2025-05-23 10:27:00 +02:00
Daniel Gak Anagrov 1bf7075ae6 fix(fileimport): nextGen fileimport improvements (#4746) 2025-05-15 13:17:38 +02:00
Daniel Gak Anagrov 82547b1411 feat(fileuploads): nexGenFileUpload populates messages to new job queue (#4702)
* add file import job to nextGen import endpoint

*  moved queue, organized code

* add tests, minor refactoring
2025-05-14 12:32:54 +02:00
Daniel Gak Anagrov b15d5b836c feat(fileupload): create new endpoint for file upload (#4693)
* fix(server): remove redundant code from bad merge

* wrong router

* WIP: http api for receiving file import results

* Lints

* fix

* WIP: tests

* test fix

* auth middleware has implicit requirement for param named streamId

* complete tests and fixes

* can run the old way alongside the existing, no conflicts

* tidying

* correct error thrown

* feat(fileUploads): new upload endpoint skeleton

* feat(fileUploads): added coverage for new file importer endpoint

* fix(workspaces): added tests, fix bugged on handling after stream

* refactor: removed useless file

* fix: comments, added tests

* feat: added modelId to file upload table

* fix: ensureError log

---------

Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
2025-05-09 00:16:29 +02:00
Iain Sproat 8a74d9cbe3 feat(file importer): server has REST API to receive results about a file import (#4688)
* fix(server): remove redundant code from bad merge

* wrong router

* WIP: http api for receiving file import results

* Lints

* fix

* WIP: tests

* test fix

* auth middleware has implicit requirement for param named streamId

* complete tests and fixes

* can run the old way alongside the existing, no conflicts

* tidying

* correct error thrown

* incorporate review comments
2025-05-08 19:12:15 +02:00
Kristaps Fabians Geikins c6dcf18bdb feat(server): workspace project invite auto-accept (for existing wp members) (#4622)
* WIP ts

* DI fix & tests moved to TS

* auto-accept seems to work

* CR comments
2025-04-30 17:39:07 +03:00
Kristaps Fabians Geikins eef0a37042 fix: blob upload should use auth policies (#4594)
* fix: blob upload should use auth policies

* tests fix
2025-04-25 09:13:22 +03:00
Iain Sproat 920099c39f chore(readable code): use TIME_MS for readable durations
- we recently added TIME_MS helper; this applies it to existing durations
2025-04-18 17:51:43 +01:00
Iain Sproat f75b77e899 push work on to database, simplify application logic 2025-03-31 12:03:43 +01:00
Iain Sproat 3af5ad16fe Add another test 2025-03-31 10:27:28 +01:00
Iain Sproat 5960788da1 add test 2025-03-31 09:53:36 +01:00
Mike 2ecb98146a Feat: prevent non work emails (#3993) 2025-02-26 10:55:02 +01:00
Kristaps Fabians Geikins aee14edc5b chore(server): getting rid of module-scoped event emitters - batch #1 (#3766)
* got rid of models emitter + added tests

* got rid of projects emitter + added tests

* replaced user event bus

* test migrate
2025-01-13 11:39:46 +02:00
Iain Sproat f8072faf01 fix(server/blobstorage): handles errors with missing content-type header (#3758) 2025-01-03 08:03:39 +00:00
Iain Sproat 35eb2090bc fix(POST /api/file): REST API endpoint returns json content-type (#3463) 2025-01-02 10:15:39 +00:00
Iain Sproat a1ee8a89a7 chore(server): graceful shutdown (#3125)
* chore(server): graceful shutdown
- stop() on the apollo server should be called

* chore(server): gracefully drain apollo server

* Allow grace period to be configured

* Terminus manages the readiness and liveness endpoints

* terminus is responsible for stopping the graphql server

* remove logging on shutdown

* Remove redundant parameter

* move healthchecks out of business modules to top-level directory
- terminus can only handle readiness check, not liveness
- app needs to return readiness handler, so that server terminus can use it

* fix tests

* Fix broken merge

* fix broken merge

* incorporate review comments

* fix invalid merge

* fix readinesscheck not being passed as parameter
2024-11-19 11:08:38 +01:00
Kristaps Fabians Geikins f961a6da81 chore(server): various useful multiregion changes from comments pr (#3499)
* chore(server): various useful multiregion changes from comments pr

* hopefully fixing CI?

* test fix

* more test fixes??

* more test fixes..

* plz work ;((((

* revert multiregion on by default
2024-11-15 15:49:34 +02:00
Kristaps Fabians Geikins 2d0aea90cf linter fixes 2024-11-11 16:00:37 +02:00
Gergő Jedlicska 73cc7e67d3 gergo/webhookRegions (#3459)
* feat(webhooks): multi region webhook resolver

* feat(webhooks): multi region webhook cleanup

* fix(webhooks): DI fixes

* feat(activitystream): region aware save activity

* feat(accessrequests): multi region

* feat(cli): allow multi region project and commit download

* feat(postgres): make docker postgres 0 day multi region ready

* feat(cli): allow multi region project and commit download properly

* fix(cross-server-sync): di fix

* feat(activitystream): non region aware activities, they are not project data

* fix(webhooks): triggers need to be included

* feat(stream/projectCreate): activity save is not needed any more, its all event based

* feat(multiRegion): get all registered db clients

* fix(regions): test equal in any order

* fix(projectDownload): need to await
2024-11-08 10:45:39 +01:00
Iain Sproat 0ec4a87c7c fix(file import): server port was broken when localhost was canonical url (#3360)
* fix(file import): server port was broken when localhost was canonical url
- use getServerOrigin() instead of directly calling process.env.CANONICAL_URL
- where server is calling itself, use 127.0.0.1 instead of CANONICAL_URL.
- where server is calling itself (i.e. 127.0.0.1, use the server's listening port and not default for protocol http/80, https/443
- tidy some areas where process.env was called directly; should use envHelper.

* fix tests
2024-10-22 21:12:18 +02:00
Kristaps Fabians Geikins f27f289dc6 chore(server): core IoC #78 - getServerInfoFactory 2024-10-17 12:37:43 +03:00
Kristaps Fabians Geikins 2ba9d86908 chore(server): core IoC #70 - createTokenFactory 2024-10-16 16:46:21 +03:00
Kristaps Fabians Geikins 573970fc6c chore(server): core IoC #55 - createUserFactory 2024-10-15 12:55:33 +03:00
Kristaps Fabians Geikins 1cb27e6fdd chore(server): core IoC #51 - getUser(s)Factory 2024-10-14 17:24:11 +03:00
Kristaps Fabians Geikins aa4f319277 chore(server): IoC 32 - createStreamFactory (#3204) 2024-10-09 11:10:20 +03:00
Kristaps Fabians Geikins 7d9bb3c4ea fix: single workspace admin cant accept own invite (#3197) 2024-10-08 12:44:17 +03:00
Kristaps Fabians Geikins c92938eff3 chore(server): apollo server v3 -> v4 (#2880)
* main changes seem to be done?

* lint fix

* minor cleanup

* dataloader clear
2024-09-05 12:27:13 +03:00