Commit Graph

262 Commits

Author SHA1 Message Date
Jedd Morgan 537d58dd9e Update Dockerfile (#5080) 2025-07-15 16:57:33 +01:00
Jedd Morgan 53e2f11704 bump ifc deps (#5074) 2025-07-15 11:46:51 +01:00
Gergő Jedlicska 520e931211 Gergo/web 3685 create new standalone python app for speckleifc based parser (#5051)
* feat(backgroundjobs): add new background jobs module for file imports
queueing

* fix(fileuploads): a merge gone wrong

* feat(backgroundjobs): rename rhino queue env var

* test(backgroundjob): use deep equal claude

* fix(fileuploads): sync PR review

* feat(ifc_importer): initial importer app implementation with a sleeping
worker

* chore(pre-commit): remove black as a formatter, its now handled by ruff

* fix(ifc-importer): better handling of max job attempt

* feat(eslint): ignore package from eslint
2025-07-11 20:12:43 +02:00
Jedd Morgan 3fbaf6c252 Update Dockerfile (#5054)
* Update Dockerfile

* Update Dockerfile

* Update Dockerfile
2025-07-10 10:36:35 +01:00
Jedd Morgan c4778bfa42 feat(file-import): experimental ifc openshell importer (#5028)
* first pass

* second pass

* undo local changes

* Refactors env vars to feature flags, adds helm chart values

* fix documentation of feature flag

* fix typo

* adjusted speckleifc pinned version

* renamed enableLegacyIfcImporter helm ff to legacyIfcImporterEnabled

* Added experimental ifcimporter to next-gen file importer

* Fix next-gen import service redis url fallbakcs

* add missing server url envvar

---------

Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
2025-07-04 17:01:25 +01:00
Jedd Morgan 5e61e2e9fa Chore(file-import): Major bump py dependencies for STL and OBJ importers (#5029)
* Updated stl importer to latest specklepy

* obj importer

* USER_TOKEN envvars
2025-07-03 15:58:00 +01:00
Iain Sproat 318c728516 chore(file import service): additional error logging (#5030) 2025-07-03 11:37:07 +01:00
Iain Sproat 73f5fd4cf5 chore(deps): bump gosu version (#5010)
- ubuntu have removed the version we were using
2025-06-30 19:47:46 +01:00
Kristaps Fabians Geikins 5ef0e67ac7 fix: making pg LISTEN/NOTIFY skip pool (#5007) 2025-06-30 14:54:12 +03:00
Kristaps Fabians Geikins c57262aa44 chore: log listen/notify connection strings (#5006)
* chore: log listen/notify connection strings

* re-adding multiRegionConfig alias

* CR comments

* type fix
2025-06-30 11:59:28 +03:00
Iain Sproat 4446cbf77c chore(deps): bump python3-pip package version (#4997) 2025-06-27 09:40:27 +01:00
Iain Sproat e12f050d83 fix(file import): existing gen should have token longer than time limit (#4996) 2025-06-27 09:12:25 +01:00
Iain Sproat 5e6b58e821 fix(helm chart): correctly set MAX_OBJECT_SIZE_MB & fix typo in connectionCreateTimeoutMillis (#4993) 2025-06-26 19:10:40 +01:00
Kristaps Fabians Geikins ed507c265b fix(server): not firing upload created/processed subs in next gen uploads (#4991)
* fix(server): not firing upload created subs in next gen uploads

* lint fixes

* upload finished fixes

* test fixes

* fixed file_import_started
2025-06-26 15:33:11 +03:00
Iain Sproat febe2a9e4e fix(file imports): ensure correct server url is used by parser (#4987) 2025-06-26 09:08:39 +01:00
Iain Sproat a7e928438d fix(file import): consistently apply time outs and increase default to 30 min (#4982) 2025-06-25 15:48:57 +01:00
Iain Sproat 4784fdec4f chore(file imports): metrics for next gen file importer (#4977) 2025-06-25 09:36:28 +01:00
Iain Sproat 4a58e66d99 feat(file importer): send skp to rhino parser (#4973) 2025-06-23 13:45:26 +01:00
Iain Sproat 7d0f598b3a fix(fileimport service): prevent fatal crash when send results (#4970) 2025-06-20 16:00:58 +01:00
Iain Sproat d486ffc09a fix(logging): ensure metadata is present in logging context (#4968) 2025-06-20 15:40:26 +01:00
Iain Sproat 498c1b0e53 chore(file import): bump python dependency version (#4966) 2025-06-20 11:54:04 +01:00
Iain Sproat 09cb93469e fix(fileimport service): health check for next gen importer (#4963) 2025-06-19 15:44:07 +01:00
Iain Sproat d7d564d0d4 feat(file imports): next gen allows multiple file types (#4953) 2025-06-19 08:50:42 +01:00
Iain Sproat 55cffc3782 ci(docker): tidy dockerfiles with aim to increase cache hits (#4819) 2025-06-09 16:54:24 +01:00
Iain Sproat d20cde3a68 feat(fileimport): provide admin dashboard for queue (#4904)
* feat(fileimport): provide admin dashboard for queue

* mitigate deprecation warning

* fix typo in GitHub Action
2025-06-09 13:36:43 +02:00
Iain Sproat dae238f705 feat(fileimport service): allow dedicated Redis server for queue (#4858) 2025-06-03 08:55:54 +01:00
Gergő Jedlicska 2e86a723c6 feat(fileimport-service): add next gen file importer (#4697)
* feat(fileimport-service): add next gen file importer

* feat(fileimports): integrate server and fileimporter

* chore(dui3): remove leftover artifacts

* fix(server): test typing fixes

* fix(fileimports): test and pr comment fixes

* feat(fileimports: moare test fixes

* fix(fileimports): tests and yarn dedupe
2025-05-23 10:27:00 +02:00
Iain Sproat 6e8a868c40 Merge pull request #2671 from specklesystems/iain/dockerfile-node18-node22
chore(node): bump from 18 to 22
2025-05-13 09:19:00 +01:00
Kristaps Fabians Geikins e98029ef63 fix: disable pino-pretty in inspector to avoid freezes (#4718)
* fix: disable pino-pretty in inspector to avoid freezes

* CR comments
2025-05-13 11:14:50 +03:00
Iain Sproat ab06b712a7 Merge branch 'main' into iain/dockerfile-node18-node22 2025-05-12 14:46:40 +01:00
Iain Sproat 951e9d5314 fix(fileimport): import Knex type correctly 2025-05-12 14:15:43 +01:00
Kristaps Fabians Geikins b406d0e32d chore(server): node16 & export maps support for server (#4698)
* chore(server): node16 & export maps support for server

* moar cleanup

* lint fixc
2025-05-12 14:27:34 +03:00
Gergő Jedlicska 2fdcf1bd1d refactor(shared): unified queue handling (#4691)
* feat(shared): unified queue initialization in shared

* feat(queues): use the new queue creation everywhere

* chore(shared): move to redis module

* chore(shared): fix export maps

* chore(fileimport): add deps properly

* fix(shared): import fix

* fix(everything): moear imports

* fix(server): cjs imports
2025-05-08 16:58:43 +02:00
andrewwallacespeckle 5b924a508e Add USE_LEGACY_IFC_IMPORTER to env.example 2025-05-08 11:00:27 +02:00
Iain Sproat 44140c1de3 feat(fileimport service): use new feature flag but throw on use 2025-05-07 17:37:47 +02:00
Iain Sproat 8a32b3f2ae chore(CI): speed up yarn install by skipping install of browsers 2025-05-01 20:35:43 +01:00
Iain Sproat 920099c39f chore(readable code): use TIME_MS for readable durations
- we recently added TIME_MS helper; this applies it to existing durations
2025-04-18 17:51:43 +01:00
Kristaps Fabians Geikins b6c21fd506 feat: comment read/write auth policies in BE & FE (#4368)
* webhooks perm minor fix

* tryna get fileimport service to work

* new comment policies - shared

* BE done?

* checks implemented in FE

* lint fix

* tests fix

* readme fix
2025-04-10 15:14:34 +03:00
Kristaps Fabians Geikins e9a25a247f feat(fileimport): move back to previous python-util version (#4346) 2025-04-08 13:47:19 +03:00
Kristaps Fabians Geikins ec2fb34283 feat(shared): new policies for minor fragment refactor (#4343)
* canUpdate + reworked fragments

* feat(shared): new policies for minor fragment refactor

* some cleanup

* canUpdate tests

* ensureMinimumProjectRoleFragment test

* checkIfPubliclyReadableProjectFragment tests

* get rid of LoadersOf/ErrorsOf

* python dep update
2025-04-08 12:28:20 +03:00
Jedd Morgan 468eaccdfa feat(fileimport-service): Make .NET parser default (#4272)
* first pass removing the FF

* legacyimporter

* removed unused import
2025-03-27 18:24:04 +00:00
Jedd Morgan abc8076a82 bump ifc nuget (#4269) 2025-03-26 16:04:12 +00:00
Iain Sproat f90ffb7eca fix broken merge from main 2025-03-20 10:25:41 +00:00
Iain Sproat 335f6a2a06 Merge branch 'main' into iain/dockerfile-node18-node22 2025-03-20 09:35:59 +00:00
Iain Sproat 9947673171 chore(fileimport service): pin image digest and bump nodejs version (#4216)
* chore(fileimport service): pin image digest and bump nodejs version

* Bump node to 22 in file import service
2025-03-19 10:49:30 +01:00
Iain Sproat 6c11fd5561 feat(fileimport service): allows nodejs inspection to be configured 2025-03-18 12:00:28 +00:00
Iain Sproat f667eb1d48 fix(fileimport service): re-add undici for legacy ifc importer 2025-03-15 08:02:10 +00:00
Iain Sproat d617b4a9cb fix duplication of variable 2025-03-14 16:43:51 +00:00
Iain Sproat 299de80d81 Fix linting issue 2025-03-14 16:41:58 +00:00
Iain Sproat df725f290c Merge branch 'main' into iain/file-import-more-fixes 2025-03-14 16:33:15 +00:00