Commit Graph

38 Commits

Author SHA1 Message Date
Daniel Gak Anagrov c7d97eb25c feat(activity): cronjob to backfill workspace seats, plans and subcriptions and project acl activity (#5027)
* feat: backfill as a scheduled execution
2025-07-14 16:28:04 +02:00
Kristaps Fabians Geikins bde148f286 chore(server): migrating fully to ESM (#5042)
* wip

* some extra fixes

* stuff kinda works?

* need to figure out mocks

* need to figure out mocks

* fix db listener

* gqlgen fix

* minor gqlgen watch adjustment

* lint fixes

* delete old codegen file

* converting migrations to ESM

* getModuleDIrectory

* vitest sort of works

* added back ts-vitest

* resolve gql double load

* fixing test timeout configs

* TSC lint fix

* fix automate tests

* moar debugging

* debugging

* more debugging

* codegen update

* server works

* yargs migrated

* chore(server): getting rid of global mocks for Server ESM (#5046)

* got rid of email mock

* got rid of comment mocks

* got rid of multi region mocks

* got rid of stripe mock

* admin override mock updated

* removed final mock

* fixing import.meta.resolve calls

* another import.meta.resolve fix

* added requested test

* nyc ESM fix

* removed unneeded deps + linting

* yarn lock forgot to commit

* tryna fix flakyness

* email capture util fix

* sendEmail fix

* fix TSX check

* sender transporter fix + CR comments

* merge main fix

* test fixx

* circleci fix

* gqlgen bigint fix

* error formatter fix

* more error formatting improvements

* esmloader added to Dockerfile

* more dockerfile fixes

* bg jobs fix
2025-07-14 10:26:19 +03:00
Daniel Gak Anagrov ba3d9ade22 feat(activity): moved project roles to activity (#5019)
* feat: moved project roles to activity
* fix: added previous project role filed on activity
* fix: remvoke does not get logged as an actvity for every project in a workspace when removed from workspace
* fix: on delete user account, emit also seat and project event

Co-authored-by: Charles Driesler <chuck@speckle.systems>
2025-07-10 11:44:54 +02:00
Daniel Gak Anagrov a4ab20c938 feat(activity): add workspace billing events and seats to activity (#4944)
* feat: improved gatekeeper eventsfor workspace history
* feat: userId on seat assign
* feat: record workspace seat events
* feat: workspace removals as activity
* feat: emit role and seat events on create workspace
* fix: reordered events for workspace creation
2025-06-30 16:49:36 +02:00
Iain Sproat 920099c39f chore(readable code): use TIME_MS for readable durations
- we recently added TIME_MS helper; this applies it to existing durations
2025-04-18 17:51:43 +01:00
Iain Sproat 5655b3e758 fix(metrics): pass registry in to modules (#4113)
- ensure metrics are added to the correct registry
2025-03-06 08:37:05 +00:00
Iain Sproat 47556d68c7 chore(server/scheduled tasks): should log task name and unique task Id (#4094) 2025-03-03 15:12:11 +00:00
Iain Sproat ce08e5e02d refactor(server): logging directory renamed to observability (#4077) 2025-02-28 08:43:54 +00:00
Kristaps Fabians Geikins 6051917b59 chore(server): refactor activityStream invocations - batch #7 - streams (#4014)
* chore(server): refactor activityStream invocations - batch #7 - streams

* test fix

* more test fixes

* CR comment fix
2025-02-24 12:34:56 +02:00
Kristaps Fabians Geikins 9b46747815 chore(server): refactor activityStream invocations - batch #6 - project invites 2025-01-24 18:35:21 +02:00
Kristaps Fabians Geikins b6871a61cc chore(server): refactor activityStream invocations - batch #5 - comments 2025-01-24 17:54:16 +02:00
Kristaps Fabians Geikins ac88c503e3 chore(server): refactor activityStream invocations - batch #4 - commits 2025-01-24 16:57:15 +02:00
Kristaps Fabians Geikins 8ff0ae91ca chore(server): refactor activityStream invocations - batch #3 - branches 2025-01-23 12:06:54 +02:00
Kristaps Fabians Geikins 90e94d9cd7 chore(server): refactor activityStream invocations - batch #2 - accessRequest 2025-01-22 15:41:45 +02:00
Kristaps Fabians Geikins a2bc9ffbcb chore(server): refactor activityStream invocations - batch #1 - user (#3845)
* chore(server): refactor activityStream invocations - batch #1 - user

* minor cleanup

* test fix + swap to TS

* move listeners to new loc
2025-01-21 09:48:02 +01:00
Kristaps Fabians Geikins 0b81f9979e chore(server): event bus refactor - access req emitter - batch #3 2025-01-13 11:03:58 +02:00
Kristaps Fabians Geikins f403d51330 replaced user event bus 2025-01-07 11:24:07 +02:00
Kristaps Fabians Geikins 7ce306569f got rid of projects emitter + added tests 2025-01-06 17:42:24 +02:00
Gergő Jedlicska 73cc7e67d3 gergo/webhookRegions (#3459)
* feat(webhooks): multi region webhook resolver

* feat(webhooks): multi region webhook cleanup

* fix(webhooks): DI fixes

* feat(activitystream): region aware save activity

* feat(accessrequests): multi region

* feat(cli): allow multi region project and commit download

* feat(postgres): make docker postgres 0 day multi region ready

* feat(cli): allow multi region project and commit download properly

* fix(cross-server-sync): di fix

* feat(activitystream): non region aware activities, they are not project data

* fix(webhooks): triggers need to be included

* feat(stream/projectCreate): activity save is not needed any more, its all event based

* feat(multiRegion): get all registered db clients

* fix(regions): test equal in any order

* fix(projectDownload): need to await
2024-11-08 10:45:39 +01:00
Gergő Jedlicska da7f0dda0e gergo/web 2047 user joins the workspace event (#3412)
* feat(gatekeeper): add gatekeeper module feature flag

* feat(gatekeeper): add workspace pricing table domain

* feat(gatekeeper): add checkout session creation

* feat(gatekeeper): verify stripe signature

* wip(gatekeeper): checkout callbacks

* feat(gatekeeper): add unlimited and academia plan types

* refactor(envHelper): getStringFromEnv helper

* chore(gatekeeper): add future todos

* feat(gatekeeper): add productId to the subscription domain

* feat(gatekeeper): add in memory repositories

* feat(gatekeeper): add more errors

* feat(gatekeeper): complete checkout session service

* feat(gatekeeper): add stripe client implementation

* feat(gatekeeper): add checkout session completion webhook callback path

* feat(gendo): fix not needing env vars if gendo module is not enabled

* feat(gatekeeper): require a license for billing

* chore(gatekeeper): cleanup before testing

* feat(gatekeeper): subscriptionData parsing model

* ci: add billing integration and gatekeeper modules to test config

* test(gatekeeper): add checkout service tests

* feat(gatekeeper): make completeCheckout callback idempotent properly

* feat(gatekeeper): move to knex based repositories

* test(gatekeeper): billing repository tests

* feat(gatekeeper): add yearly billing cycle toggle

* feat(ci): add stripe integration context to test job

* feat(billingPage): conditionally render the checkout CTAs

* fix(gatekeeper): remove flaky test condition

* feat(helm): add billing integration feature flag

* WIP billing gql api

* feat(gatekeeper): cancel checkout session api

* feat(gatekeeper): handle existing checkout sessions, when trying to create a new one

* feat(gatekeeper): add workspace plans gql api

* feat(gatekeeper): handle cancelation and subscription updates

* fix(gatekeeper): scope initialization

* fix(gatekeeper): eliminate stripe client import sideeffect

* fix(gatekeeper): eliminate stripe client import sideeffect 2

* feat(gatekeeper): upsize subscription on workspace role change

* feat(shared): add command pattern implementation

* refactor(eventBus): remove return capabilities from the event bus

* refactor(workspaces): use new commandFactory in workspace resolver

* feat(core): facelift taskLock

* feat(gatekeeper): shedule subscription downscale

* feat(gatekeeper): manage subscription downscale

* feat(gatekeeper): get workspace subscriptions, that are about to expire

* feat(gatekeeper): manage subscription downscale

* fix(gatekeeper): do not update subscription to canceled subs

* ci: bump postgres and max connections

* feat(workspaces): fix command factory event bugs
2024-10-30 15:51:40 +01:00
Kristaps Fabians Geikins 53844db17b chore(server): core IoC #43 - scheduleExecutionFactory 2024-10-14 12:00:47 +03:00
Alessandro Magionami 78ae75eb25 Activitystream IoC 3 onUserCreated (#3234)
* chore(activitystream): addStreamUpdatedActivity refactor multi region

* chore(activitystream): addStreamAccessRequestedActivity refactor multiregion

* chore(activitystream): addStreamAccessRequestDeclinedActivity refactor multiregion

* chore(activitystream): onUserCreated refactor multiregion

* chore(activitystream): fix build
2024-10-11 13:51:02 +03:00
Alessandro Magionami 20bf7181b9 Activitystream IoC 2 addStreamAccessRequestDeclinedActivity (#3231)
* chore(activitystream): addStreamUpdatedActivity refactor multi region

* chore(activitystream): addStreamAccessRequestedActivity refactor multiregion

* chore(activitystream): addStreamAccessRequestDeclinedActivity refactor multiregion
2024-10-11 11:37:41 +02:00
Alessandro Magionami 689bed652d Activitystream IoC 1 addStreamAccessRequestedActivity (#3227)
* chore(activitystream): addStreamUpdatedActivity refactor multi region

* chore(activitystream): addStreamAccessRequestedActivity refactor multiregion
2024-10-11 11:25:42 +03:00
Kristaps Fabians Geikins 5b5dd201e2 chore(server): IoC 29 - getStream(s)Factory 2024-10-09 10:39:00 +03:00
Alessandro Magionami 60d6ee0280 chore(activitystream): IoC 1 addStreamSentOutInvite 2024-09-27 17:00:30 +02:00
Alessandro Magionami 25bc4477d6 chore(activitystream): refactor sendActivityNotification 2024-09-26 15:38:22 +02:00
Kristaps Fabians Geikins ede566eed9 feat(server): serverInvites refactor + workspace invites CRUD & GQL API (#2530)
* prep for new resources algo

* typescriptifying stuff

* minor types fix

* migrate to resources col

* repo & creation updated, WIP processing/retrieval

* WIP invite processing

* finished finalization refactor

* project invite management

* transformed all invites services

* fixed up projects & core serverinvites resolvers

* test fixes

* WIP workspace create GQL & test

* basic invite creation test works

* a buncha working tests

* more tests

* cancelation tests

* minor invite use refactor

* invite retrieval tasks

* invite use() works as expected

* filtering out broken invites

* enabled invite retrieval by token irregardless of who is it for

* minor adjustments

* tests fix

* test config improvements

* test env adjustment

* extra test case

* making resource access limits harder to ignore

* linter fixes

* eventBus type cleanup

* better generic names

* refactored serverinvites resource migration

* fix(server): better error message in project invite edge case
2024-07-29 14:37:54 +03:00
Gergő Jedlicska 9628a8a516 Gergo/configurable email digest (#2006)
* feat: add weekly digest feature flag

* feat: add weekly digest feature flag to helm chart
2024-02-02 15:25:12 +01:00
Iain Sproat 4d01e13a84 feat(structured logging) (#1242)
* Revert "Revert structured logging 2 (#1240)"
This reverts commit 78ecaeffcb.
* Logging should not be bundled into core shared directory
* making sure observability stuff isnt bundled into frontend


Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
2022-12-06 11:51:18 +00:00
Iain Sproat 78ecaeffcb Revert structured logging 2 (#1240)
* Revert "'@' shortcut must come after it is configured in bootstrap (#1239)"

This reverts commit 967329473f.

* Revert "Structured logging (attempt 2) (#1234)"

This reverts commit 444d2ca7dd.
2022-12-05 15:46:09 +00:00
Iain Sproat 444d2ca7dd Structured logging (attempt 2) (#1234)
* Revert "Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)"

This reverts commit 63e6581162.

* Use pino-http instead of express pino logger
* Use correct reference to knex and do not instantiate HttpLogger prematurely
* Adds missing dependency for pino to webhook-service
* Do not instantiate middleware when passed to express
* Refactor to move logging into shared
* Copy shared packages into dockerfiles
* Build shared workspace in docker build-stage for fileimport & webhook
2022-12-05 14:49:52 +00:00
Iain Sproat 63e6581162 Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)
This reverts commit 84cb74e8b3.
2022-11-25 16:57:28 +00:00
Iain Sproat 84cb74e8b3 feat(structured logging): implements structured logging for backend (#1217)
* each log line is a json object
* structured logging allows logs to be ingested by machines and the logs to be indexed and queried addresses #1105
* structured logging allows arbitrary properties to be appended to each log line, and ingestion of logs to remain robust
* Structured logging provided by `pino` library
* Add `express-pino-logger` dependency
* Remove `debug`, `morgan`, and `morgan-debug` and replace with structured logging
* `console.log` & `console.error` replaced with structured logging in backend
* Remove `DEBUG` environment variable and replace with `LOG_LEVEL`
- Note that there is a test which reads from a logged line on `stdout`. This is not robust, it would be better to use the childProcess.pid to look up the port number.
* Log errors at points we explicitly send error to Sentry
* Amend indentation of a couple of log messages to align indentation with others
2022-11-25 16:05:05 +00:00
Gergő Jedlicska 8b386ff078 gergo/weeklyDigestProdConfig (#1072)
* feat(server task scheduler): sketch out core task scheduler implementation

* feat(server weekly activity digests): add function lock duration to the weekly digest execution

* feat(server scheduled tasks): add scheduled tasks type definition, db schema and migration

* feat(server scheduled tasks): add scheduled tasks repository

* feat(server task scheduler): add task scheduler service implementation

* chore(server deps): add mocha type definitions

* refactor(server scheduled tasks): refactor scheduled tasks migration

* refactor(server scheduled tasks): refactor scheduled task db schema and type definitions

* feat(server scheduled tasks): implement db side lock acquire

* refactor(server scheduled tasks): refactor task scheduler with lock on query mechanism

* test(server scheduled tasks): add tests for scheduled tasks implementation

* refactor(server weekly activity digests): refactor to new task scheduler implementation

* feat(server weekly activity digest): switch to a 1000 seconds trigger period for testing purposes

* fix(server task scheduler): fix not catching lock acquire function errors

* feat(server weekly digest): switch weekly digest cron trigger to the prod ready value
2022-10-04 16:55:08 +02:00
Kristaps Fabians Geikins 1351b6b82a feat(server): fixing double summary emails per week (#1054)
* feat(server task scheduler): sketch out core task scheduler implementation

* feat(server weekly activity digests): add function lock duration to the weekly digest execution

* feat(server scheduled tasks): add scheduled tasks type definition, db schema and migration

* feat(server scheduled tasks): add scheduled tasks repository

* feat(server task scheduler): add task scheduler service implementation

* chore(server deps): add mocha type definitions

* refactor(server scheduled tasks): refactor scheduled tasks migration

* refactor(server scheduled tasks): refactor scheduled task db schema and type definitions

* feat(server scheduled tasks): implement db side lock acquire

* refactor(server scheduled tasks): refactor task scheduler with lock on query mechanism

* test(server scheduled tasks): add tests for scheduled tasks implementation

* refactor(server weekly activity digests): refactor to new task scheduler implementation

* feat(server weekly activity digest): switch to a 1000 seconds trigger period for testing purposes

* fix(server task scheduler): fix not catching lock acquire function errors

Co-authored-by: Gergő Jedlicska <gergo@jedlicska.com>
2022-10-04 16:11:01 +02:00
Gergő Jedlicska d1d5984e30 gergo/summaryEmails (#979)
* refactor(server emails): email transports module refactor to TypeScript

* refactor(docker-compose deps): move local email server to common dev compose file

* chore(server launch.json): add ts-node script running example

* chore(server deps): add nodemailer types package

* refactor(server activitystream): add strongly typed activity definitions

* feat(server activitystream): add activity repository

* feat(server info): add canonical url on the service level

* feat(server): add static file serving route to server core

* feat(server): add dependencies for periodical email digests

* feat(server activity stream): call the initialization step from the activity stream module

* feat(server activity digest): add WIP weekly email digest implementation

* feat(server digest email): smul upgrades and fixes to the email template and its contents

* just for Fabs to test

* chore(root package.json): remove deleted docker-compose references

* feat(frontend profile): add notification preferences panel

* feat(server digest emails): set prod ready cron tab and timespan

* refactor(server email digest): move templates into the email module

* refactor(server activity digests): refactor to use notifications infrastructure

* test(server activities): add tests and some refactor to activities and notification preferences

* refactor(notification preferences): fix minor issues

* test(server notification preferences test): fix describe nesting

* fix(server activities): add missing action types

* fix(server activities): fix errors after merging main

* test(server activity notifications): add test coverage for activity notifications service

* refactor(server activities): fixing tests and some cleanup

* feat(server cli): add summary notification command to cli

* chore(dev env db versions): upgrade local dev env versions

* chore(server deps): upgrade local dev db to pg 14

* fix(docker-compose): bind maildev to localhost

* process-scoped notifications test queues

* test(activity tests): add  sleep to fix flaky CI

* feat(activity digests): add demo date for digest trigger

* feat(activity digest): add UK timezone trigger date

Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
Co-authored-by: Fabians <fabis94@live.com>
2022-09-09 12:46:57 +02:00
Kristaps Fabians Geikins ecb9f825bb refactor: email verification flow & email template (#967) 2022-08-31 17:11:19 +03:00