Commit Graph

22 Commits

Author SHA1 Message Date
Iain Sproat 76decb78f3 Fix references 2025-04-28 15:23:32 +01:00
Iain Sproat c6e8d1ba77 Merge branch 'main' into iain/task-id-to-request-context 2025-04-26 08:30:20 +01:00
Iain Sproat 920099c39f chore(readable code): use TIME_MS for readable durations
- we recently added TIME_MS helper; this applies it to existing durations
2025-04-18 17:51:43 +01:00
Gergő Jedlicska 2d26c76ef2 feat(limits): better limits messages and some more fixes 2025-04-18 15:58:28 +02:00
Iain Sproat b0bdeb9768 chore(logging): improve logs & error messages for workspace migration 2025-04-16 12:52:03 +01:00
Kristaps Fabians Geikins f76a2c34d3 chore: add no floating promises lint rule (#4249)
* chore: add no floating promises lint rule

* minor cleanup

* fix test by only running if node 22 or greater

---------

Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
2025-03-25 13:36:49 +02:00
Iain Sproat d79f561647 fix broken imports 2025-03-04 16:27:44 +00:00
Iain Sproat bce8f8fd5e Merge branch 'main' into iain/task-id-to-request-context 2025-03-04 16:25:49 +00:00
Iain Sproat 47556d68c7 chore(server/scheduled tasks): should log task name and unique task Id (#4094) 2025-03-03 15:12:11 +00:00
Iain Sproat 193e7d870e chore(server/logging): email logging should include request or task Id in log context
- allows us to better link events to triggering actions
2025-03-01 23:30:57 +00:00
Iain Sproat d46b9fc5ae chore(server/scheduled tasks): should log task name and unique task Id 2025-03-01 21:16:23 +00:00
Iain Sproat ce08e5e02d refactor(server): logging directory renamed to observability (#4077) 2025-02-28 08:43:54 +00:00
Iain Sproat 54085d1dd5 fix(logging): taskScheduler should use pino parameters (#3586)
- log messages were being interpolated at runtime and not using structured logging
2024-11-29 16:46:14 +00:00
Gergő Jedlicska da7f0dda0e gergo/web 2047 user joins the workspace event (#3412)
* feat(gatekeeper): add gatekeeper module feature flag

* feat(gatekeeper): add workspace pricing table domain

* feat(gatekeeper): add checkout session creation

* feat(gatekeeper): verify stripe signature

* wip(gatekeeper): checkout callbacks

* feat(gatekeeper): add unlimited and academia plan types

* refactor(envHelper): getStringFromEnv helper

* chore(gatekeeper): add future todos

* feat(gatekeeper): add productId to the subscription domain

* feat(gatekeeper): add in memory repositories

* feat(gatekeeper): add more errors

* feat(gatekeeper): complete checkout session service

* feat(gatekeeper): add stripe client implementation

* feat(gatekeeper): add checkout session completion webhook callback path

* feat(gendo): fix not needing env vars if gendo module is not enabled

* feat(gatekeeper): require a license for billing

* chore(gatekeeper): cleanup before testing

* feat(gatekeeper): subscriptionData parsing model

* ci: add billing integration and gatekeeper modules to test config

* test(gatekeeper): add checkout service tests

* feat(gatekeeper): make completeCheckout callback idempotent properly

* feat(gatekeeper): move to knex based repositories

* test(gatekeeper): billing repository tests

* feat(gatekeeper): add yearly billing cycle toggle

* feat(ci): add stripe integration context to test job

* feat(billingPage): conditionally render the checkout CTAs

* fix(gatekeeper): remove flaky test condition

* feat(helm): add billing integration feature flag

* WIP billing gql api

* feat(gatekeeper): cancel checkout session api

* feat(gatekeeper): handle existing checkout sessions, when trying to create a new one

* feat(gatekeeper): add workspace plans gql api

* feat(gatekeeper): handle cancelation and subscription updates

* fix(gatekeeper): scope initialization

* fix(gatekeeper): eliminate stripe client import sideeffect

* fix(gatekeeper): eliminate stripe client import sideeffect 2

* feat(gatekeeper): upsize subscription on workspace role change

* feat(shared): add command pattern implementation

* refactor(eventBus): remove return capabilities from the event bus

* refactor(workspaces): use new commandFactory in workspace resolver

* feat(core): facelift taskLock

* feat(gatekeeper): shedule subscription downscale

* feat(gatekeeper): manage subscription downscale

* feat(gatekeeper): get workspace subscriptions, that are about to expire

* feat(gatekeeper): manage subscription downscale

* fix(gatekeeper): do not update subscription to canceled subs

* ci: bump postgres and max connections

* feat(workspaces): fix command factory event bugs
2024-10-30 15:51:40 +01:00
Kristaps Fabians Geikins 53844db17b chore(server): core IoC #43 - scheduleExecutionFactory 2024-10-14 12:00:47 +03:00
Iain Sproat 96bed71022 fix(logging): Improves error logging and pretty-prints logs during dev & test (#1255)
* Improves error logging
- use pino error logger correctly by passing in error as first argument

* monitor deployment: Filter logging at INFO level and above
* Use structured logging to create parameters for monitoring results
* Add structured logging to obj fileimport service
* Fileimport service, fix and improve logging
    - use child logger with additional context where possible
    - select appropriate logging level
- fix duplicated context in log statement
* REST endpoints, add context to structured logging and remove same context from message
* Webhook service provides context to bound logger to properly use structured logging
    - Pass bound logger containing context to `makeNetworkRequest`
    - do not log url, as it may contain a secret (like Discord's webhook urls), instead log the webhook Id
     - log error message when network call fails
* upload: make better use of structured logging when recording data
* pino-pretty when in dev or test mode
    - pino-pretty configured to send to stderr
* LOG_PRETTY env var
* Silence structured logging during testing
     - can not rely on determining the port number by reading from stdout/stderr
     - instead we determine which port is free, then create our server on that port
     - we then poll that port until the server is ready before commencing tests
* Allow puppeteer to install chromium
* Do not need to install chromium separately
2022-12-13 09:18:28 +00:00
Iain Sproat 4d01e13a84 feat(structured logging) (#1242)
* Revert "Revert structured logging 2 (#1240)"
This reverts commit 78ecaeffcb.
* Logging should not be bundled into core shared directory
* making sure observability stuff isnt bundled into frontend


Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
2022-12-06 11:51:18 +00:00
Iain Sproat 78ecaeffcb Revert structured logging 2 (#1240)
* Revert "'@' shortcut must come after it is configured in bootstrap (#1239)"

This reverts commit 967329473f.

* Revert "Structured logging (attempt 2) (#1234)"

This reverts commit 444d2ca7dd.
2022-12-05 15:46:09 +00:00
Iain Sproat 444d2ca7dd Structured logging (attempt 2) (#1234)
* Revert "Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)"

This reverts commit 63e6581162.

* Use pino-http instead of express pino logger
* Use correct reference to knex and do not instantiate HttpLogger prematurely
* Adds missing dependency for pino to webhook-service
* Do not instantiate middleware when passed to express
* Refactor to move logging into shared
* Copy shared packages into dockerfiles
* Build shared workspace in docker build-stage for fileimport & webhook
2022-12-05 14:49:52 +00:00
Iain Sproat 63e6581162 Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)
This reverts commit 84cb74e8b3.
2022-11-25 16:57:28 +00:00
Iain Sproat 84cb74e8b3 feat(structured logging): implements structured logging for backend (#1217)
* each log line is a json object
* structured logging allows logs to be ingested by machines and the logs to be indexed and queried addresses #1105
* structured logging allows arbitrary properties to be appended to each log line, and ingestion of logs to remain robust
* Structured logging provided by `pino` library
* Add `express-pino-logger` dependency
* Remove `debug`, `morgan`, and `morgan-debug` and replace with structured logging
* `console.log` & `console.error` replaced with structured logging in backend
* Remove `DEBUG` environment variable and replace with `LOG_LEVEL`
- Note that there is a test which reads from a logged line on `stdout`. This is not robust, it would be better to use the childProcess.pid to look up the port number.
* Log errors at points we explicitly send error to Sentry
* Amend indentation of a couple of log messages to align indentation with others
2022-11-25 16:05:05 +00:00
Kristaps Fabians Geikins 1351b6b82a feat(server): fixing double summary emails per week (#1054)
* feat(server task scheduler): sketch out core task scheduler implementation

* feat(server weekly activity digests): add function lock duration to the weekly digest execution

* feat(server scheduled tasks): add scheduled tasks type definition, db schema and migration

* feat(server scheduled tasks): add scheduled tasks repository

* feat(server task scheduler): add task scheduler service implementation

* chore(server deps): add mocha type definitions

* refactor(server scheduled tasks): refactor scheduled tasks migration

* refactor(server scheduled tasks): refactor scheduled task db schema and type definitions

* feat(server scheduled tasks): implement db side lock acquire

* refactor(server scheduled tasks): refactor task scheduler with lock on query mechanism

* test(server scheduled tasks): add tests for scheduled tasks implementation

* refactor(server weekly activity digests): refactor to new task scheduler implementation

* feat(server weekly activity digest): switch to a 1000 seconds trigger period for testing purposes

* fix(server task scheduler): fix not catching lock acquire function errors

Co-authored-by: Gergő Jedlicska <gergo@jedlicska.com>
2022-10-04 16:11:01 +02:00