* feat(webhook-service): learn to speak multi region
* feat(fileimport-service): talk multi region to me
* feat(fileuploads, blobs): multi region
* feat(fileimport-service): multi region fixes
* feat(branchesAndCommits): multi region resolvers
* fix(fileimports): no need for ts ignore
* fix(fileimports): fix pr comments
* feat(webhook-service): learn to speak multi region
* refactor(webhook-service): remove unnecesary factories
* docs(activities): brain dump
* fix(shared): need to add knex as a dev dep
* fix(shared): align dev dep
* fix(server): database connection pool timeouts reduced
- previous acquisition timeout was 60s; this is reduced to 16s for faster responsiveness
- this allows for 3x attempts of 5s each to create a connection, plus idle time between attempts
* Apply to other knex instances
* chore(Dockerfile): pin images by digest this ensures consistent builds with images pinned by SHA
* introduces a GitHub Action will create a PR to update the images when there is a new version, and will pin to that newer version SHA.
* Bump debian to 12 'bookworm'
* Bump pypi libraries
* except frontend-1, as only debian 11 is currently supported by openresty
* Bump debian packages
* fix(webhook): do not log problems connecting to external webhook environment as error severity
- these are user controlled variables and do not necessarily indicate problems with our system
* Improves error logging
- use pino error logger correctly by passing in error as first argument
* monitor deployment: Filter logging at INFO level and above
* Use structured logging to create parameters for monitoring results
* Add structured logging to obj fileimport service
* Fileimport service, fix and improve logging
- use child logger with additional context where possible
- select appropriate logging level
- fix duplicated context in log statement
* REST endpoints, add context to structured logging and remove same context from message
* Webhook service provides context to bound logger to properly use structured logging
- Pass bound logger containing context to `makeNetworkRequest`
- do not log url, as it may contain a secret (like Discord's webhook urls), instead log the webhook Id
- log error message when network call fails
* upload: make better use of structured logging when recording data
* pino-pretty when in dev or test mode
- pino-pretty configured to send to stderr
* LOG_PRETTY env var
* Silence structured logging during testing
- can not rely on determining the port number by reading from stdout/stderr
- instead we determine which port is free, then create our server on that port
- we then poll that port until the server is ready before commencing tests
* Allow puppeteer to install chromium
* Do not need to install chromium separately
* Moves speckle-server, webhook-service, fileimport-service, monitoring-deployment, and test-deployment images to Distroless.
Partially addresses https://github.com/specklesystems/speckle-server/issues/883
* preview-service uses similar image for building and production stages
* explicitly include chromium-common dependency to prevent error in preview service
* Bump chromium packages due to package versions not being found
* Handle machine-id in distroless
- distroless has no shell, so node-machine-id will result in an error
- this commit introduces error handling and defaults to a uuid v4 in the case of an error
* Update binary location for readiness and liveness checks to match the binary location in Distroless
* Allow node binary path to be set as environment variable in fileimport service
* chore(node): upgrades to node 18
Node 16 was out of support (but not security upgrades), so bumping to next stable version.
https://github.com/specklesystems/speckle-server/issues/1187
* Update server liveness and readiness probes for node 18
* Bump web-ifc to 0.0.36
* Apply `--no-experimental-fetch` flag to fileimport-service to prevent issues in web-ifc (via emscripten) with node 18
* Revert "Revert structured logging 2 (#1240)"
This reverts commit 78ecaeffcb.
* Logging should not be bundled into core shared directory
* making sure observability stuff isnt bundled into frontend
Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
* Revert "'@' shortcut must come after it is configured in bootstrap (#1239)"
This reverts commit 967329473f.
* Revert "Structured logging (attempt 2) (#1234)"
This reverts commit 444d2ca7dd.
* Revert "Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)"
This reverts commit 63e6581162.
* Use pino-http instead of express pino logger
* Use correct reference to knex and do not instantiate HttpLogger prematurely
* Adds missing dependency for pino to webhook-service
* Do not instantiate middleware when passed to express
* Refactor to move logging into shared
* Copy shared packages into dockerfiles
* Build shared workspace in docker build-stage for fileimport & webhook
* each log line is a json object
* structured logging allows logs to be ingested by machines and the logs to be indexed and queried addresses #1105
* structured logging allows arbitrary properties to be appended to each log line, and ingestion of logs to remain robust
* Structured logging provided by `pino` library
* Add `express-pino-logger` dependency
* Remove `debug`, `morgan`, and `morgan-debug` and replace with structured logging
* `console.log` & `console.error` replaced with structured logging in backend
* Remove `DEBUG` environment variable and replace with `LOG_LEVEL`
- Note that there is a test which reads from a logged line on `stdout`. This is not robust, it would be better to use the childProcess.pid to look up the port number.
* Log errors at points we explicitly send error to Sentry
* Amend indentation of a couple of log messages to align indentation with others
Iain Sproat