Commit Graph

13 Commits

Author SHA1 Message Date
Iain Sproat 444d2ca7dd Structured logging (attempt 2) (#1234)
* Revert "Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)"

This reverts commit 63e6581162.

* Use pino-http instead of express pino logger
* Use correct reference to knex and do not instantiate HttpLogger prematurely
* Adds missing dependency for pino to webhook-service
* Do not instantiate middleware when passed to express
* Refactor to move logging into shared
* Copy shared packages into dockerfiles
* Build shared workspace in docker build-stage for fileimport & webhook
2022-12-05 14:49:52 +00:00
Iain Sproat 63e6581162 Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)
This reverts commit 84cb74e8b3.
2022-11-25 16:57:28 +00:00
Iain Sproat 84cb74e8b3 feat(structured logging): implements structured logging for backend (#1217)
* each log line is a json object
* structured logging allows logs to be ingested by machines and the logs to be indexed and queried addresses #1105
* structured logging allows arbitrary properties to be appended to each log line, and ingestion of logs to remain robust
* Structured logging provided by `pino` library
* Add `express-pino-logger` dependency
* Remove `debug`, `morgan`, and `morgan-debug` and replace with structured logging
* `console.log` & `console.error` replaced with structured logging in backend
* Remove `DEBUG` environment variable and replace with `LOG_LEVEL`
- Note that there is a test which reads from a logged line on `stdout`. This is not robust, it would be better to use the childProcess.pid to look up the port number.
* Log errors at points we explicitly send error to Sentry
* Amend indentation of a couple of log messages to align indentation with others
2022-11-25 16:05:05 +00:00
Iain Sproat d755136f19 Upgrade postgres in DO 1click template to 14.5 (#1090) 2022-10-21 09:51:50 +01:00
Iain Sproat c59084f4fd Upgrades redis to 7.0.5 in circleci & DO 1click configuration (#1087)
* Upgrades redis to 7.0.5 in docker-compose & circleci

* Upgrade redis on minikube to 7.0
2022-10-20 12:53:12 +01:00
Iain Sproat da7dafe819 fix(fileimport service): s3 is not required by fileimport service (#924)
Fileimport service retreives blobs via the server storage API, and not directly from s3.  Fileimport
service no longer requires information or credentials about s3.
2022-08-15 15:49:10 +01:00
Peter Grainger 72d27b9a7c Allow save object to S3 in different region (#910)
* Allow save object to S3 in different region

* feat(helm & docker-compose): adds S3_REGION to helm chart & docker-compose

Explicitly adding the environment variable to deployment configuration files provides system operators with documentation of its existence.

Set to empty by default, which will result in the default value being used.

Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
2022-08-15 14:24:30 +02:00
Iain Sproat 90847e422d Feat: configurable file limits (#835)
* Feat: configurable file limits

* ci(circleci): container build speed imporvements

* feat(frontend nginx): add file size limit configurability to frontend nginx

* feat(server blobstorage): use the new file size limit customization value

* feat(helm chart): implement the file size configuration in the helm chart

* fix(frontend docker): fix entrypoint script

* fix(server blobstorage): fix env var parsing NaN

* feat(fileimport-service): add customizable import timeout

* feat(helm chart): add fileimport service timeout value to helm chart

* feat(blobstorage): add server side blob storage size limits

* feat(docker-compose): add blob size limit env var to  docker-compose files

* refactor(frontend file uploads): refactor file uploads to use `useQuery`

* refactor(server env helper): move env helper to shared module

* refactor(blobstorage): use env helper for file size limit

* refactor(frontend file uploads): use generated query document

* fix(server blob sotrage): fix file size limit function call

Co-authored-by: Gergő Jedlicska <gergo@jedlicska.com>
2022-07-29 12:00:29 +02:00
Fabians abe6d5e086 chore: re-formatted everything with prettier 2022-03-30 12:54:24 +03:00
cristi8 e840060d8a Added FileImport Service to DigitalOcean 1click image 2021-11-05 12:33:09 +02:00
Cristian Balas 1d6fc56052 DO OneClickImage - added webhook service (#358) 2021-07-27 12:46:39 +03:00
Cristian Balas 37f18be68d Added memory limits to previewservice (#349) 2021-07-21 10:39:16 +03:00
Cristian Balas 2172a06e4f Scripts for DigitalOcean oneclick image (#209) 2021-05-03 15:21:37 +03:00