Commit Graph

5 Commits

Author SHA1 Message Date
Iain Sproat b396029cf6 chore(Dockerfile): bump images to debian bookworm, pin images by digest, and github action to auto-bump (#1937)
* chore(Dockerfile): pin images by digest this ensures consistent builds with images pinned by SHA
* introduces a GitHub Action will create a PR to update the images when there is a new version, and will pin to that newer version SHA.

* Bump debian to 12 'bookworm'
* Bump pypi libraries
  * except frontend-1, as only debian 11 is currently supported by openresty
* Bump debian packages
2024-01-05 11:51:04 +00:00
Iain Sproat 96bed71022 fix(logging): Improves error logging and pretty-prints logs during dev & test (#1255)
* Improves error logging
- use pino error logger correctly by passing in error as first argument

* monitor deployment: Filter logging at INFO level and above
* Use structured logging to create parameters for monitoring results
* Add structured logging to obj fileimport service
* Fileimport service, fix and improve logging
    - use child logger with additional context where possible
    - select appropriate logging level
- fix duplicated context in log statement
* REST endpoints, add context to structured logging and remove same context from message
* Webhook service provides context to bound logger to properly use structured logging
    - Pass bound logger containing context to `makeNetworkRequest`
    - do not log url, as it may contain a secret (like Discord's webhook urls), instead log the webhook Id
     - log error message when network call fails
* upload: make better use of structured logging when recording data
* pino-pretty when in dev or test mode
    - pino-pretty configured to send to stderr
* LOG_PRETTY env var
* Silence structured logging during testing
     - can not rely on determining the port number by reading from stdout/stderr
     - instead we determine which port is free, then create our server on that port
     - we then poll that port until the server is ready before commencing tests
* Allow puppeteer to install chromium
* Do not need to install chromium separately
2022-12-13 09:18:28 +00:00
Iain Sproat 3d4b3b6979 Pin python requirements and bump to latest versions (#1140)
* Pin python requirements and bump to latest versions
* fix(fileimports): add exception printing to file imports

Co-authored-by: Gergő Jedlicska <gergo@jedlicska.com>
2022-11-01 11:17:03 +00:00
Iain Sproat 81bed0c760 style: at newlines at end of files (#893) 2022-08-08 11:06:56 +02:00
Gergő Jedlicska c894f40f83 minor updates (#817)
* feat(server): add server authz pipeline rework first sketch

* feat(server authz): add new server authz middleware poc implementation

* test(server authz): add unittests for the new server authz workflow

* feat(wip rework of fileuploads vs blob storage): add basim impl of separate blob storage service

* feat(fileimport service): refactored file import service to utilize the new asssetstorage service

* refactor(server errors): refactor server errors to use the shared module definitions

Now all the errors inherit from BaseError

* refactor(fileimport service): cleanup after refactor

* feat(frontend fileimports): use the new blob storage for downloading the original file

* refactor(server fileimports): clean up the remnants of S3 storage from file imports

* refactor(server authz): centralize generic authz pipeline configs

* refactor(server blob storage): refactor / rename everything to use the `blob-storage` name

* ci(circleci): add s3 objectstorage environment variables

* ci(circleci): fix missing env variables

* ci(circleci): add minio test container

* ci(circleci): fix minio app startup

* ci(circleci): enable circleci remote docker

* ci(circleci): fix minio startup

* ci(cirleci): detach and wait properly for minio to start

* ci(circleci): revert to additional minio img config, it only fails when the container is stopped ?!

* ci(circleci): disable file uploads

* fix(fileimports): update with blob storage refactor leftovers

* feat(server blob storage): add blob storage graphql api

* refactor(server errors): merge new errors to shared module

* fix(server comments rte): fix import for RTE error

* chore(fileimports): remove node-fetch from dependency

* chore(server): remove body parser dependency

* fix(server blob storage): fix gql api

* fix(frontend): fix fileupload item not loading the new upload status, cause of premature event fire

* feat(server blob storage): fix file size limit and allow for public streams

* Update packages/server/modules/blobstorage/graph/schemas/blobstorage.graphql

Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>

* chore(blobstorage): fix PR review issues

* fix(server): fix import bugs

* test(server): blob storage first test

* test(server blob services): add tests for blob storage services

* test(server blob storage): add service and rest api tests

* test(server blob storage): add server blob storage graphql api tests

* feat(server blob storage): store and make available blob fileHash attribute

* feat(server authz): add fatal failure option to server authz pipeline

* test(server authz): add optional stream context checks with tests

* feat(monitor deployment): add shutdown signal handling to monitor deployment container

Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
2022-07-04 19:29:45 +02:00