Commit Graph

23 Commits

Author SHA1 Message Date
Iain Sproat 1f7620a281 chore(logging): log migration of databases (#3700)
- tidy up Database error handling context data
2024-12-16 13:40:29 +00:00
Iain Sproat 6d3ba0de22 chore(server): errors for database append additional context (#3698)
* chore(server): errors for database append additional context

* Refactor
2024-12-16 10:13:20 +00:00
Iain Sproat f5c29791ba fix(logging): remove duplicate log (#3678)
- the error is logged at the top level, so we do not need to log deeper
- instead, wrap the error with the additional information and throw
2024-12-11 17:21:40 +01:00
Kristaps Fabians Geikins 8d0cbad8b6 feat(server): multi region blob support (#3653)
Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
2024-12-10 09:05:18 +00:00
Iain Sproat bde5431022 feat(logging): improve error handling around db migrations (#3599) 2024-12-02 17:23:47 +00:00
Kristaps Fabians Geikins a9a313ee63 feat(server): cli and cross-server-sync multiregion support (#3527)
* feat(server): cross-server-sync multiregion ready

* fixed various db commands

* db cli works

* final changes
2024-11-22 19:52:58 +01:00
Iain Sproat 2a7c51f3df Revert "feat(multi-region): metrics for knex for all regional databases (#3508)" (#3534)
This reverts commit f51eb91861.
2024-11-21 11:14:04 +00:00
Iain Sproat f51eb91861 feat(multi-region): metrics for knex for all regional databases (#3508)
* feat(multi-region): metrics for knex for all regional databases

* improve typing in knex monitoring

* error logging around migrations
2024-11-21 12:11:56 +01:00
Kristaps Fabians Geikins f961a6da81 chore(server): various useful multiregion changes from comments pr (#3499)
* chore(server): various useful multiregion changes from comments pr

* hopefully fixing CI?

* test fix

* more test fixes??

* more test fixes..

* plz work ;((((

* revert multiregion on by default
2024-11-15 15:49:34 +02:00
Gergő Jedlicska f210d9b749 gergo/web 2109 project region based db connection selector (#3434)
* feat(projects): add project regions, default to null

* feat(multiregion): add projectRegion Db client lookup logic

* feat(multiregion): add project region repositories and caching

* feat(multiRegion): db initialization and get project db client

* feat(docker-compose): add second db for regions testing

* feat(multiRegion): initialize region with pubs and subs working

* fix(multiRegion): get region client even if it was registered in another pod

* feat(workspaces): create workspace resolver split

* feat: update server region metadata

* feat(projects): rewrite project creation

* feat(multiRegion): getRegionDb

* fix(workspaces): get projects now can retur null

* feat(multiRegion): make local multi region DB-s work

* feat: set d efault workspace region

* CR changes

* tests

* feat(multiRegion): bind region properly

* fe update

* test fixes

* feat(multiRegion): automatically create aiven extras plugin

* ci(postgres): use published postgres with aiven extras

* fix(multiRegion): roll back the aiven extras migration, there is a better way

* tests fix

* fix(billing): we do not need to add a seat, if the workspace is on a plan, but has no sub

---------

Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
2024-11-06 17:29:08 +01:00
Kristaps Fabians Geikins 284e9968dc knex still busted! ffs 2024-10-28 11:33:35 +02:00
Kristaps Fabians Geikins 23b9a533c6 minor fixes 2024-10-28 10:44:07 +02:00
Kristaps Fabians Geikins 01bc2e0253 chore(server): quick js to ts #5 - remaining stragglers 2024-10-25 15:47:28 +03:00
Alessandro Magionami 8c7ed4448e chore(core): minor changes to tests 2024-07-16 14:17:40 +02:00
Gergő Jedlicska c6506641a9 feat: multi region capable migrations (#2281) 2024-05-24 14:39:23 +02:00
Iain Sproat 4d01e13a84 feat(structured logging) (#1242)
* Revert "Revert structured logging 2 (#1240)"
This reverts commit 78ecaeffcb.
* Logging should not be bundled into core shared directory
* making sure observability stuff isnt bundled into frontend


Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
2022-12-06 11:51:18 +00:00
Iain Sproat 78ecaeffcb Revert structured logging 2 (#1240)
* Revert "'@' shortcut must come after it is configured in bootstrap (#1239)"

This reverts commit 967329473f.

* Revert "Structured logging (attempt 2) (#1234)"

This reverts commit 444d2ca7dd.
2022-12-05 15:46:09 +00:00
Iain Sproat 444d2ca7dd Structured logging (attempt 2) (#1234)
* Revert "Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)"

This reverts commit 63e6581162.

* Use pino-http instead of express pino logger
* Use correct reference to knex and do not instantiate HttpLogger prematurely
* Adds missing dependency for pino to webhook-service
* Do not instantiate middleware when passed to express
* Refactor to move logging into shared
* Copy shared packages into dockerfiles
* Build shared workspace in docker build-stage for fileimport & webhook
2022-12-05 14:49:52 +00:00
Iain Sproat 63e6581162 Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)
This reverts commit 84cb74e8b3.
2022-11-25 16:57:28 +00:00
Iain Sproat 84cb74e8b3 feat(structured logging): implements structured logging for backend (#1217)
* each log line is a json object
* structured logging allows logs to be ingested by machines and the logs to be indexed and queried addresses #1105
* structured logging allows arbitrary properties to be appended to each log line, and ingestion of logs to remain robust
* Structured logging provided by `pino` library
* Add `express-pino-logger` dependency
* Remove `debug`, `morgan`, and `morgan-debug` and replace with structured logging
* `console.log` & `console.error` replaced with structured logging in backend
* Remove `DEBUG` environment variable and replace with `LOG_LEVEL`
- Note that there is a test which reads from a logged line on `stdout`. This is not robust, it would be better to use the childProcess.pid to look up the port number.
* Log errors at points we explicitly send error to Sentry
* Amend indentation of a couple of log messages to align indentation with others
2022-11-25 16:05:05 +00:00
Kristaps Fabians Geikins aeeb88340d feat: favoriting streams #620 (#633) 2022-03-29 16:30:49 +03:00
Kristaps Fabians Geikins b157a98901 feat(server): simplified test running & IDE integrations 2022-03-16 17:31:53 +02:00
Dimitrie Stefanescu cd95c28939 refactor(global): restructures repo with lerna 2021-01-03 13:35:53 +02:00