* feat(backgroundjobs): add new background jobs module for file imports
queueing
* fix(fileuploads): a merge gone wrong
* feat(backgroundjobs): rename rhino queue env var
* test(backgroundjob): use deep equal claude
* fix(fileuploads): sync PR review
* feat(ifc_importer): initial importer app implementation with a sleeping
worker
* chore(pre-commit): remove black as a formatter, its now handled by ruff
* fix(ifc-importer): better handling of max job attempt
* feat(eslint): ignore package from eslint
* Improves error logging
- use pino error logger correctly by passing in error as first argument
* monitor deployment: Filter logging at INFO level and above
* Use structured logging to create parameters for monitoring results
* Add structured logging to obj fileimport service
* Fileimport service, fix and improve logging
- use child logger with additional context where possible
- select appropriate logging level
- fix duplicated context in log statement
* REST endpoints, add context to structured logging and remove same context from message
* Webhook service provides context to bound logger to properly use structured logging
- Pass bound logger containing context to `makeNetworkRequest`
- do not log url, as it may contain a secret (like Discord's webhook urls), instead log the webhook Id
- log error message when network call fails
* upload: make better use of structured logging when recording data
* pino-pretty when in dev or test mode
- pino-pretty configured to send to stderr
* LOG_PRETTY env var
* Silence structured logging during testing
- can not rely on determining the port number by reading from stdout/stderr
- instead we determine which port is free, then create our server on that port
- we then poll that port until the server is ready before commencing tests
* Allow puppeteer to install chromium
* Do not need to install chromium separately
* feat(helm chart): deployes Cilium Network Policies when configured
Cilium Network Policies provide more features over regular Kubernetes Network Policies, but Cilium
is not available everywhere. When selected by an operator, Cilium Network Policies will be deployed
instead of Kubernetes Network Policies.
Fixes https://github.com/specklesystems/speckle-server/issues/913
* Cilium Network Policy for fileimport service.
* tested only for external host.
* Still to test internal pod and external IP.
* Cilium network policy for file import service restricts DNS
* allows egress to service instead of endpoint
* file import service uses service url of speckle-server
* helper functions for server and dns
* DRY the prometheus selector
* CiliumNetworkPolicy for frontend
* CiliumNetworkPolicy for monitoring service
* CiliumNetworkPolicy for preview service
* CiliumNetworkPolicy for test
* CiliumNetworkPolicy for webhook_service
* CiliumNetworkPolicy for Server
* Test should egress to domain, not internally
* Test should be in tests directory to match Helm convention for tests
* Test should explicitly deny ingress from everywhere
* Server needs to egress to canonical domain (i.e. itself)
- DNS and egress for canonical domain added to Server
- As Test also egresses via canonical domain to access Server, we do not require the intra-cluster ingress to the server from the test pod
- Explicitly deny all egress from frontend
* WIP update to schema.json
* Breaking Change: inCluster network policies supported for cilium
* Breaking change: kubernetes network policy podSelector and namespaceSelector are now at a different level
* Updates schema.json
* add notes to remove egress once bug is fixed
Gergő Jedlicska