* Fixed an issue with curves doubling up on geometry and also not being selectable after the last filtering changes. Added the options to make lines transparent. Added lines to diffing
* Points now are diff-able and support proper visual diff-ing. Visual diff filters are now chosen internally by the Differ. Fixed an issue with LineBatch and transparency
* Implemented PLAIN visual diff mode, where all objects keep their original materil, but opacity is manipulated via the diff time. Added API member function to switch between the PLAIN and COLORED visual diff modes
* feat(fe2): diffs wip
* Diffing fixes for instances and blocks. Things seem to be working fine, but there are some caveats. Additionally, some older issues were fixed and diffing now works better on all the rest of the streams
* feat(fe2): de-dupes diff results
* feat(fe2): wip diffs
* feat(fe2): diff transparency goes from 0 to 1
* feat(fe2): diff results display work
* feat(fe2): diff results display work
* feat(fe2): diff panel work
* feat(fe2): diff work: various display changes, coloring toggle, selection logic, selection object display wip
* feat(fe2): diff work: cleaned up old/new version, fixed minor bug in viewer diff time when swapping color mode
* feat(fe2): diff work: implements custom selection logic and selection display for modified objects (they come in pairs now)
* feat(fe2): diff minor fix in selected object display
* feat(fe2): wip; trying to fix diff order to be consistent (ordered by date)
* feat(fe2): wip, broken state right now
* feat(fe2): fixes scrollbars in viewer
* feat(fe2): fixes slider sync with diff time
* feat(fe2): WIP syncs of diffs (threads, refreshes, etc.)
* feat(fe2): diffing polish
* speckle shared fix
* speckle shared fix
* more bugfixes
* linter fixess
* more CI fixes
* fix viewerState serialization
* more linting fixess
* template fixes
* moving tailwind classes to theme package
* migrated away from diffString + simplified postSetup
* moved diff new/old version resolution to use state.resources
* cleanup
* updating url threadId & diff command correctly
* minor improvements to diff state
---------
Co-authored-by: AlexandruPopovici <alexandrupopoviciioan@gmail.com>
Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
* fix(helm): update documentation to match helm chart
* fix(helm): update documentation should not update README in helm repository
* fix(helm): should ensure output schema conforms to prettier requirements
* feat(pre-commit): update helm documentation as part of pre-commit
* feat(circleci): update Helm README when publishing new Helm chart
* fix(pre-commit): need to npm install before using readme generator
* feat(server): add speckle automate as a configurable default app
* feat(server): add default automate url, and helm values
* fix default app tests reporting the old number
* Revert "fix(ratelimit): reduce /graphql limit based on incident (#1505)"
This reverts commit 2a35fe6178.
* Revert helm chart defaults to value in code
- fix typo
* fix(helm): network policy should allow egress to analytics.speckle.systems
- previously only allowed DNS lookup
* matchName not matchPattern on analytics.speckle.systems
* fix(helm): fix logic in networkpolicies to check for existence of object
- object must exist before we can query its parameters
- if the 'mp' object is set and it is explicitly set to 'false' then the endpoint is blocked, otherwise it is allowed.
* fix(helm chart): accessing an unset object in the second part of an and statement breaks helm
* fix(helm): cilium network policy updated to put mp within monitoring object
- object must exist before we can query its parameters
- if the 'mp' object is set and it is explicitly set to 'false' then the endpoint is blocked, otherwise it is allowed.
* feat: register flag passed to fe
* feat: mixpanel tracking for all sign ups
* feat: utm first touch & last touch tracking
* feat(helm): Allows Environment Variable for MP to be configured
- default is enabled
- renames environment variable to ENABLE_MP
* feat(helm network policy): allowlist analytics.speckle.systems
---------
Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
* docs(helm): schematic diagram in mermaid format
* Clarifies that dependencies can be external or internal to cluster
* Explicitly show namespace containing secrets
* feat(server): add switchable admin authz override
* fix(server): make sure tests work with the new admin override
* feat(server authz): make sure to add all requested roles to server admins in admin override mode
* chore(frontend): use bitnami/openresty as base image for frontend Dockerfile
openresty/openresty was not being patched as frequently as we would like, resulting in numerous
vulnerabilities without resolution. bitnami/openresty is being patched more frequently.
Some additional changes were necessary when porting our frontend between these distributions:
- html files are in /app
- nginx.conf is in /opt/bitnami/openresty/nginx/conf/nginx.conf
- envsubst is not available by default in bitnami/openresty and needs to be copied in
- Nginx.conf - we wrap the server block in http block and overwrite root nginx.conf
- using the existing bitnami/openresty nginx.conf as a server block alone causes issues with bitnami/openresty, as bitnami/openresty provides a root nginx.conf which conflicts with directives in Speckle's server block
- we copy the directives from openresty/openresty (which are known to work with Speckle's server block), and apply them alongside Speckle's server block. This creates a new root nginx.conf which we can overwrite the default on the image.
- nginx should use a port available to non sudo/root user, we have selected 8080 instead of previous 80
- need to explicitly output nginx logs to stderr / stdout
Created a readonly root file system on Kubernetes. This requires the following changes:
- emptyDir volumes are mounted in kubernetes to allow bitnami/openresty to write to specific locations
- explicitly include and copy mime.types file to nginx configuration directory
Due to the change to non-privileged port number (8080), the following subsequent changes were required:
- Update 1-click deployment script to match frontend at port 8080
- Updates docker-compose-speckle.yaml file
Co-authored-by: Gergő Jedlicska <gergo@jedlicska.com>
* style(server): fix formatting
* fix(preview-service): fix chromium deps in Dockerfile
* feat(helm chart): expose file uploads disable flag in the helm chart
* fix(helm chart): value name fix
* fix(helm): its values
* style(server): fix formatting
* fix(preview-service): fix chromium deps in Dockerfile
* feat(helm chart): expose file uploads disable flag in the helm chart
* fix(helm chart): value name fix
* Improves error logging
- use pino error logger correctly by passing in error as first argument
* monitor deployment: Filter logging at INFO level and above
* Use structured logging to create parameters for monitoring results
* Add structured logging to obj fileimport service
* Fileimport service, fix and improve logging
- use child logger with additional context where possible
- select appropriate logging level
- fix duplicated context in log statement
* REST endpoints, add context to structured logging and remove same context from message
* Webhook service provides context to bound logger to properly use structured logging
- Pass bound logger containing context to `makeNetworkRequest`
- do not log url, as it may contain a secret (like Discord's webhook urls), instead log the webhook Id
- log error message when network call fails
* upload: make better use of structured logging when recording data
* pino-pretty when in dev or test mode
- pino-pretty configured to send to stderr
* LOG_PRETTY env var
* Silence structured logging during testing
- can not rely on determining the port number by reading from stdout/stderr
- instead we determine which port is free, then create our server on that port
- we then poll that port until the server is ready before commencing tests
* Allow puppeteer to install chromium
* Do not need to install chromium separately
* Moves speckle-server, webhook-service, fileimport-service, monitoring-deployment, and test-deployment images to Distroless.
Partially addresses https://github.com/specklesystems/speckle-server/issues/883
* preview-service uses similar image for building and production stages
* explicitly include chromium-common dependency to prevent error in preview service
* Bump chromium packages due to package versions not being found
* Handle machine-id in distroless
- distroless has no shell, so node-machine-id will result in an error
- this commit introduces error handling and defaults to a uuid v4 in the case of an error
* Update binary location for readiness and liveness checks to match the binary location in Distroless
* Allow node binary path to be set as environment variable in fileimport service
* chore(node): upgrades to node 18
Node 16 was out of support (but not security upgrades), so bumping to next stable version.
https://github.com/specklesystems/speckle-server/issues/1187
* Update server liveness and readiness probes for node 18
* Bump web-ifc to 0.0.36
* Apply `--no-experimental-fetch` flag to fileimport-service to prevent issues in web-ifc (via emscripten) with node 18
* Revert "Revert structured logging 2 (#1240)"
This reverts commit 78ecaeffcb.
* Logging should not be bundled into core shared directory
* making sure observability stuff isnt bundled into frontend
Co-authored-by: Kristaps Fabians Geikins <fabis94@live.com>
* Revert "'@' shortcut must come after it is configured in bootstrap (#1239)"
This reverts commit 967329473f.
* Revert "Structured logging (attempt 2) (#1234)"
This reverts commit 444d2ca7dd.
* Revert "Revert "feat(structured logging): implements structured logging for backend (#1217)" (#1227)"
This reverts commit 63e6581162.
* Use pino-http instead of express pino logger
* Use correct reference to knex and do not instantiate HttpLogger prematurely
* Adds missing dependency for pino to webhook-service
* Do not instantiate middleware when passed to express
* Refactor to move logging into shared
* Copy shared packages into dockerfiles
* Build shared workspace in docker build-stage for fileimport & webhook
* each log line is a json object
* structured logging allows logs to be ingested by machines and the logs to be indexed and queried addresses #1105
* structured logging allows arbitrary properties to be appended to each log line, and ingestion of logs to remain robust
* Structured logging provided by `pino` library
* Add `express-pino-logger` dependency
* Remove `debug`, `morgan`, and `morgan-debug` and replace with structured logging
* `console.log` & `console.error` replaced with structured logging in backend
* Remove `DEBUG` environment variable and replace with `LOG_LEVEL`
- Note that there is a test which reads from a logged line on `stdout`. This is not robust, it would be better to use the childProcess.pid to look up the port number.
* Log errors at points we explicitly send error to Sentry
* Amend indentation of a couple of log messages to align indentation with others
* fix(helm chart): networkPolicy supports distinct namespaces for prometheus pod & servicemonitor
Network policy did not allow ingress from prometheus if it was deployed in a different namespace
from the servicemonitor. This PR allows the ingress to be configured to match the operator's
requirements.
addresses https://github.com/specklesystems/gitOps/issues/68
* provides additional validation and error output when getting secrets
* Fix for kubernetes network policies using s3 details from ConfigMap
* Remove blocking of 10.0.0.0/8 range as this also prevents access to cloud provider private IPs
* Update values.schema.json
dependabot[bot]