Commit Graph

3161 Commits

Author SHA1 Message Date
Alexandru Popovici 4aef572a44 Multi Viewer (#1518)
* Fixed an issue with HDRIs not generating proper PMREMs over multiple viewer instances

* WIP on makign the world tree multiple instanced

* The WorldTree is no longer static. Each viewer has it's own instance and it hands it over to whoever needs it.

* Fixed an issue with filtering and the new non-static tree. Also removed the 'root' key from the NodeData structure since it's not needed

* Added an guard when building batches for situations where all render views have invalid geometries. This generally means there is somethign wrong with the stream itself

* multi-viewer css fixes

---------

Co-authored-by: Dimitrie Stefanescu <didimitrie@gmail.com>
2023-04-21 13:12:15 +03:00
Alexandru Popovici 90439d7d41 Alex/#1447 advanced batching (#1488)
* #1051 Implemented batch splitting based on a configurable mximum batch vertex count and configurable maximum batch object count. We're not splitting single objects though since that's not required right now and will do it when the need arrives

* Implemented dynamic batch object transformation using all available vertex uniforms.

* Fixed batch object index buffer size

* Added debug per object movement for debugging

* WIP on texture based object transform

* Finally fixed a bug which was making three.js use the same shader for multiple batches, and thus messing up individual obects transforms

* Integrated shadows, AO, depth, and selection material with the recent batching changes.

* Implemented per object BVH building and correct intersection testing when each object has its own transformation

* Implemented BatchObject which holds the single source of truth for individual transformations, it's BVH and some other WIP things

* Restructed how object level and batch level BVHs are constructed and used. Also added a bounds check for intersecting the batch level BVH (which is not really a bvh, just a flat list of small bvhs)

* Shapecasting now provides the intersected BatchObject inside the callbacks, so required data could be fetched from it. SectionOutlines now work correctly

* Fixed some issues regarding the shadowcatcher and the colored and ghost material uniform updates

* Fixed an issue with colored material and shadows. Restructed how speckle materials are defined and extended. This removes duplicate code as well as duplicate functionality. Implemented fro the standard material, still need to do it for the basic material

* Removed unused material and shaders

* Custom material defintion and extension MK2

* Improved the RTE and TRANSFORM_STORAGE shader blocks for minimal redundancy. Updated all shaders. Updated all materials to MK2. Still need to find a way to avoid some boilerplate code in the base materials

* Fixed an issue where three was incorrectly frustum culling objects

* Fixed the issue where depth material programs were getting ignored for batches with same object count, leading to incorrect depth

* Removed the need to have a preprocessor define for 0BJ_COUNT inside the batched shaders. This eliminates the extra startup time taken by all the program compiles. In order to do it, I had to drop using the scene override material in the depth pass. Instead we're applying the override material before the pass render on each objecti individually

* Added a generic method for overriding the batch's material, which can be used by any pass. Added a material cache for each batch so that we don't need to clone materials more than we need to. Fixed an issue in the depth mateial clone function which made three not update our uniforms (this happens so often I'm getting sick of it). Filters now use cached material so they work correctly

* Fixed a dreaded three.js bug where it would sporadically not update our custom uniforms for reasons unknown

* Finally managed to get mixins working for out material system. With this, we avoid needing to write duplicate boilerplate code for all speckle base material types. Only applied it to SpeckleStandardMaterial because I want to test more before I adopt it globally. Implemented our custom material overriding approach to the stencil and stencil mask passes.

* A lot of dead brain cells. Transforms and RTE and BVH and all them things work correctly, with the exception of transforming around a point very far from origin. That will cause jitter because we end up multiplying large floats

* Managed to get RTE working properly with individual object transforms, transforms around random pivot. Fixed an issue caused by the fact that three-mesh-bvh messes up the bounding box of objects very far away from origin. Worked around it. WIP on cleaning things up

* Solution for object transforms in a real RTE context (far from origin). First pass on cleanup. Solution will be applied to the rest of the shaders in the next commit

* Updated WebGL 1.0 transform texture sampling and matrix array transform fallback

* Updated all shaders with object transforms RTE solution

* Updated remaining materials to the new mixin'ed approach implemented a little while ago. This removes a ton of boilerplate repeated code

* Update the normals pass. Simplified and removed reduandant batch update ranges from the before/after pass callbacks

* Reorganized BatchObject so we're not wastefull with matrix and vector instantiations

* Added explode function in the viewer API

* Enabled cleaning of redundant geometry

* Some html crap

* Fixed an issue with filtering out invalid render views. Removed the red sphere of doom (forgot it in). Fixed an older issue (not related to this branch) where instance points were deleted while still being shared by other lines, yielding invalid lines

* Fixed ShowBatches

* Fixed some sandbox related issues and improved qol
2023-04-20 15:44:31 +03:00
Gergő Jedlicska 89dccd8495 feat(server): add speckle automate as a configurable default app (#1540)
* feat(server): add speckle automate as a configurable default app

* feat(server): add default automate url, and helm values

* fix default app tests reporting the old number
2023-04-19 12:59:59 +02:00
Andrew Gartrell 0b10fc3a07 Update values.yaml (#1536)
* updated tag to current tag
* dropped "v" prefix which hasn't been used since 2.3.3
2023-04-19 12:57:23 +02:00
Dimitrie Stefanescu 4c22d36f9d Merge pull request #1535 from specklesystems/dim/hotfix-ipv6
fix(server): logging ipv6 catches
2023-04-15 12:46:22 +01:00
Dimitrie Stefanescu d402837f9a fix(server): prettier saves the day as always (snark) 2023-04-15 12:46:02 +01:00
Dimitrie Stefanescu 6cc1046a86 fix(server): logging ipv6 catches 2023-04-15 12:39:24 +01:00
Iain Sproat 43c339bccf fix(logging): use broadcast address to mask logged ip (#1532) 2023-04-13 19:43:54 +01:00
Iain Sproat 8bc04f97d9 fix(logging): hash ip for all requests with an ip (#1531) 2023-04-13 17:38:17 +01:00
Iain Sproat c6e36cdfce fix(logging): log level for graphql errors (#1530) 2023-04-13 16:47:31 +01:00
Iain Sproat 6f718d8d91 fix(logging): should log headers in request; add missing return statement (#1529) 2023-04-13 15:30:34 +01:00
Iain Sproat 1515e2fee6 revert(ratelimit): defaults should remain as was prior to 2a35fe6 (#1528)
* Revert "fix(ratelimit): reduce /graphql limit based on incident (#1505)"
This reverts commit 2a35fe6178.
* Revert helm chart defaults to value in code
- fix typo
2023-04-13 15:24:51 +01:00
Iain Sproat 122f4c731f feat(log): log the ip address if a user is not logged in (#1527)
- we do not log both the ip if the user is signed in, as this may be a privacy issue
- the ip is only logged if there is no associated user information
2023-04-13 14:57:07 +01:00
Iain Sproat 477fc109d0 fix(logging): redact email values from graphql log messages (#1523) 2023-04-13 11:24:01 +01:00
Iain Sproat 271888ccd6 fix(log): graphql logger should log everything (#1521)
- improve log field naming
- remove duplicated configuring of log fields
2023-04-12 18:30:29 +01:00
Gergő Jedlicska 84ea2b1043 fix(server): make sure apollo logging works and it doesn't leak sensitive stuff (#1520) 2023-04-12 13:39:03 +01:00
Iain Sproat d61138e157 feat(log): logs all headers, except authorization and cookie (#1517)
* feat(log): logs all headers, except authorization and cookie
* fix(logging): print headers for raw response
2023-04-11 20:11:56 +01:00
Iain Sproat 4c723781b5 feat(server): authentication middleware should log auth context creation status (#1508)
* feat(server): authentication middleware should log auth context creation status
- this uses the pino http logger provided via prior express middleware, ensuring a request ID is associated with the log messages
- userID, scopes and roles will be logged
* Appends the authContext to the req.log, which makes it available on all subsequent calls
2023-04-11 18:43:46 +01:00
Iain Sproat 6c66049248 feat(logging): log apollo (graphql) requests and responses (#1509)
* feat(logging): log apollo (graphql) requests and responses
2023-04-11 18:42:40 +01:00
Snyk bot ead926dea6 [Snyk] Security upgrade numpy from 1.21.3 to 1.22.2 (#1512)
* fix: packages/fileimport-service/requirements.txt to reduce vulnerabilities


The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-NUMPY-2321964
- https://snyk.io/vuln/SNYK-PYTHON-NUMPY-2321966
- https://snyk.io/vuln/SNYK-PYTHON-NUMPY-2321970

* chore(deps): file-import python requirements should include required peers

* Improve comment on requirement

---------

Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
2023-04-10 15:25:26 +01:00
Iain Sproat f4ce7a2a5d chore(deps): remove unused aws-sdk (#1510)
* Remove aws-sdk as it does not appear to be used
2023-04-10 11:00:44 +01:00
dependabot[bot] ed005f8e89 build(deps-dev): bump vite from 2.9.8 to 3.1.0 (#975)
* build(deps-dev): bump vite from 2.9.8 to 2.9.13

Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 2.9.8 to 2.9.13.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v2.9.13/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bumps viewer-sandbox vite to match frontend version

* It was complaining about some html tags thing. Fixed it

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
Co-authored-by: AlexandruPopovici <alexandrupopoviciioan@gmail.com>
2023-04-10 10:59:48 +01:00
Iain Sproat 59c0b7ada0 feat(helm): rate limits are configurable via helm chart (#1507)
* feat(helm): rate limits are configurable via helm chart
* Document server.monitoring.mp
* Update schema.json
2023-04-07 15:17:03 +01:00
Iain Sproat 2a35fe6178 fix(ratelimit): reduce /graphql limit based on incident (#1505) 2023-04-07 12:49:45 +01:00
Iain Sproat cff0fa8e31 chore(deps): bump node images to 18.15.0 (#1500) 2023-04-05 12:15:26 +01:00
Kristaps Fabians Geikins 483a28c211 fix(server): correct server_id format (#1495) 2023-03-30 16:37:24 +03:00
Kristaps Fabians Geikins 5d5c0e012f feat(server): awaitable track calls (#1493) 2023-03-30 15:54:24 +03:00
Iain Sproat aee6de9ee0 fix(helm): network policy should allow egress to analytics.speckle.sy… (#1494)
* fix(helm): network policy should allow egress to analytics.speckle.systems
- previously only allowed DNS lookup

* matchName not matchPattern on analytics.speckle.systems
2023-03-30 14:52:02 +02:00
Kristaps Fabians Geikins 9f50a11188 feat(server): extra server tracking props + better logging (#1492) 2023-03-30 15:04:11 +03:00
Iain Sproat ca0c0437f1 Fix network policies (#1491)
* fix(helm): fix logic in networkpolicies to check for existence of object
- object must exist before we can query its parameters
- if the 'mp' object is set and it is explicitly set to 'false' then the endpoint is blocked, otherwise it is allowed.

* fix(helm chart): accessing an unset object in the second part of an and statement breaks helm

* fix(helm): cilium network policy updated to put mp within monitoring object
2023-03-30 11:23:29 +01:00
Iain Sproat 03bc5a4ead fix(helm): fix logic in networkpolicies to check for existence of object (#1490)
- object must exist before we can query its parameters
- if the 'mp' object is set and it is explicitly set to 'false' then the endpoint is blocked, otherwise it is allowed.
2023-03-30 10:52:03 +01:00
Kristaps Fabians Geikins 5d0fceaaf3 feat: proper sign up tracking (#1489)
* feat: register flag passed to fe

* feat: mixpanel tracking for all sign ups

* feat: utm first touch & last touch tracking

* feat(helm): Allows Environment Variable for MP to be configured
- default is enabled
- renames environment variable to ENABLE_MP

* feat(helm network policy): allowlist analytics.speckle.systems

---------

Co-authored-by: Iain Sproat <68657+iainsproat@users.noreply.github.com>
2023-03-30 12:21:59 +03:00
Alexandru Popovici 9b6be5ba52 Disabled rendering of line-based geometry in the shadowcatcher pass (#1476) 2023-03-23 14:04:05 +02:00
dependabot[bot] 4ebb776feb chore(deps-dev): bump webpack from 5.72.0 to 5.76.0 (#1456)
Bumps [webpack](https://github.com/webpack/webpack) from 5.72.0 to 5.76.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.72.0...v5.76.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-16 12:27:44 +00:00
Iain Sproat 3bec105c03 ci(version number): fix version number to have build number before commit number (#1458)
- helm charts are sorted in descending order. As the commit sha is not an incremented value and is effectively random, it was breaking the Helm Chart index
2023-03-15 16:40:49 +00:00
Gergő Jedlicska 0be2d5ee54 fix: (server password reset): email mjml was missing mj tags (#1452) 2023-03-14 10:41:00 +01:00
Iain Sproat 05838ac479 chore(server/fileimports): add more logging around file import failures (#1429) 2023-03-13 14:43:49 +00:00
Gergő Jedlicska c4454c2ede refactor(server email templates): use the mjml ejs rendering for all emails (#1448)
* refactor(server email templates): use the mjml ejs rendering for all emails

* fix(server access request): await the server info pls
2023-03-13 15:07:13 +01:00
Gergő Jedlicska 4e3e1de8d2 gergo/invalid token throw (#1444)
* fix(server authz): make sure to forbid access with invalid tokens

fix #927

* test(server authz tests): update tests to reflect the changes in the invalid token forbidden flow
2023-03-13 14:07:49 +01:00
Iain Sproat 87e8d08e64 chore(deps): bump node to 18.15.0 (#1446) 2023-03-10 11:01:30 +00:00
Iain Sproat c461397aa2 fix(helm): schema.json now matches values file (#1445)
- small typo fix to values.yaml documentation
2023-03-10 10:39:26 +00:00
Iain Sproat 0652af4033 fix(server): preview REST endpoint shall return 403 not 500 if no permissions (#1443) 2023-03-09 12:11:40 +00:00
Dimitrie Stefanescu c05f39556a Merge pull request #1441 from specklesystems/alex/instances
Alex/instances
2023-03-08 13:54:44 +00:00
AlexandruPopovici f187510467 Merged 2023-03-08 14:34:37 +02:00
Gergő Jedlicska 560d544daf fix(server emails): outlook doesnt support linear gradients in css (#1439)
* fix(server emails): outlook doesnt support linear gradients in css

* using the same bg color in the default eamil template
2023-03-07 18:26:09 +01:00
dependabot[bot] a647f72969 chore(deps): bump dns-packet from 5.3.1 to 5.4.0 (#1435)
Bumps [dns-packet](https://github.com/mafintosh/dns-packet) from 5.3.1 to 5.4.0.
- [Release notes](https://github.com/mafintosh/dns-packet/releases)
- [Changelog](https://github.com/mafintosh/dns-packet/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mafintosh/dns-packet/compare/v5.3.1...5.4.0)

---
updated-dependencies:
- dependency-name: dns-packet
  dependency-type: indirect
...

I checked as to what depends on it, it's a part of preview-service that we shouldn't be using at all:
```
yarn why dns-packet -R
└─ @speckle/preview-service@workspace:packages/preview-service
   └─ webpack-dev-server@npm:4.9.0 [1ebc8] (via npm:^4.6.0 [1ebc8])
      └─ bonjour-service@npm:1.0.12 (via npm:^1.0.11)
         └─ multicast-dns@npm:7.2.4 (via npm:^7.2.4)
            └─ dns-packet@npm:5.3.1 (via npm:^5.2.2)
```

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-07 12:13:03 +00:00
AlexandruPopovici b2e1737617 Fixed compile error 2023-03-06 17:46:40 +02:00
Alexandru Popovici 3932a4bb93 Occlusion queries no longer report the ocluding speckle object, only it's id. This avoids walking the tree to fetch the object itself, which is unused in 99% of uses cases for occlusion testing. If the users need the object itself, they can still fetch it via the viewer API using it's id. This improves total occlusion query speed by 30-100x (#1438) 2023-03-06 17:33:34 +02:00
Gergő Jedlicska 7d8ff2048a fix(test): make sure users are created sequentially (#1437) 2023-03-06 14:40:35 +01:00
Alexandru Popovici 7bac769180 Fixed the issue with polycurve elements being conveted twice. Added the concept of nesteNodes in NodeData interface, but I'm wary of using it extensively (#1436) 2023-03-03 16:17:23 +02:00