Compare commits

...

73 Commits

Author SHA1 Message Date
Iain Sproat e26e1077e0 Merge remote-tracking branch 'template/main' into github-template-update 2022-08-12 17:54:32 +01:00
Iain Sproat aa739c30c6 fix(pull request template): pR template should be the default and not an option
PR template was in a directory which allows selection using queries.  The PR template should be
provided by default so should be renamed and placed in the .github directory.
2022-08-12 17:34:08 +01:00
Iain Sproat 0a321c66fe Remove redundant issue template 2022-08-09 09:37:28 +01:00
Iain Sproat c112f46f01 Merge remote-tracking branch 'template/main' into github-template-update 2022-08-09 09:34:41 +01:00
Iain Sproat a4f764e178 Merge remote-tracking branch 'template/main' into github-template-update 2022-08-09 09:33:55 +01:00
Iain Sproat 59f5ee5452 chore(pr_template): adds a reference section to the PR template
The SpecklePY PR template had a reference section, and it made sense to include it for all
repositories.
2022-08-09 09:32:49 +01:00
Iain Sproat f8b057b990 Refer to the code of conduct in the contributing section of the README 2022-08-08 15:11:09 +01:00
Iain Sproat e2ba8b144a Add a SECURITY.md file 2022-08-08 10:00:36 +01:00
Iain Sproat 8d320abe00 style: tidy newlines and other small formatting 2022-08-08 09:41:18 +01:00
Iain Sproat b77e346736 Merge pull request #3 from specklesystems/revise-issue-templates
Feature: separates issue template into bugs and feature requests
2022-07-27 16:02:55 +01:00
izzy lyseggen 9fa6b661eb feat(client): stream invites (#202)
* chore: update dev container

* feat(client): add server version request

* docs(client): enhanced server version docstring

* test(client): sever version test

* test(base): type check for union type

* refactor(base): move _ attrs to `RegisteringBase`

* refactor: some light cleanup

* chore(container): add pylint line length

* feat(metrics): add "Invite Action"

* feat(client): pass server version to resources

* feat(models): pending stream collaborator model

* fix(client): do not parse if request res is null

* feat(exceptions): add unsupported exception

to distinguish for new invites as of server 2.6.4

* feat(client): add stream invite queries/mutations

* test(client): test stream invites

* feat(client): incorporate last invites changes
2022-07-26 17:58:46 +01:00
Iain Sproat 0d1c2735d8 checklist is clearer 2022-07-21 17:14:16 +01:00
Iain Sproat 8f3a683851 Retain some sections from previous issue template 2022-07-21 17:08:34 +01:00
Iain Sproat aa8c7b6f42 Add link to contribution guidelines 2022-07-21 17:03:08 +01:00
Iain Sproat 68036ee130 Feature: separates issue template into bugs and feature requests
* Provides checklist for both issue templates
* Hides instructions in comments
2022-07-21 16:56:10 +01:00
Iain Sproat 447f28c9f1 Merge pull request #2 from specklesystems/revise-pr-template
Fix: PR template updated to provide detailed instructions
2022-07-21 13:08:23 +01:00
Iain Sproat 1e7291277e Fix link to relative to the repo pull requests 2022-07-21 13:06:25 +01:00
Iain Sproat 46773aa9d3 Add link to speckle-server contribution guide 2022-07-21 12:54:41 +01:00
Iain Sproat 480ea91ebb Fixes: PR template updated to provide detailed instructions 2022-07-21 12:43:28 +01:00
izzy lyseggen d1adccba00 feat(serialization): cache bases on deserialize (#200)
* feat(serialization): cache bases on deserialize

allows the deserializer to return the same instance when encountering
objects with the same id

closes #191

* feat(serialization): cache bases on deserialize

allows the deserializer to return the same instance when encountering
objects with the same id

closes #191

* fix(serialization): check before accessing id

obj may not be detached and therefore might not have an id

* feat(serializer): cache w id from obj not base

* chore: update deps

* feat(operations): lil send helper

on send, if `transports` is just a transport, add it to a list.
i see this error a lot so just a friendly lil fix!

* feat(objects): breps omg!

* test(geometry): updates for breps

* test(geometry): more brep serialization tests

* refactor(test): formatting

* style: formatting

* test(geometry): clean up test file

* fix(test): brep trims test fix

* refactor(geometry): clean up encoding outputs

* feat(objects): allow kwargs in encoding

* feat(objects): align curve encodings w sharp

* test(geometry): new curve encodings

* feat(client): update stream permission mutation

guess this changed some time recently and i wasn't made aware of it

* fix(objects): brep face and edge encoding

* fix(geometry): breps units 'none' fix

* test(objects): fix 'none' units issue

* revert(486ea99): use `streamGrantPermission`

to be updated for next server release
2022-07-20 18:02:25 +01:00
Reynold Chan 9e30250446 Create test_structural.py (#180) 2022-07-15 16:38:55 +01:00
Morten Engen 3e7d657e2e feat(base): sort serializable attributes (#198)
Sorts the serializable attributes before they are returned from
.get_serializable_attributes(). Prevents the object id to change if the
order of the member names from .get_member_names() changes.
2022-07-11 12:19:22 +01:00
izzy lyseggen 7484d8441b fix(credentials): clean up sqlite transport directly after getting accts (#199)
* fix(sqlite): use `self.close()` on del

* fix(credentials): close sqlite directly after use

* refactor(serializer): consolidate var name for ids
2022-07-11 10:31:19 +01:00
izzy lyseggen 682afce05f fix(transport): call begin/end write in server (#195)
* fix(transport): call begin/end write in server

* style: remove unused code
2022-06-30 16:16:05 +01:00
izzy lyseggen 21b27e2f3b feat(metrics): add ids for unknown users (#194)
* feat(metrics): add `merge_ids` helper method

* fix(metrics): use alias ids instead

* fix(metrics): final cleanup for aliasing

* fix(metrics): lol jk scratch the aliasing
2022-06-28 12:23:19 +01:00
izzy lyseggen 69cd9706cf fix(base): remove default units to Base (#193)
* fix(base): remove default units to `Base`

change in sharp that wasn't propagated to py!

* fix(objects): add `None` to unit encodings
2022-06-22 17:03:50 +01:00
izzy lyseggen 98075fa2cf fix(metrics): remove unused prop (#192) 2022-06-22 14:52:49 +01:00
izzy lyseggen 782f70fb49 chore: drop python 3.6 and update ujson (#190)
* chore: depreciate python 3.6 support & upate ujson

after collection python version info metrics, we fount that only 2 users
are still using python 3.6. since it has been eol for 5 months now,
we believe it's safe to let it go.

rest easy 3.6 ⚰- you served us well 🫡

closes Please upgrade the ujson dependency, which has a CVE #160

* chore: upgrade and clean some deps
2022-06-20 12:19:09 +01:00
Gergő Jedlicska 52ab27e60f SQLite write batching (#188)
### SUMMARY
**sqlite transport**

This transport now batches and bulk inserts objects when writing resulting in huge performance improvements (100x).

**base object serializer**

Batching in the sqlite transport necessitated some refactoring here in order to safely call end_write when not using operations.send/receive. This has been resolved by turning traverse_base into a wrapper for _traverse_base which can take care of calling begin/end_write and resetting the writer at the top level. This is not breaking since the top level methods to call have not changed names and the original method has just been prepended with a _

Additionally, missing referenced child objects in the read transport used to raise a SpeckleException. However, using the gql client to call objects.get() will return an object with missing references by design thus throwing an error in serialization. This has been resolved by instead raising a SpeckleWarning when child objects can't be found and just returning the reference + id. ((this method of interacting with objects is discouraged so it is not surprising to me that this bug was lurking for so long - but an oopsie nonetheless!))

**ci / dev**

Updates for the ci config and the dev container to work with the recent changes in server.

NOTE: dev container seems to be pulling an older version of server -- not resolved yet

---

* quick and hacky sqlite batching

* feat(transports): batching sqlite inserts

* chore: upgrade gql3

also removed py-spy as it's not used and i was getting install errors :/

* ci: bump node version

* ci: formatting

* update CI versions

* update to new circleci redis baseimage

* update test fixture auth to non deprecated token based method

* add start and finish write method calls to base object serialize

* chore: dev container update

* fix(serialization): move end and begin write

* style: formatting

* fix(serializer): warn but don't throw if ref not found

this is _not_ an issue with the transports, but an issue with using the
graphql api to fetch objects. since you are only receiving one obj and none of
the children, the transport has no way to find them and should simply
return the reference as is. idk why anyone would really use `object.get`
so tbh i'm not surprised no one has found this bug yet lol

* fix(client): don't parse obj create response

* fix(serializer): wrap `traverse_base`

moving `begin` and `end_write` to the seriazlier due to the new
sqlite transport with batched writes necessitates a wrapper around
`traverse_base` so end/begin write can be called once at the top level.
just adding begin/end write to the original traversal method would make
tons of calls to `end_write` since the traversal is recursive

Co-authored-by: izzy lyseggen <izzy.lyseggen@gmail.com>
2022-06-20 12:00:09 +01:00
izzy lyseggen 61e7ebeabd fix(client): add faves count to mutation return (#186) 2022-04-25 11:45:54 +01:00
izzy lyseggen 3a8121c306 feat(client): stream model update + favoriting (#185)
* feat(models): a quickie lil update

- adds fave and comment count
(mutations to come)
- adds source app to commit within stream query

* feat(client): add favorite mutation
2022-04-25 11:32:24 +01:00
izzy lyseggen 209a95879f fix(metrics): patch app version check 2022-04-22 11:46:27 +01:00
izzy lyseggen 4f829d9908 feat(metrics): some cleanup and updates (#184)
- add metrics for client init / auth
- add server metrics
- remove incompatible server check in client
(at this point, it's been long enough that I think it's fine and will
save time on request / esp in places like blender)
2022-04-22 11:26:28 +01:00
luzpaz ac5345f528 Fix various typos (#181) 2022-04-21 17:56:11 +01:00
izzy lyseggen 1142481d89 fix(geometry): int(index vals) for curve encoding (#183)
* fix(geometry): int(index vals) for curve encoding

* fix(client): update poss invalid token check

server now returns `None` instead of a `GraphqlExcetion` when asking for
the user with an invalid token (or no scopes token)
2022-04-21 17:50:43 +01:00
izzy lyseggen b4690f082f feat(objects): revit params in objects for blender (#179) 2022-04-01 11:58:49 +01:00
izzy lyseggen 81a98ea938 feat(client): stream and user activity (#176)
* feat(models): `ActivityCollection` and `Activity`

* feat(client): stream activity method

* test(client): test for stream activity

* refactor(client): use datetime args for activity

* docs(client): clean up stream activity docstring

* feat/test(client): user activity
2022-03-23 17:34:10 +00:00
izzy lyseggen 9b387da77a feat(serialisation): enums (#175)
note that this won't re-serialise dynamic members as enums.
they will come back as ints for consistency w sharp

closes 🃏 Enum serialisation bug #174
2022-03-23 11:49:16 +00:00
izzy lyseggen d0724c7d06 fix(objects): brep display val fix (#170)
* fix(client): auth fix

* fix(objects): temp displayValue prop setter

will be removed in the future, but keeping it now for backwards compat
2022-03-02 14:17:04 +00:00
izzy lyseggen 1414a3611b fix(wrapper): use full url for creating shell account (#169)
used for creating a transport if you don't have a local account
for the specified server
2022-03-01 10:33:47 +00:00
izzy lyseggen a553c17c43 fix/test(serialization): null values in dicts (#168) 2022-02-24 11:31:04 +00:00
izzy lyseggen 0be3fac6ab docs: update streamwrapper docstring 2022-02-23 16:52:23 +00:00
Matteo Cominetti 1c0d6ce8f4 Create close-issue.yml 2021-10-02 17:03:18 +01:00
Matteo Cominetti 1431e306b8 Create open-issue.yml 2021-10-02 17:02:55 +01:00
Dimitrie Stefanescu 83bca13c8b Update README.md 2021-05-23 16:28:34 +01:00
Matteo Cominetti 1bcef9faf6 docs: adds link to docs 2021-02-19 18:40:56 +00:00
Matteo Cominetti 8d3e511d18 docs: removes links to slack 2021-01-06 16:45:48 +00:00
Alan Rynne 162f999100 fix: added yaml frontmatter block to issue template 2020-10-05 17:48:16 +02:00
Alan Rynne 2765c4fa69 Merge pull request #1 from specklesystems/alan/github-folder
Moved relevant files to .github/ folder
2020-10-05 17:35:52 +02:00
Alan Rynne 69cb2c79c7 fix: updated old link 2020-10-05 17:01:14 +02:00
Alan Rynne e2daad36e9 feat: added PR template
Updated docs to reflect it.
2020-10-05 16:58:20 +02:00
Alan Rynne d6b06298ed refactor: moved files to .github/ folder 2020-10-05 16:56:52 +02:00
Matteo Cominetti 7ddd827340 fix: more links 2020-08-21 17:56:51 +01:00
Matteo Cominetti 2a30278e04 fix: link and typos 2020-08-21 17:52:00 +01:00
Dimitrie Stefanescu ecd9089e29 Update README.md 2020-08-21 19:08:32 +03:00
izzy lyseggen bcecaef380 docs: add slack link and badge 2020-08-20 17:11:53 +01:00
Dimitrie Stefanescu 8e986e59aa Update CODE_OF_CONDUCT.md 2020-08-20 18:45:50 +03:00
Dimitrie Stefanescu 9e110a125b Update CONTRIBUTING.md
fixes link
2020-08-20 18:45:35 +03:00
Dimitrie Stefanescu ec8635401b Update README.md 2020-08-20 18:44:18 +03:00
Dimitrie Stefanescu f7b867c219 Update README.md 2020-08-20 18:37:15 +03:00
Dimitrie Stefanescu e69310619e Create LICENSE 2020-08-20 18:21:43 +03:00
Dimitrie Stefanescu 4a924593b3 Update README.md 2020-08-20 18:16:52 +03:00
Dimitrie Stefanescu 1f57e81ddc Update README.md 2020-08-20 18:16:14 +03:00
Dimitrie Stefanescu e42a3d4147 Update README.md 2020-08-20 18:04:36 +03:00
Dimitrie Stefanescu d3d53ef6a5 Update README.md 2020-08-20 18:01:04 +03:00
Dimitrie Stefanescu acb7156bf2 Update README.md
adds basic default social badges - discourse and twitter
2020-08-20 17:56:41 +03:00
Dimitrie Stefanescu 42cda6a477 Update and rename CONTRIBUTING.MD to CONTRIBUTING.md 2020-08-20 17:44:23 +03:00
Dimitrie Stefanescu c1dfe5f11f Update CODE_OF_CONDUCT.md 2020-08-20 17:41:43 +03:00
Dimitrie Stefanescu 7e57b4cfb6 Create ISSUE_TEMPLATE.md 2020-08-20 17:40:39 +03:00
Dimitrie Stefanescu b87237b88f Update CODE_OF_CONDUCT.md
adds authoritative source notice to this repo
2020-08-20 17:28:11 +03:00
Dimitrie Stefanescu fb797e64cb Create CONTRIBUTING.MD 2020-08-20 17:25:04 +03:00
Dimitrie Stefanescu 040a49baea Create CODE_OF_CONDUCT.md 2020-08-20 17:15:20 +03:00
Dimitrie Stefanescu 105ae0316c Initial commit 2020-08-20 17:11:10 +03:00
51 changed files with 2945 additions and 1385 deletions
+9 -8
View File
@@ -1,16 +1,16 @@
version: 2.1
orbs:
python: circleci/python@1.3.2
python: circleci/python@2.0.3
codecov: codecov/codecov@3.2.2
jobs:
test:
docker:
- image: "cimg/python:<<parameters.tag>>"
- image: 'cimg/node:14.18'
- image: 'circleci/redis:6'
- image: 'cimg/postgres:12.8'
- image: "cimg/node:16.15"
- image: "cimg/redis:6.2"
- image: "cimg/postgres:14.2"
environment:
POSTGRES_DB: speckle2_test
POSTGRES_PASSWORD: speckle
@@ -27,6 +27,7 @@ jobs:
STRATEGY_LOCAL: "true"
CANONICAL_URL: "http://localhost:3000"
WAIT_HOSTS: localhost:5432, localhost:6379
DISABLE_FILE_UPLOADS: "true"
parameters:
tag:
default: "3.8"
@@ -51,7 +52,7 @@ jobs:
deploy:
docker:
- image: "circleci/python:3.8"
- image: "cimg/python:3.8"
steps:
- checkout
- run: python patch_version.py $CIRCLE_TAG
@@ -60,17 +61,17 @@ jobs:
workflows:
main:
jobs:
jobs:
- test:
matrix:
parameters:
tag: ["3.6", "3.7", "3.8", "3.9"]
tag: ["3.7", "3.8", "3.9", "3.10"]
filters:
tags:
only: /.*/
- deploy:
requires:
- test
- test
filters:
tags:
only: /[0-9]+(\.[0-9]+)*/
+4 -4
View File
@@ -1,11 +1,11 @@
# See here for image contents: https://github.com/microsoft/vscode-dev-containers/tree/v0.191.1/containers/python-3/.devcontainer/base.Dockerfile
# See here for image contents: https://github.com/microsoft/vscode-dev-containers/blob/main/containers/python-3/.devcontainer/base.Dockerfile
# [Choice] Python version: 3, 3.9, 3.8, 3.7, 3.6
ARG VARIANT="3.9"
FROM mcr.microsoft.com/vscode/devcontainers/python:0-${VARIANT}
ARG VARIANT="3.10"
FROM mcr.microsoft.com/vscode/devcontainers/python:${VARIANT}
# [Choice] Node.js version: none, lts/*, 16, 14, 12, 10
ARG NODE_VERSION="none"
ARG NODE_VERSION="16"
RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION} 2>&1"; fi
# [Optional] If your pip requirements rarely change, uncomment this section to add them to the image.
+3
View File
@@ -22,6 +22,9 @@
"python.languageServer": "Pylance",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.linting.pylintArgs": [
"--max-line-length=120"
],
"python.formatting.autopep8Path": "/usr/local/py-utils/bin/autopep8",
"python.formatting.blackPath": "/usr/local/py-utils/bin/black",
"python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
+4 -9
View File
@@ -1,21 +1,17 @@
version: "3.3" # optional since v1.27.0
services:
postgres:
image: circleci/postgres:12
image: cimg/postgres:14.2
environment:
POSTGRES_DB: speckle2_test
POSTGRES_PASSWORD: speckle
POSTGRES_USER: speckle
# ports:
# - "5432:5432"
network_mode: host
redis:
image: circleci/redis:6
# ports:
# - "6379:6379"
image: cimg/redis:6.2
network_mode: host
speckle-server:
image: speckle/speckle-server
image: speckle/speckle-server:latest
command: ["bash", "-c", "/wait && node bin/www"]
environment:
POSTGRES_URL: "localhost"
@@ -27,8 +23,7 @@ services:
STRATEGY_LOCAL: "true"
CANONICAL_URL: "http://localhost:3000"
WAIT_HOSTS: localhost:5432, localhost:6379
# ports:
# - "3000:3000"
DISABLE_FILE_UPLOADS: "true"
network_mode: host
specklepy:
-25
View File
@@ -1,25 +0,0 @@
---
name: New issue
about: Create a report to help us improve
title:
labels:
assignees:
---
If it's your first time here - or you forgot about them - make sure you read the [contribution guidelines](CONTRIBUTING.md), and then feel free to delete this line!
### Expected vs. Actual Behavior
Describe the problem here.
### Reproduction Steps & System Config (win, osx, web, etc.)
Let us know how we can reproduce this, and attach relevant files (if any).
### Proposed Solution (if any)
Let us know what how you would solve this.
#### Optional: Affected Projects
Does this issue propagate to other dependencies or dependents? If so, list them here!
+113
View File
@@ -0,0 +1,113 @@
---
name: Bug report
about: Help improve Speckle!
title: ''
labels: bug
assignees: ''
---
<!---
Provide a short summary in the Title above. Examples of good Issue titles:
* "Bug: Error from server when reticulating splines"
* "Bug: Revit crashes when installing connector"
-->
## Prerequisites
<!---
Please answer the following questions before submitting an issue.
-->
- [ ] I read the [contribution guidelines](https://github.com/specklesystems/speckle-server/blob/main/CONTRIBUTING.md)
- [ ] I checked the [documentation](https://speckle.guide/) and found no answer.
- [ ] I checked [existing issues](../issues?q=is%3Aissue) and found no similar issue. <!-- If you do find an existing issue, please show your support by liking it :+1: instead of creating a new issue -->
- [ ] I checked the [community forum](https://speckle.community/) for related discussions and found no answer.
- [ ] I'm reporting the issue to the correct repository (see also [speckle-server](https://github.com/specklesystems/speckle-server), [speckle-sharp](https://github.com/specklesystems/speckle-sharp), [specklepy](https://github.com/specklesystems/specklepy), [speckle-docs](https://github.com/specklesystems/speckle-docs), and [others](https://github.com/orgs/specklesystems/repositories))
## What package are you referring to?
<!---
Is it related to the server (backend) only, or does this bug relate to the frontend, viewer, objectloader or any other package?
-->
## Describe the bug
<!---
A clear and concise description of what the bug is.
-->
## To Reproduce
<!---
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
-->
## Expected behavior
<!---
A clear and concise description of what you expected to happen.
-->
## Screenshots
<!---
If applicable, add screenshots to help explain your problem.
-->
## System Info
If applicable, please fill in the below details - they help a lot!
### Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
### Smartphone (please complete the following information):
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
## Failure Logs
<!---
Please include any relevant log snippets or files here, or upload as a file.
If including inline, please use markdown code block syntax. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks
For example:
```
your log output here
```
-->
## Additional context
<!---
Add any other context about the problem here.
-->
## Proposed Solution (if any)
<!---
Let us know what how you would solve this.
-->
#### Optional: Affected Projects
<!---
Does this issue propagate to other dependencies or dependents? If so, list them here with links!
-->
+71
View File
@@ -0,0 +1,71 @@
---
name: Feature request
about: Suggest an idea for Speckle!
title: ''
labels: enhancement, question
assignees: ''
---
<!---
Provide a short summary in the Title above. Examples of good Issue titles:
* "Enhancement: Connector for Minecraft"
* "Enhancement: Web viewer should support tesseracts"
-->
## Prerequisites
<!---
Please answer the following questions before submitting an issue.
-->
- [ ] I read the [contribution guidelines](https://github.com/specklesystems/speckle-server/blob/main/CONTRIBUTING.md)
- [ ] I checked the [documentation](https://speckle.guide/) and found no answer.
- [ ] I checked [existing issues](../issues?q=is%3Aissue) and found no similar issue. <!-- If you do find an existing issue, please show your support by liking it :+1: instead of creating a new issue -->
- [ ] I checked the [community forum](https://speckle.community/) for related discussions and found no answer.
- [ ] I'm requesting the feature to the correct repository (see also [speckle-server](https://github.com/specklesystems/speckle-server), [speckle-sharp](https://github.com/specklesystems/speckle-sharp), [specklepy](https://github.com/specklesystems/specklepy), [speckle-docs](https://github.com/specklesystems/speckle-docs), and [others](https://github.com/orgs/specklesystems/repositories))
## What package are you referring to?
<!---
Is it related to the server (backend) only, or does this feature request relate to the frontend, viewer, objectloader or any other package?
-->
## Is your feature request related to a problem? Please describe.
<!---
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
-->
## Describe the solution you'd like
<!---
A clear and concise description of what you want to happen.
-->
## Describe alternatives you've considered
<!---
A clear and concise description of any alternative solutions or features you've considered.
-->
## Additional context
<!---
Add any other context or screenshots about the feature request here.
Have you seen this feature implemented in any other software? Can you provide screenshots or links to video or documentation?
What works well about these existing features in other software? What doesn't work well?
-->
## Related issues or community discussions
<!---
Is this feature request related to (but sufficiently distinct from) any existing issues?
Does this feature request require other features to be available beforehand?
Has this feature been discussed in the community forum, please link here? https://speckle.community/
-->
@@ -1,25 +0,0 @@
Description of PR...
## Changes
- Item 1
- Item 2
## Checklist
- [ ] Unit tests
- [ ] Documentation
## References
(optional)
Include **important** links regarding the implementation of this PR.
This usually includes and RFC or an aggregation of issues and/or individual conversations
that helped put this solution together. This helps ensure there is a good aggregation
of resources regarding the implementation.
```text
Fixes #85, Fixes #22, Fixes username/repo#123
Connects #123
```
+102
View File
@@ -0,0 +1,102 @@
<!---
Provide a short summary in the Title above. Examples of good PR titles:
* "Feature: adds metrics to component"
* "Fix: resolves duplication in comment thread"
* "Update: apollo v2.34.0"
-->
## Description & motivation
<!---
Describe your changes, and why you're making them. What benefit will this have to others?
Is this linked to an open Github issue, a thread in Speckle community,
or another pull request? Link it here.
If it is related to a Github issue, and resolves it, please link to the issue number, e.g.:
Fixes #85, Fixes #22, Fixes username/repo#123
Connects #123
-->
## Changes:
<!---
- Item 1
- Item 2
-->
## To-do before merge:
<!---
(Optional -- remove this section if not needed)
Include any notes about things that need to happen before this PR is merged, e.g.:
- [ ] Change the base branch
- [ ] Ensure PR #56 is merged
-->
## Screenshots:
<!---
Include a screenshot the before and after. This can be a screenshot of a plugin, web frontend, or output in a terminal.
-->
## Validation of changes:
<!---
Describe what tests have been added or amended, and why these demonstrate it works and will prevent this feature being accidentally broken by future changes.
-->
## Checklist:
<!---
This checklist is mostly useful as a reminder of small things that can easily be
forgotten it is meant as a helpful tool rather than hoops to jump through.
Put an `x` between the square brackets, e.g. [x], for all the items that apply,
make notes next to any that haven't been addressed, and remove any items that are not relevant to this PR.
-->
- [ ] My pull request follows the guidelines in the [Contributing guide](https://github.com/specklesystems/speckle-server/blob/main/CONTRIBUTING.md)?
- [ ] My pull request does not duplicate any other open [Pull Requests](../../pulls) for the same update/change?
- [ ] My commits are related to the pull request and do not amend unrelated code or documentation.
- [ ] My code follows a similar style to existing code.
- [ ] I have added appropriate tests.
- [ ] I have updated or added relevant documentation.
## References
<!---
(Optional -- remove this section if not needed )
Include **important** links regarding the implementation of this PR.
This usually includes a RFC or an aggregation of issues and/or individual conversations
that helped put this solution together. This helps ensure we retain and share knowledge
regarding the implementation, and may help others understand motivation and design decisions etc..
-->
+4 -5
View File
@@ -32,10 +32,10 @@ jobs:
echo 'PROJECT_ID='$(jq '.data.organization.projectNext.id' project_data.json) >> $GITHUB_ENV
echo 'STATUS_FIELD_ID='$(jq '.data.organization.projectNext.fields.nodes[] | select(.name== "Status") | .id' project_data.json) >> $GITHUB_ENV
echo "$PROJECT_ID"
echo "$STATUS_FIELD_ID"
echo 'DONE_ID='$(jq '.data.organization.projectNext.fields.nodes[] | select(.name== "Status") | .settings | fromjson | .options[] | select(.name== "Done") | .id' project_data.json) >> $GITHUB_ENV
echo "$DONE_ID"
@@ -52,9 +52,9 @@ jobs:
}
}
}' -f project=$PROJECT_ID -f id=$ISSUE_ID --jq '.data.addProjectNextItem.projectNextItem.id')"
echo 'ITEM_ID='$item_id >> $GITHUB_ENV
- name: Update Status
env:
GITHUB_TOKEN: ${{secrets.GHPROJECT_TOKEN}}
@@ -75,4 +75,3 @@ jobs:
}
}
}' -f project=$PROJECT_ID -f status=$STATUS_FIELD_ID -f id=$ITEM_ID -f value=${{ env.DONE_ID }}
+2 -2
View File
@@ -32,7 +32,7 @@ jobs:
echo 'PROJECT_ID='$(jq '.data.organization.projectNext.id' project_data.json) >> $GITHUB_ENV
echo 'STATUS_FIELD_ID='$(jq '.data.organization.projectNext.fields.nodes[] | select(.name== "Status") | .id' project_data.json) >> $GITHUB_ENV
- name: Add Issue to project
env:
GITHUB_TOKEN: ${{secrets.GHPROJECT_TOKEN}}
@@ -46,5 +46,5 @@ jobs:
}
}
}' -f project=$PROJECT_ID -f id=$ISSUE_ID --jq '.data.addProjectNextItem.projectNextItem.id')"
echo 'ITEM_ID='$item_id >> $GITHUB_ENV
+6 -2
View File
@@ -4,6 +4,7 @@
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File",
"type": "python",
@@ -13,10 +14,13 @@
"justMyCode": false
},
{
"name": "Python: Test debug config",
"name": "Pytest",
"type": "python",
"request": "test",
"request": "launch",
"program": "poetry",
"args": ["run", "pytest"],
"console": "integratedTerminal",
"justMyCode": true
}
]
}
+1 -1
View File
@@ -74,7 +74,7 @@ It may be helpful to know where the local accounts and object cache dbs are stor
## Contributing
Please make sure you read the [contribution guidelines](.github/CONTRIBUTING.md) for an overview of the best practices we try to follow.
Please make sure you read the [contribution guidelines](.github/CONTRIBUTING.md) and [code of conduct](.github/CODE_OF_CONDUCT.md) for an overview of the practices we try to follow.
## Community
+12
View File
@@ -0,0 +1,12 @@
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 2.2.+ | :white_check_mark: |
| < 2.2 | :x: |
## Reporting a Vulnerability
Hi! If you've found something off, we'd be more than happy if you would report it via security@speckle.systems. We will work together with you to correctly identify the cause and implement a fix. Thanks for helping make Speckle safer!
+63
View File
@@ -0,0 +1,63 @@
from typing import List
from specklepy.objects import Base
from specklepy.api import operations
from specklepy.transports.sqlite import SQLiteTransport
import time
from pathlib import Path
import os
import string
import random
class Sub(Base):
bar: List[str]
def random_string():
letters = string.ascii_lowercase
return "".join(random.choice(letters) for _ in range(10))
BASE_PATH = SQLiteTransport.get_base_path("Speckle")
def clean_db():
os.remove(Path(BASE_PATH, "Objects.db"))
def one_pass(clean: bool, randomize: bool, child_count: int):
foo = Base()
for i in range(child_count):
stuff = random_string() if randomize else "stuff"
foo[f"@child_{i}"] = Sub(bar=["asdf", "bar", i, stuff])
if clean:
clean_db()
transport = SQLiteTransport()
start = time.time()
hash = operations.send(base=foo, transports=[transport])
send_time = time.time() - start
receive_start = time.time()
operations.receive(hash, transport)
receive_time = time.time() - receive_start
return send_time, receive_time
if __name__ == "__main__":
sample_size = 4
test_permutations = [
(True, True),
(False, False),
(False, True),
(True, False),
]
for clean, randomize in test_permutations:
print(f"CLEAN: {clean}, RANDOMIZE: {randomize}")
for child_count in [10, 100, 1000, 10000]:
print(f"\tCHILD COUNT: {child_count}")
for _ in range(sample_size):
send_time, receive_time = one_pass(clean, randomize, child_count)
print(f"\t\tSend: {send_time} Receive: {receive_time}")
+2 -2
View File
@@ -50,10 +50,10 @@ if __name__ == "__main__":
)
# support for dynamic attributes
custom_sub.extra_extra = "what is this?"
debug(custom_sub.json())
debug(custom_sub)
serialized = operations.serialize(custom_sub)
deserialized = operations.deserialize(serialized)
# the only difference should be between the two data is that the deserialized
# instance id attribute is not None.
debug(deserialized.json())
debug(deserialized)
Generated
+634 -667
View File
File diff suppressed because it is too large Load Diff
+6 -4
View File
@@ -11,11 +11,11 @@ homepage = "https://speckle.systems/"
[tool.poetry.dependencies]
python = "^3.6.5"
python = ">=3.7.2, <4.0"
pydantic = "^1.8.2"
appdirs = "^1.4.4"
gql = {version = ">=3.0.0b1", extras = ["all"], allow-prereleases = true}
ujson = "^4.3.0"
gql = {extras = ["requests", "websockets"], version = "^3.3.0"}
ujson = "^5.3.0"
Deprecated = "^1.2.13"
[tool.poetry.dev-dependencies]
@@ -24,6 +24,8 @@ isort = "^5.7.0"
pytest = "^6.2.2"
pytest-ordering = "^0.6"
pytest-cov = "^3.0.0"
devtools = "^0.8.0"
pylint = "^2.14.4"
[tool.black]
@@ -43,7 +45,7 @@ exclude = '''
'''
include = '\.pyi?$'
line-length = 88
target-version = ["py36", "py37", "py38"]
target-version = ["py37", "py38", "py39", "py310"]
[build-system]
+36 -18
View File
@@ -2,8 +2,8 @@ import re
from warnings import warn
from deprecated import deprecated
from specklepy.api.credentials import Account, get_account_from_token
from specklepy.logging import metrics
from specklepy.logging.exceptions import (
GraphQLException,
SpeckleException,
SpeckleWarning,
)
@@ -57,6 +57,7 @@ class SpeckleClient:
USE_SSL = True
def __init__(self, host: str = DEFAULT_HOST, use_ssl: bool = USE_SSL) -> None:
metrics.track(metrics.CLIENT, custom_props={"name": "create"})
ws_protocol = "ws"
http_protocol = "http"
@@ -79,15 +80,17 @@ class SpeckleClient:
self._init_resources()
# Check compatibility with the server
try:
serverInfo = self.server.get()
if isinstance(serverInfo, Exception):
raise serverInfo
if not isinstance(serverInfo, ServerInfo):
raise Exception("Couldn't get ServerInfo")
except Exception as ex:
raise SpeckleException(f"{self.url} is not a compatible Speckle Server", ex)
# ? Check compatibility with the server - i think we can skip this at this point? save a request
# try:
# server_info = self.server.get()
# if isinstance(server_info, Exception):
# raise server_info
# if not isinstance(server_info, ServerInfo):
# raise Exception("Couldn't get ServerInfo")
# except Exception as ex:
# raise SpeckleException(
# f"{self.url} is not a compatible Speckle Server", ex
# ) from ex
def __repr__(self):
return f"SpeckleClient( server: {self.url}, authenticated: {self.account.token is not None} )"
@@ -114,6 +117,7 @@ class SpeckleClient:
token {str} -- an api token
"""
self.account = get_account_from_token(token, self.url)
metrics.track(metrics.CLIENT, self.account, {"name": "authenticate with token"})
self._set_up_client()
def authenticate_with_account(self, account: Account) -> None:
@@ -123,9 +127,12 @@ class SpeckleClient:
Arguments:
account {Account} -- the account object which can be found with `get_default_account` or `get_local_accounts`
"""
metrics.track(metrics.CLIENT, account, {"name": "authenticate with account"})
self.account = account
self._set_up_client()
def _set_up_client(self) -> None:
metrics.track(metrics.CLIENT, self.account, {"name": "set up client"})
headers = {
"Authorization": f"Bearer {self.account.token}",
"Content-Type": "application/json",
@@ -142,7 +149,7 @@ class SpeckleClient:
self._init_resources()
if isinstance(self.user.get(), GraphQLException):
if self.user.get() is None:
warn(
SpeckleWarning(
f"Possibly invalid token - could not authenticate Speckle Client for server {self.url}"
@@ -153,9 +160,26 @@ class SpeckleClient:
return self.httpclient.execute(query)
def _init_resources(self) -> None:
self.stream = stream.Resource(
self.server = server.Resource(
account=self.account, basepath=self.url, client=self.httpclient
)
server_version = None
try:
server_version = self.server.version()
except:
pass
self.user = user.Resource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.stream = stream.Resource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.commit = commit.Resource(
account=self.account, basepath=self.url, client=self.httpclient
)
@@ -165,12 +189,6 @@ class SpeckleClient:
self.object = object.Resource(
account=self.account, basepath=self.url, client=self.httpclient
)
self.server = server.Resource(
account=self.account, basepath=self.url, client=self.httpclient
)
self.user = user.Resource(
account=self.account, basepath=self.url, client=self.httpclient
)
self.subscribe = subscriptions.Resource(
account=self.account,
basepath=self.ws_url,
+10 -6
View File
@@ -1,5 +1,5 @@
import os
from pydantic import BaseModel, Field
from pydantic import BaseModel, Field # pylint: disable=no-name-in-module
from typing import List, Optional
from specklepy.logging import metrics
from specklepy.api.models import ServerInfo
@@ -16,11 +16,11 @@ class UserInfo(BaseModel):
class Account(BaseModel):
isDefault: bool = False
token: str = None
refreshToken: str = None
token: Optional[str] = None
refreshToken: Optional[str] = None
serverInfo: ServerInfo = Field(default_factory=ServerInfo)
userInfo: UserInfo = Field(default_factory=UserInfo)
id: str = None
id: Optional[str] = None
def __repr__(self) -> str:
return f"Account(email: {self.userInfo.email}, server: {self.serverInfo.url}, isDefault: {self.isDefault})"
@@ -45,12 +45,15 @@ def get_local_accounts(base_path: str = None) -> List[Account]:
List[Account] -- list of all local accounts or an empty list if no accounts were found
"""
account_storage = SQLiteTransport(scope="Accounts", base_path=base_path)
# pylint: disable=protected-access
json_path = os.path.join(account_storage._base_path, "Accounts")
os.makedirs(json_path, exist_ok=True)
json_acct_files = [file for file in os.listdir(json_path) if file.endswith(".json")]
accounts = []
accounts: List[Account] = []
res = account_storage.get_all_objects()
account_storage.close()
if res:
accounts.extend(Account.parse_raw(r[1]) for r in res)
if json_acct_files:
@@ -63,7 +66,8 @@ def get_local_accounts(base_path: str = None) -> List[Account]:
raise SpeckleException(
"Invalid json accounts could not be read. Please fix or remove them.",
ex,
)
) from ex
metrics.track(
metrics.ACCOUNTS,
next(
+59 -8
View File
@@ -3,10 +3,10 @@
# timestamp: 2020-11-17T14:33:13+00:00
from datetime import datetime
from typing import Any, Dict, List, Optional
from typing import List, Optional
from pydantic import BaseModel
from pydantic import BaseModel # pylint: disable=no-name-in-module
class Collaborator(BaseModel):
@@ -23,7 +23,7 @@ class Commit(BaseModel):
authorId: Optional[str]
authorAvatar: Optional[str]
branchName: Optional[str]
createdAt: Optional[str]
createdAt: Optional[datetime]
sourceApplication: Optional[str]
referencedObject: Optional[str]
totalChildrenCount: Optional[int]
@@ -38,7 +38,7 @@ class Commit(BaseModel):
class Commits(BaseModel):
totalCount: Optional[int]
cursor: Optional[Any]
cursor: Optional[datetime]
items: List[Commit] = []
@@ -47,7 +47,7 @@ class Object(BaseModel):
speckleType: Optional[str]
applicationId: Optional[str]
totalChildrenCount: Optional[int]
createdAt: Optional[str]
createdAt: Optional[datetime]
class Branch(BaseModel):
@@ -66,14 +66,18 @@ class Branches(BaseModel):
class Stream(BaseModel):
id: Optional[str]
name: Optional[str]
description: Optional[str]
role: Optional[str]
isPublic: Optional[bool]
createdAt: Optional[str]
updatedAt: Optional[str]
description: Optional[str]
createdAt: Optional[datetime]
updatedAt: Optional[datetime]
collaborators: List[Collaborator] = []
branches: Optional[Branches]
commit: Optional[Commit]
object: Optional[Object]
commentCount: Optional[int]
favoritedDate: Optional[datetime]
favoritesCount: Optional[int]
def __repr__(self):
return f"Stream( id: {self.id}, name: {self.name}, description: {self.description}, isPublic: {self.isPublic})"
@@ -106,6 +110,53 @@ class User(BaseModel):
return self.__repr__()
class PendingStreamCollaborator(BaseModel):
id: Optional[str]
inviteId: Optional[str]
streamId: Optional[str]
streamName: Optional[str]
title: Optional[str]
role: Optional[str]
invitedBy: Optional[User]
user: Optional[User]
token: Optional[str]
def __repr__(self):
return f"PendingStreamCollaborator( inviteId: {self.inviteId}, streamId: {self.streamId}, role: {self.role}, title: {self.title}, invitedBy: {self.user.name if self.user else None})"
def __str__(self) -> str:
return self.__repr__()
class Activity(BaseModel):
actionType: Optional[str]
info: Optional[dict]
userId: Optional[str]
streamId: Optional[str]
resourceId: Optional[str]
resourceType: Optional[str]
message: Optional[str]
time: Optional[datetime]
def __repr__(self) -> str:
return f"Activity( streamId: {self.streamId}, actionType: {self.actionType}, message: {self.message}, userId: {self.userId} )"
def __str__(self) -> str:
return self.__repr__()
class ActivityCollection(BaseModel):
totalCount: Optional[int]
items: Optional[List[Activity]]
cursor: Optional[datetime]
def __repr__(self) -> str:
return f"ActivityCollection( totalCount: {self.totalCount}, items: {len(self.items) if self.items else 0}, cursor: {self.cursor.isoformat() if self.cursor else None} )"
def __str__(self) -> str:
return self.__repr__()
class ServerInfo(BaseModel):
name: Optional[str]
company: Optional[str]
+4 -6
View File
@@ -29,6 +29,9 @@ def send(
message="You need to provide at least one transport: cannot send with an empty transport list and no default cache"
)
if isinstance(transports, AbstractTransport):
transports = [transports]
if transports is None:
metrics.track(metrics.SEND)
transports = []
@@ -40,13 +43,8 @@ def send(
serializer = BaseObjectSerializer(write_transports=transports)
for t in transports:
t.begin_write()
obj_hash, _ = serializer.write_json(base=base)
for t in transports:
t.end_write()
return obj_hash
@@ -72,7 +70,7 @@ def receive(
serializer = BaseObjectSerializer(read_transport=local_transport)
# try local transport first. if the parent is there, we assume all the children are there and continue wth deserialisation using the local transport
# try local transport first. if the parent is there, we assume all the children are there and continue with deserialisation using the local transport
obj_string = local_transport.get_object(obj_id)
if obj_string:
return serializer.read_json(obj_string=obj_string)
+52 -20
View File
@@ -1,10 +1,14 @@
from graphql import DocumentNode
from specklepy.api.credentials import Account
from specklepy.transports.sqlite import SQLiteTransport
from typing import Dict, List
from typing import Any, Dict, List, Optional, Tuple, Type, Union
from gql.client import Client
from gql.gql import gql
from gql.transport.exceptions import TransportQueryError
from specklepy.logging.exceptions import GraphQLException, SpeckleException
from specklepy.logging.exceptions import (
GraphQLException,
SpeckleException,
UnsupportedException,
)
from specklepy.serialization.base_object_serializer import BaseObjectSerializer
@@ -15,28 +19,30 @@ class ResourceBase(object):
basepath: str,
client: Client,
name: str,
methods: list,
server_version: Optional[Tuple[Any, ...]] = None,
) -> None:
self.account = account
self.basepath = basepath
self.client = client
self.name = name
self.methods = methods
self.schema = None
self.server_version = server_version
self.schema: Optional[Type] = None
def _step_into_response(self, response: dict, return_type: str or List):
def _step_into_response(self, response: dict, return_type: Union[str, List, None]):
"""Step into the dict to get the relevant data"""
if return_type is None:
return response
elif isinstance(return_type, str):
if isinstance(return_type, str):
return response[return_type]
elif isinstance(return_type, List):
if isinstance(return_type, List):
for key in return_type:
response = response[key]
return response
def _parse_response(self, response: dict or list, schema=None):
def _parse_response(self, response: Union[dict, list, None], schema=None):
"""Try to create a class instance from the response"""
if response is None:
return None
if isinstance(response, list):
return [self._parse_response(response=r, schema=schema) for r in response]
if schema:
@@ -52,26 +58,26 @@ class ResourceBase(object):
def make_request(
self,
query: gql,
query: DocumentNode,
params: Dict = None,
return_type: str or List = None,
return_type: Union[str, List, None] = None,
schema=None,
parse_response: bool = True,
) -> Dict or GraphQLException:
) -> Any:
"""Executes the GraphQL query"""
try:
response = self.client.execute(query, variable_values=params)
except Exception as e:
if isinstance(e, TransportQueryError):
except Exception as ex:
if isinstance(ex, TransportQueryError):
return GraphQLException(
message=f"Failed to execute the GraphQL {self.name} request. Errors: {e.errors}",
errors=e.errors,
data=e.data,
message=f"Failed to execute the GraphQL {self.name} request. Errors: {ex.errors}",
errors=ex.errors,
data=ex.data,
)
else:
return SpeckleException(
message=f"Failed to execute the GraphQL {self.name} request. Inner exception: {e}",
exception=e,
message=f"Failed to execute the GraphQL {self.name} request. Inner exception: {ex}",
exception=ex,
)
response = self._step_into_response(response=response, return_type=return_type)
@@ -80,3 +86,29 @@ class ResourceBase(object):
return self._parse_response(response=response, schema=schema)
else:
return response
def _check_server_version_at_least(
self, target_version: Tuple[Any, ...], unsupported_message: str = None
):
"""Use this check to guard against making unsupported requests on older servers.
Arguments:
target_version {tuple} -- the minimum server version in the format (major, minor, patch, (tag, build))
eg (2, 6, 3) for a stable build and (2, 6, 4, 'alpha', 4711) for alpha
"""
if not unsupported_message:
unsupported_message = f"The client method used is not supported on Speckle Server versios prior to v{'.'.join(target_version)}"
if self.server_version and self.server_version < target_version:
raise UnsupportedException(unsupported_message)
def _check_invites_supported(self):
"""Invites are only supported for Speckle Server >= 2.6.4.
Use this check to guard against making unsupported requests on older servers.
"""
self._check_server_version_at_least(
(2, 6, 4),
(
"Stream invites are only supported as of Speckle Server v2.6.4. "
"Please update your Speckle Server to use this method or use the `grant_permission` flow instead."
),
)
-1
View File
@@ -1,6 +1,5 @@
from pathlib import Path
import sys
import inspect
import pkgutil
from importlib import import_module
+1 -3
View File
@@ -4,7 +4,6 @@ from specklepy.api.models import Branch
from specklepy.logging import metrics
NAME = "branch"
METHODS = ["create"]
class Resource(ResourceBase):
@@ -16,7 +15,6 @@ class Resource(ResourceBase):
basepath=basepath,
client=client,
name=NAME,
methods=METHODS,
)
self.schema = Branch
@@ -164,7 +162,7 @@ class Resource(ResourceBase):
description {str} -- optional: the updated branch description
Returns:
bool -- True if update is successfull
bool -- True if update is successful
"""
metrics.track(metrics.BRANCH, self.account, {"name": "update"})
query = gql(
-2
View File
@@ -6,7 +6,6 @@ from specklepy.logging import metrics
NAME = "commit"
METHODS = []
class Resource(ResourceBase):
@@ -18,7 +17,6 @@ class Resource(ResourceBase):
basepath=basepath,
client=client,
name=NAME,
methods=METHODS,
)
self.schema = Commit
+3 -3
View File
@@ -4,7 +4,6 @@ from specklepy.api.resource import ResourceBase
from specklepy.objects.base import Base
NAME = "object"
METHODS = []
class Resource(ResourceBase):
@@ -16,7 +15,6 @@ class Resource(ResourceBase):
basepath=basepath,
client=client,
name=NAME,
methods=METHODS,
)
self.schema = Base
@@ -52,7 +50,9 @@ class Resource(ResourceBase):
params = {"stream_id": stream_id, "object_id": object_id}
return self.make_request(
query=query, params=params, return_type=["stream", "object", "data"]
query=query,
params=params,
return_type=["stream", "object", "data"],
)
def create(self, stream_id: str, objects: List[Dict]) -> str:
+41 -3
View File
@@ -1,11 +1,13 @@
from typing import Dict, List
import re
from typing import Any, Dict, List, Tuple
from gql import gql
from specklepy.api.models import ServerInfo
from specklepy.api.resource import ResourceBase
from specklepy.logging import metrics
from specklepy.logging.exceptions import GraphQLException
NAME = "server"
METHODS = ["get", "apps"]
class Resource(ResourceBase):
@@ -17,7 +19,6 @@ class Resource(ResourceBase):
basepath=basepath,
client=client,
name=NAME,
methods=METHODS,
)
def get(self) -> ServerInfo:
@@ -26,6 +27,7 @@ class Resource(ResourceBase):
Returns:
dict -- the server info in dictionary form
"""
metrics.track(metrics.SERVER, self.account, {"name": "get"})
query = gql(
"""
query Server {
@@ -59,12 +61,46 @@ class Resource(ResourceBase):
query=query, return_type="serverInfo", schema=ServerInfo
)
def version(self) -> Tuple[Any, ...]:
"""Get the server version
Returns:
tuple -- the server version in the format (major, minor, patch, (tag, build))
eg (2, 6, 3) for a stable build and (2, 6, 4, 'alpha', 4711) for alpha
"""
# not tracking as it will be called along with other mutations / queries as a check
query = gql(
"""
query Server {
serverInfo {
version
}
}
"""
)
ver = self.make_request(
query=query, return_type=["serverInfo", "version"], parse_response=False
)
if isinstance(ver, Exception):
raise GraphQLException(
f"Could not get server version for {self.basepath}", [ver]
)
# pylint: disable=consider-using-generator; (list comp is faster)
return tuple(
[
int(segment) if segment.isdigit() else segment
for segment in re.split(r"\.|-", ver)
]
)
def apps(self) -> Dict:
"""Get the apps registered on the server
Returns:
dict -- a dictionary of apps registered on the server
"""
metrics.track(metrics.SERVER, self.account, {"name": "apps"})
query = gql(
"""
query Apps {
@@ -98,6 +134,7 @@ class Resource(ResourceBase):
Returns:
str -- the new API token. note: this is the only time you'll see the token!
"""
metrics.track(metrics.SERVER, self.account, {"name": "create_token"})
query = gql(
"""
mutation TokenCreate($token: ApiTokenCreateInput!) {
@@ -123,6 +160,7 @@ class Resource(ResourceBase):
Returns:
bool -- True if the token was successfully deleted
"""
metrics.track(metrics.SERVER, self.account, {"name": "revoke_token"})
query = gql(
"""
mutation TokenRevoke($token: String!) {
+501 -102
View File
@@ -1,31 +1,26 @@
from datetime import datetime, timezone
from typing import List, Optional
from deprecated import deprecated
from gql import gql
from typing import List
from specklepy.logging import metrics
from specklepy.api.models import Stream
from specklepy.api.models import ActivityCollection, PendingStreamCollaborator, Stream
from specklepy.api.resource import ResourceBase
from specklepy.logging.exceptions import UnsupportedException, SpeckleException
NAME = "stream"
METHODS = [
"list",
"create",
"get",
"update",
"delete",
"search",
]
class Resource(ResourceBase):
"""API Access class for streams"""
def __init__(self, account, basepath, client) -> None:
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
account=account,
basepath=basepath,
client=client,
name=NAME,
methods=METHODS,
server_version=server_version,
)
self.schema = Stream
@@ -45,41 +40,45 @@ class Resource(ResourceBase):
query = gql(
"""
query Stream($id: String!, $branch_limit: Int!, $commit_limit: Int!) {
stream(id: $id) {
id
name
description
isPublic
createdAt
updatedAt
collaborators {
id
name
role
avatar
}
branches(limit: $branch_limit) {
totalCount
cursor
items {
stream(id: $id) {
id
name
role
description
commits(limit: $commit_limit) {
totalCount
cursor
items {
isPublic
createdAt
updatedAt
commentCount
favoritesCount
collaborators {
id
referencedObject
message
authorName
authorId
createdAt
}
name
role
avatar
}
}
branches(limit: $branch_limit) {
totalCount
cursor
items {
id
name
description
commits(limit: $commit_limit) {
totalCount
cursor
items {
id
message
authorId
createdAt
authorName
referencedObject
sourceApplication
}
}
}
}
}
}
}
"""
)
@@ -101,34 +100,37 @@ class Resource(ResourceBase):
query = gql(
"""
query User($stream_limit: Int!) {
user {
id
email
name
bio
company
avatar
verified
profiles
role
streams(limit: $stream_limit) {
totalCount
cursor
items {
user {
id
bio
name
description
isPublic
createdAt
updatedAt
collaborators {
id
name
role
}
email
avatar
company
verified
profiles
role
streams(limit: $stream_limit) {
totalCount
cursor
items {
id
name
role
isPublic
createdAt
updatedAt
description
commentCount
favoritesCount
collaborators {
id
name
role
}
}
}
}
}
}
}
"""
)
@@ -190,7 +192,7 @@ class Resource(ResourceBase):
query = gql(
"""
mutation StreamUpdate($stream: StreamUpdateInput!) {
streamUpdate(stream: $stream)
streamUpdate(stream: $stream)
}
"""
)
@@ -221,9 +223,9 @@ class Resource(ResourceBase):
query = gql(
"""
mutation StreamDelete($id: String!) {
streamDelete(id: $id)
streamDelete(id: $id)
}
"""
"""
)
params = {"id": id}
@@ -254,43 +256,44 @@ class Resource(ResourceBase):
query = gql(
"""
query StreamSearch($search_query: String!,$limit: Int!, $branch_limit:Int!, $commit_limit:Int!) {
streams(query: $search_query, limit: $limit) {
items {
id
name
description
isPublic
createdAt
updatedAt
collaborators {
id
name
role
avatar
}
branches(limit: $branch_limit) {
totalCount
cursor
streams(query: $search_query, limit: $limit) {
items {
id
name
description
commits(limit: $commit_limit) {
totalCount
cursor
items {
id
referencedObject
message
authorName
authorId
createdAt
id
name
role
description
isPublic
createdAt
updatedAt
collaborators {
id
name
role
avatar
}
branches(limit: $branch_limit) {
totalCount
cursor
items {
id
name
description
commits(limit: $commit_limit) {
totalCount
cursor
items {
id
referencedObject
message
authorName
authorId
createdAt
}
}
}
}
}
}
}
}
}
}
"""
)
@@ -306,9 +309,51 @@ class Resource(ResourceBase):
query=query, params=params, return_type=["streams", "items"]
)
def favorite(self, stream_id: str, favorited: bool = True):
"""Favorite or unfavorite the given stream.
Arguments:
stream_id {str} -- the id of the stream to favorite / unfavorite
favorited {bool} -- whether to favorite (True) or unfavorite (False) the stream
Returns:
Stream -- the stream with its `id`, `name`, and `favoritedDate`
"""
metrics.track(metrics.STREAM, self.account, {"name": "favorite"})
query = gql(
"""
mutation StreamFavorite($stream_id: String!, $favorited: Boolean!) {
streamFavorite(streamId: $stream_id, favorited: $favorited) {
id
name
favoritedDate
favoritesCount
}
}
"""
)
params = {
"stream_id": stream_id,
"favorited": favorited,
}
return self.make_request(
query=query, params=params, return_type=["streamFavorite"]
)
@deprecated(
version="2.6.4",
reason=(
"As of Speckle Server v2.6.4, this method is deprecated. "
"Users need to be invited and accept the invite before being added to a stream"
),
)
def grant_permission(self, stream_id: str, user_id: str, role: str):
"""Grant permissions to a user on a given stream
Valid for Speckle Server version < 2.6.4
Arguments:
stream_id {str} -- the id of the stream to grant permissions to
user_id {str} -- the id of the user to grant permissions for
@@ -318,6 +363,15 @@ class Resource(ResourceBase):
bool -- True if the operation was successful
"""
metrics.track(metrics.PERMISSION, self.account, {"name": "add", "role": role})
if self.server_version and self.server_version >= (2, 6, 4):
raise UnsupportedException(
(
"Server mutation `grant_permission` is no longer supported as of Speckle Server v2.6.4. "
"Please use the new `update_permission` method to change an existing user's permission "
"or use the `invite` method to invite a user to a stream."
)
)
query = gql(
"""
mutation StreamGrantPermission($permission_params: StreamGrantPermissionInput !) {
@@ -341,6 +395,282 @@ class Resource(ResourceBase):
parse_response=False,
)
def get_all_pending_invites(
self, stream_id: str
) -> List[PendingStreamCollaborator]:
"""Get all of the pending invites on a stream.
You must be a `stream:owner` to query this.
Requires Speckle Server version >= 2.6.4
Arguments:
stream_id {str} -- the stream id from which to get the pending invites
Returns:
List[PendingStreamCollaborator] -- a list of pending invites for the specified stream
"""
metrics.track(metrics.INVITE, self.account, {"name": "get"})
self._check_invites_supported()
query = gql(
"""
query StreamInvites($streamId: String!) {
stream(id: $streamId){
pendingCollaborators {
id
token
inviteId
streamId
streamName
title
role
invitedBy{
id
name
company
avatar
}
user {
id
name
company
avatar
}
}
}
}
"""
)
params = {"streamId": stream_id}
return self.make_request(
query=query,
params=params,
return_type=["stream", "pendingCollaborators"],
schema=PendingStreamCollaborator,
)
def invite(
self,
stream_id: str,
email: str = None,
user_id: str = None,
role: str = "stream:contributor", # should default be reviewer?
message: str = None,
):
"""Invite someone to a stream using either their email or user id
Requires Speckle Server version >= 2.6.4
Arguments:
stream_id {str} -- the id of the stream to invite the user to
email {str} -- the email of the user to invite (use this OR `user_id`)
user_id {str} -- the id of the user to invite (use this OR `email`)
role {str} -- the role to assing to the user (defaults to `stream:contributor`)
message {str} -- a message to send along with this invite to the specified user
Returns:
bool -- True if the operation was successful
"""
metrics.track(metrics.INVITE, self.account, {"name": "create"})
self._check_invites_supported()
if email is None and user_id is None:
raise SpeckleException(
"You must provide either an email or a user id to use the `stream.invite` method"
)
query = gql(
"""
mutation StreamInviteCreate($input: StreamInviteCreateInput!) {
streamInviteCreate(input: $input)
}
"""
)
params = {
"email": email,
"userId": user_id,
"streamId": stream_id,
"message": message,
"role": role,
}
params = {"input": {k: v for k, v in params.items() if v is not None}}
return self.make_request(
query=query,
params=params,
return_type="streamInviteCreate",
parse_response=False,
)
def invite_batch(
self,
stream_id: str,
emails: List[str] = None,
user_ids: List[None] = None,
message: str = None,
) -> bool:
"""Invite a batch of users to a specified stream.
Requires Speckle Server version >= 2.6.4
Arguments:
stream_id {str} -- the id of the stream to invite the user to
emails {List[str]} -- the email of the user to invite (use this and/or `user_ids`)
user_id {List[str]} -- the id of the user to invite (use this and/or `emails`)
message {str} -- a message to send along with this invite to the specified user
Returns:
bool -- True if the operation was successful
"""
metrics.track(metrics.INVITE, self.account, {"name": "batch create"})
self._check_invites_supported()
if emails is None and user_ids is None:
raise SpeckleException(
"You must provide either an email or a user id to use the `stream.invite` method"
)
query = gql(
"""
mutation StreamInviteBatchCreate($input: [StreamInviteCreateInput!]!) {
streamInviteBatchCreate(input: $input)
}
"""
)
email_invites = [
{"streamId": stream_id, "message": message, "email": email}
for email in emails
if emails is not None
]
user_invites = [
{"streamId": stream_id, "message": message, "userId": user_id}
for user_id in user_ids
if user_ids is not None
]
params = {"input": [*email_invites, *user_invites]}
return self.make_request(
query=query,
params=params,
return_type="streamInviteBatchCreate",
parse_response=False,
)
def invite_cancel(self, stream_id: str, invite_id: str) -> bool:
"""Cancel an existing stream invite
Requires Speckle Server version >= 2.6.4
Arguments:
stream_id {str} -- the id of the stream invite
invite_id {str} -- the id of the invite to use
Returns:
bool -- true if the operation was successful
"""
metrics.track(metrics.INVITE, self.account, {"name": "cancel"})
self._check_invites_supported()
query = gql(
"""
mutation StreamInviteCancel($streamId: String!, $inviteId: String!) {
streamInviteCancel(streamId: $streamId, inviteId: $inviteId)
}
"""
)
params = {"streamId": stream_id, "inviteId": invite_id}
return self.make_request(
query=query,
params=params,
return_type="streamInviteCancel",
parse_response=False,
)
def invite_use(self, stream_id: str, token: str, accept: bool = True) -> bool:
"""Accept or decline a stream invite
Requires Speckle Server version >= 2.6.4
Arguments:
stream_id {str} -- the id of the stream for which the user has a pending invite
token {str} -- the token of the invite to use
accept {bool} -- whether or not to accept the invite (defaults to True)
Returns:
bool -- true if the operation was successful
"""
metrics.track(metrics.INVITE, self.account, {"name": "use"})
self._check_invites_supported()
query = gql(
"""
mutation StreamInviteUse($accept: Boolean!, $streamId: String!, $token: String!) {
streamInviteUse(accept: $accept, streamId: $streamId, token: $token)
}
"""
)
params = {"streamId": stream_id, "token": token, "accept": accept}
return self.make_request(
query=query,
params=params,
return_type="streamInviteUse",
parse_response=False,
)
def update_permission(self, stream_id: str, user_id: str, role: str):
"""Updates permissions for a user on a given stream
Valid for Speckle Server >=2.6.4
Arguments:
stream_id {str} -- the id of the stream to grant permissions to
user_id {str} -- the id of the user to grant permissions for
role {str} -- the role to grant the user
Returns:
bool -- True if the operation was successful
"""
metrics.track(
metrics.PERMISSION, self.account, {"name": "update", "role": role}
)
if self.server_version and self.server_version < (2, 6, 4):
raise UnsupportedException(
(
"Server mutation `update_permission` is only supported as of Speckle Server v2.6.4. "
"Please update your Speckle Server to use this method or use the `grant_permission` method instead."
)
)
query = gql(
"""
mutation StreamUpdatePermission($permission_params: StreamUpdatePermissionInput !) {
streamUpdatePermission(permissionParams: $permission_params)
}
"""
)
params = {
"permission_params": {
"streamId": stream_id,
"userId": user_id,
"role": role,
}
}
return self.make_request(
query=query,
params=params,
return_type="streamUpdatePermission",
parse_response=False,
)
def revoke_permission(self, stream_id: str, user_id: str):
"""Revoke permissions from a user on a given stream
@@ -368,3 +698,72 @@ class Resource(ResourceBase):
return_type="streamRevokePermission",
parse_response=False,
)
def activity(
self,
stream_id: str,
action_type: str = None,
limit: int = 20,
before: datetime = None,
after: datetime = None,
cursor: datetime = None,
):
"""
Get the activity from a given stream in an Activity collection. Step into the activity `items` for the list of activity.
Note: all timestamps arguments should be `datetime` of any tz as they will be converted to UTC ISO format strings
stream_id {str} -- the id of the stream to get activity from
action_type {str} -- filter results to a single action type (eg: `commit_create` or `commit_receive`)
limit {int} -- max number of Activity items to return
before {datetime} -- latest cutoff for activity (ie: return all activity _before_ this time)
after {datetime} -- oldest cutoff for activity (ie: return all activity _after_ this time)
cursor {datetime} -- timestamp cursor for pagination
"""
query = gql(
"""
query StreamActivity($stream_id: String!, $action_type: String, $before:DateTime, $after: DateTime, $cursor: DateTime, $limit: Int){
stream(id: $stream_id) {
activity(actionType: $action_type, before: $before, after: $after, cursor: $cursor, limit: $limit) {
totalCount
cursor
items {
actionType
info
userId
streamId
resourceId
resourceType
message
time
}
}
}
}
"""
)
try:
params = {
"stream_id": stream_id,
"limit": limit,
"action_type": action_type,
"before": before.astimezone(timezone.utc).isoformat()
if before
else before,
"after": after.astimezone(timezone.utc).isoformat() if after else after,
"cursor": cursor.astimezone(timezone.utc).isoformat()
if cursor
else cursor,
}
except AttributeError as e:
raise SpeckleException(
"Could not get stream activity - `before`, `after`, and `cursor` must be in `datetime` format if provided",
ValueError,
) from e
return self.make_request(
query=query,
params=params,
return_type=["stream", "activity"],
schema=ActivityCollection,
)
+5 -10
View File
@@ -1,16 +1,12 @@
from typing import Callable, Dict, List
from typing import Callable, Dict, List, Union
from functools import wraps
from gql import gql
from graphql import DocumentNode
from specklepy.api.resource import ResourceBase
from specklepy.api.resources.stream import Stream
from specklepy.logging.exceptions import SpeckleException
NAME = "subscribe"
METHODS = [
"stream_added",
"stream_updated",
"stream_removed",
]
def check_wsclient(function):
@@ -35,7 +31,6 @@ class Resource(ResourceBase):
basepath=basepath,
client=client,
name=NAME,
methods=METHODS,
)
@check_wsclient
@@ -109,15 +104,15 @@ class Resource(ResourceBase):
@check_wsclient
async def subscribe(
self,
query: gql,
query: DocumentNode,
params: Dict = None,
callback: Callable = None,
return_type: str or List = None,
return_type: Union[str, List] = None,
schema=None,
parse_response: bool = True,
):
# if self.client.transport.websocket is None:
# TODO: add multiple subs to the same ws connection
# TODO: add multiple subs to the same ws connection
async with self.client as session:
async for res in session.subscribe(query, variable_values=params):
res = self._step_into_response(response=res, return_type=return_type)
+160 -7
View File
@@ -1,24 +1,24 @@
from typing import List, Optional, Union
from datetime import datetime, timezone
from gql import gql
from specklepy.logging import metrics
from specklepy.logging.exceptions import SpeckleException
from typing import List
from gql import gql
from specklepy.api.resource import ResourceBase
from specklepy.api.models import User
from specklepy.api.models import ActivityCollection, PendingStreamCollaborator, User
NAME = "user"
METHODS = ["get", "search", "update"]
class Resource(ResourceBase):
"""API Access class for users"""
def __init__(self, account, basepath, client) -> None:
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
account=account,
basepath=basepath,
client=client,
name=NAME,
methods=METHODS,
server_version=server_version,
)
self.schema = User
@@ -54,7 +54,9 @@ class Resource(ResourceBase):
return self.make_request(query=query, params=params, return_type="user")
def search(self, search_query: str, limit: int = 25) -> List[User]:
def search(
self, search_query: str, limit: int = 25
) -> Union[List[User], SpeckleException]:
"""Searches for user by name or email. The search query must be at least 3 characters long
Arguments:
@@ -125,3 +127,154 @@ class Resource(ResourceBase):
return self.make_request(
query=query, params=params, return_type="userUpdate", parse_response=False
)
def activity(
self,
user_id: str = None,
limit: int = 20,
action_type: str = None,
before: datetime = None,
after: datetime = None,
cursor: datetime = None,
):
"""
Get the activity from a given stream in an Activity collection. Step into the activity `items` for the list of activity.
If no id argument is provided, will return the current authenticated user's activity (as extracted from the authorization header).
Note: all timestamps arguments should be `datetime` of any tz as they will be converted to UTC ISO format strings
user_id {str} -- the id of the user to get the activity from
action_type {str} -- filter results to a single action type (eg: `commit_create` or `commit_receive`)
limit {int} -- max number of Activity items to return
before {datetime} -- latest cutoff for activity (ie: return all activity _before_ this time)
after {datetime} -- oldest cutoff for activity (ie: return all activity _after_ this time)
cursor {datetime} -- timestamp cursor for pagination
"""
query = gql(
"""
query UserActivity($user_id: String, $action_type: String, $before:DateTime, $after: DateTime, $cursor: DateTime, $limit: Int){
user(id: $user_id) {
activity(actionType: $action_type, before: $before, after: $after, cursor: $cursor, limit: $limit) {
totalCount
cursor
items {
actionType
info
userId
streamId
resourceId
resourceType
message
time
}
}
}
}
"""
)
params = {
"user_id": user_id,
"limit": limit,
"action_type": action_type,
"before": before.astimezone(timezone.utc).isoformat() if before else before,
"after": after.astimezone(timezone.utc).isoformat() if after else after,
"cursor": cursor.astimezone(timezone.utc).isoformat() if cursor else cursor,
}
return self.make_request(
query=query,
params=params,
return_type=["user", "activity"],
schema=ActivityCollection,
)
def get_all_pending_invites(self) -> List[PendingStreamCollaborator]:
"""Get all of the active user's pending stream invites
Requires Speckle Server version >= 2.6.4
Returns:
List[PendingStreamCollaborator] -- a list of pending invites for the current user
"""
metrics.track(metrics.INVITE, self.account, {"name": "get"})
self._check_invites_supported()
query = gql(
"""
query StreamInvites {
streamInvites{
id
token
inviteId
streamId
streamName
title
role
invitedBy {
id
name
company
avatar
}
}
}
"""
)
return self.make_request(
query=query,
return_type="streamInvites",
schema=PendingStreamCollaborator,
)
def get_pending_invite(
self, stream_id: str, token: str = None
) -> Optional[PendingStreamCollaborator]:
"""Get a particular pending invite for the active user on a given stream.
If no invite_id is provided, any valid invite will be returned.
Requires Speckle Server version >= 2.6.4
Arguments:
stream_id {str} -- the id of the stream to look for invites on
token {str} -- the token of the invite to look for (optional)
Returns:
PendingStreamCollaborator -- the invite for the given stream (or None if it isn't found)
"""
metrics.track(metrics.INVITE, self.account, {"name": "get"})
self._check_invites_supported()
query = gql(
"""
query StreamInvite($streamId: String!, $token: String) {
streamInvite(streamId: $streamId, token: $token) {
id
token
streamId
streamName
title
role
invitedBy {
id
name
company
avatar
}
}
}
"""
)
params = {"streamId": stream_id}
if token:
params["token"] = token
return self.make_request(
query=query,
params=params,
return_type="streamInvite",
schema=PendingStreamCollaborator,
)
+1 -1
View File
@@ -445,7 +445,7 @@ input ServerInfoUpdateInput {
stream( id: String! ): Stream
"""
All the streams of the current user, pass in the `query` parameter to seach by name, description or ID.
All the streams of the current user, pass in the `query` parameter to search by name, description or ID.
"""
streams( query: String, limit: Int = 25, cursor: String ): StreamCollection
@hasScope(scope: "streams:read")
+6 -2
View File
@@ -21,7 +21,7 @@ class StreamWrapper:
local account for the server.
```py
from specklepy.api.credentials import StreamWrapper
from specklepy.api.wrapper import StreamWrapper
# provide any stream, branch, commit, object, or globals url
wrapper = StreamWrapper("https://speckle.xyz/streams/3073b96e86/commits/604bea8cc6")
@@ -98,6 +98,10 @@ class StreamWrapper:
f"Cannot parse {url} into a stream wrapper class - no stream id found."
)
@property
def server_url(self):
return f"{'https' if self.use_ssl else 'http'}://{self.host}"
def get_account(self, token: str = None) -> Account:
"""
Gets an account object for this server from the local accounts db (added via Speckle Manager or a json file)
@@ -111,7 +115,7 @@ class StreamWrapper:
)
if not self._account:
self._account = get_account_from_token(token, self.host)
self._account = get_account_from_token(token, self.server_url)
if self._client:
self._client.authenticate_with_account(self._account)
+17 -6
View File
@@ -1,8 +1,9 @@
from typing import Any, List
from typing import Any, List, Optional
class SpeckleException(Exception):
def __init__(self, message: str, exception: Exception = None) -> None:
super().__init__()
self.message = message
self.exception = exception
@@ -11,17 +12,19 @@ class SpeckleException(Exception):
class SerializationException(SpeckleException):
def __init__(self, message: str, object: Any, exception: Exception = None) -> None:
super().__init__(message=message)
self.object = object
self.unhandled_type = type(object)
def __init__(self, message: str, obj: Any, exception: Exception = None) -> None:
super().__init__(message=message, exception=exception)
self.obj = obj
self.unhandled_type = type(obj)
def __str__(self) -> str:
return f"SpeckleException: Could not serialize object of type {self.unhandled_type}"
class GraphQLException(SpeckleException):
def __init__(self, message: str, errors: List, data=None) -> None:
def __init__(
self, message: str, errors: Optional[List[Any]] = None, data=None
) -> None:
super().__init__(message=message)
self.errors = errors
self.data = data
@@ -30,6 +33,14 @@ class GraphQLException(SpeckleException):
return f"GraphQLException: {self.message}"
class UnsupportedException(SpeckleException):
def __init__(self, message: str) -> None:
super().__init__(message=message)
def __str__(self) -> str:
return f"UnsupportedException: {self.message}"
class SpeckleWarning(Warning):
def __init__(self, *args: object) -> None:
super().__init__(*args)
+20 -11
View File
@@ -1,12 +1,13 @@
import json
import os
import socket
import sys
import queue
import hashlib
import getpass
import logging
import requests
import threading
import platform
import contextlib
"""
Anonymous telemetry to help us understand how to make a better Speckle.
@@ -14,6 +15,7 @@ This really helps us to deliver a better open source project and product!
"""
TRACK = True
HOST_APP = "python"
HOST_APP_VERSION = f"python {'.'.join(map(str, sys.version_info[:2]))}"
PLATFORMS = {"win32": "Windows", "cygwin": "Windows", "darwin": "Mac OS X"}
LOG = logging.getLogger(__name__)
@@ -24,9 +26,12 @@ RECEIVE = "Receive"
SEND = "Send"
STREAM = "Stream Action"
PERMISSION = "Permission Action"
INVITE = "Invite Action"
COMMIT = "Commit Action"
BRANCH = "Branch Action"
USER = "User Action"
SERVER = "Server Action"
CLIENT = "Speckle Client"
STREAM_WRAPPER = "Stream Wrapper"
ACCOUNTS = "Get Local Accounts"
@@ -45,9 +50,10 @@ def enable():
TRACK = True
def set_host_app(host_app: str):
global HOST_APP
def set_host_app(host_app: str, host_app_version: str = None):
global HOST_APP, HOST_APP_VERSION
HOST_APP = host_app
HOST_APP_VERSION = host_app_version or HOST_APP_VERSION
def track(action: str, account: "Account" = None, custom_props: dict = None):
@@ -62,6 +68,7 @@ def track(action: str, account: "Account" = None, custom_props: dict = None):
"server_id": METRICS_TRACKER.last_server,
"token": METRICS_TRACKER.analytics_token,
"hostApp": HOST_APP,
"hostAppVersion": HOST_APP_VERSION,
"$os": METRICS_TRACKER.platform,
"type": "action",
},
@@ -72,7 +79,7 @@ def track(action: str, account: "Account" = None, custom_props: dict = None):
METRICS_TRACKER.queue.put_nowait(event_params)
except Exception as ex:
# wrapping this whole thing in a try except as we never want a failure here to annoy users!
LOG.error("Error queueing metrics request: " + str(ex))
LOG.error(f"Error queueing metrics request: {str(ex)}")
def initialise_tracker(account: "Account" = None):
@@ -98,8 +105,7 @@ class Singleton(type):
class MetricsTracker(metaclass=Singleton):
analytics_url = "https://analytics.speckle.systems/track?ip=1"
analytics_token = "acd87c5a50b56df91a795e999812a3a4"
user_ip = None
last_user = None
last_user = ""
last_server = None
platform = None
sending_thread = None
@@ -111,12 +117,15 @@ class MetricsTracker(metaclass=Singleton):
)
self.platform = PLATFORMS.get(sys.platform, "linux")
self.sending_thread.start()
self.user_ip = socket.gethostbyname(socket.gethostname())
with contextlib.suppress(Exception):
node, user = platform.node(), getpass.getuser()
if node and user:
self.last_user = f"@{self.hash(f'{node}-{user}')}"
def set_last_user(self, email: str):
if not email:
return
self.last_user = "@" + self.hash(email)
self.last_user = f"@{self.hash(email)}"
def set_last_server(self, server: str):
if not server:
@@ -134,6 +143,6 @@ class MetricsTracker(metaclass=Singleton):
try:
session.post(self.analytics_url, json=event_params)
except Exception as ex:
LOG.error("Error sending metrics request: " + str(ex))
LOG.error(f"Error sending metrics request: {str(ex)}")
self.queue.task_done()
+20 -13
View File
@@ -9,6 +9,9 @@ from typing import (
Type,
get_type_hints,
)
import contextlib
from enum import EnumMeta
from warnings import warn
from specklepy.logging.exceptions import SpeckleException
@@ -89,12 +92,19 @@ class _RegisteringBase:
speckle_type: ClassVar[str]
_type_registry: ClassVar[Dict[str, "Base"]] = {}
_attr_types: ClassVar[Dict[str, Type]] = {}
# dict of chunkable props and their max chunk size
_chunkable: Dict[str, int] = {}
_chunk_size_default: int = 1000
_detachable: Set[str] = set() # list of defined detachable props
_serialize_ignore: Set[str] = set()
class Config:
validate_assignment = True
@classmethod
def get_registered_type(cls, speckle_type: str) -> Optional[Type["Base"]]:
def get_registered_type(
cls, speckle_type: str
) -> Union["Base", Type["Base"], None]:
"""Get the registered type from the protected mapping via the `speckle_type`"""
return cls._type_registry.get(speckle_type, None)
@@ -139,12 +149,7 @@ class Base(_RegisteringBase):
id: Optional[str] = None
totalChildrenCount: Optional[int] = None
applicationId: Optional[str] = None
_units: str = "m"
# dict of chunkable props and their max chunk size
_chunkable: Dict[str, int] = {}
_chunk_size_default: int = 1000
_detachable: Set[str] = set() # list of defined detachable props
_serialize_ignore: Set[str] = set()
_units: Union[str, None] = None
def __init__(self, **kwargs) -> None:
super().__init__()
@@ -248,12 +253,15 @@ class Base(_RegisteringBase):
types = getattr(self, "_attr_types", {})
t = types.get(name, None)
if t is None:
if t is None or t is Any:
return value
if value is None:
return None
if isinstance(t, EnumMeta) and (value in t._value2member_map_):
return t(value)
if t.__module__ == "typing":
origin = getattr(t, "__origin__")
t = (
@@ -275,13 +283,11 @@ class Base(_RegisteringBase):
if isinstance(t, tuple):
t = t[0]
try:
with contextlib.suppress(ValueError):
if t is float:
return float(value)
if t is str and value:
return str(value)
except ValueError:
pass
raise SpeckleException(
f"Cannot set '{self.__class__.__name__}.{name}': it expects type '{t.__name__}', but received type '{type(value).__name__}'"
@@ -327,7 +333,7 @@ class Base(_RegisteringBase):
def get_serializable_attributes(self) -> List[str]:
"""Get the attributes that should be serialized"""
return list(set(self.get_member_names()) - self._serialize_ignore)
return sorted(list(set(self.get_member_names()) - self._serialize_ignore))
def get_typed_member_names(self) -> List[str]:
"""Get all of the names of the defined (typed) properties of this object"""
@@ -374,6 +380,7 @@ class Base(_RegisteringBase):
)
def _handle_object_count(self, obj: Any, parsed: List) -> int:
# pylint: disable=isinstance-second-argument-not-valid-type
count = 0
if obj is None:
return count
@@ -402,7 +409,7 @@ Base.update_forward_refs()
class DataChunk(Base, speckle_type="Speckle.Core.Models.DataChunk"):
data: List[Any] = None
data: Union[List[Any], None] = None
def __init__(self) -> None:
super().__init__()
+14 -19
View File
@@ -1,5 +1,5 @@
from enum import Enum
from typing import Any, Callable, List, Type
from typing import Any, Callable, List, Type, Dict
from specklepy.logging.exceptions import SpeckleException
from specklepy.objects.base import Base
@@ -43,8 +43,8 @@ def curve_from_list(args: List[float]):
class ObjectArray:
def __init__(self) -> None:
self.data = []
def __init__(self, data: list = None) -> None:
self.data = data or []
@classmethod
def from_objects(cls, objects: List[Base]) -> "ObjectArray":
@@ -60,38 +60,34 @@ class ObjectArray:
"All objects in chunk should have the same speckle_type. "
f"Found {speckle_type} and {obj.speckle_type}"
)
data_list.encode_object(object=obj)
data_list.encode_object(obj=obj)
return data_list
@staticmethod
def decode_data(
data: List[Any], decoder: Callable[[List[Any]], Base]
data: List[Any], decoder: Callable[[List[Any]], Base], **kwargs: Dict[str, Any]
) -> List[Base]:
bases = []
if not data:
return bases
index = 0
while index < len(data):
item_length = data[index]
item_length = int(data[index])
item_start = index + 1
item_end = item_start + item_length
item_data = data[item_start:item_end]
index = item_end
# TODO: investigate what's going on w this fail
try:
decoded_data = decoder(item_data)
bases.append(decoded_data)
except ValueError:
continue
decoded_data = decoder(item_data, **kwargs)
bases.append(decoded_data)
return bases
def decode(self, decoder: Callable[[List[Any]], Any]):
return self.decode_data(data=self.data, decoder=decoder)
def decode(self, decoder: Callable[[List[Any]], Any], **kwargs: Dict[str, Any]):
return self.decode_data(data=self.data, decoder=decoder, **kwargs)
def encode_object(self, object: Base):
encoded = object.to_list()
def encode_object(self, obj: Base):
encoded = obj.to_list()
encoded.insert(0, len(encoded))
self.data.extend(encoded)
@@ -128,8 +124,7 @@ class CurveArray(ObjectArray):
@classmethod
def _curve_decoder(cls, data: List[float]) -> Base:
crv_array = cls()
crv_array.data = data
crv_array = cls(data)
return crv_array.to_curve()
def to_curves(self) -> List[Base]:
+10 -1
View File
@@ -1,5 +1,6 @@
from specklepy.objects.geometry import Point
from enum import Enum
from typing import List
from specklepy.objects.geometry import Point
from .base import Base
@@ -19,11 +20,19 @@ class FakeGeo(Base, chunkable={"dots": 50}, detachable={"pointslist"}):
dots: List[int] = None
class FakeDirection(Enum):
NORTH = 1
EAST = 2
SOUTH = 3
WEST = 4
class FakeMesh(FakeGeo, chunkable=CHUNKABLE_PROPS, detachable=DETACHABLE):
vertices: List[float] = None
faces: List[int] = None
colors: List[int] = None
textureCoordinates: List[float] = None
cardinal_dir: FakeDirection = None
test_bases: List[Base] = None
detach_this: Base = None
detached_list: List[Base] = None
+301 -171
View File
@@ -64,19 +64,21 @@ class Plane(Base, speckle_type=GEOMETRY + "Plane"):
@classmethod
def from_list(cls, args: List[Any]) -> "Plane":
return cls(
origin=Point.from_list(args[0:3]),
origin=Point.from_list(args[:3]),
normal=Vector.from_list(args[3:6]),
xdir=Vector.from_list(args[6:9]),
ydir=Vector.from_list(args[9:12]),
units=get_units_from_encoding(args[-1]),
)
def to_list(self) -> List[Any]:
encoded = []
encoded.extend(self.origin.to_list())
encoded.extend(self.normal.to_list())
encoded.extend(self.xdir.to_list())
encoded.extend(self.ydir.to_list())
return encoded
return [
*self.origin.to_list(),
*self.normal.to_list(),
*self.xdir.to_list(),
*self.ydir.to_list(),
get_encoding_from_units(self.units),
]
class Box(Base, speckle_type=GEOMETRY + "Box"):
@@ -98,17 +100,21 @@ class Line(Base, speckle_type=GEOMETRY + "Line"):
@classmethod
def from_list(cls, args: List[Any]) -> "Line":
return cls(
start=Point.from_list(args[0:3]),
end=Point.from_list(args[3:6]),
domain=Interval.from_list(args[6:9]),
start=Point.from_list(args[1:4]),
end=Point.from_list(args[4:7]),
domain=Interval.from_list(args[7:10]),
units=get_units_from_encoding(args[-1]),
)
def to_list(self) -> List[Any]:
encoded = []
encoded.extend(self.start.to_list())
encoded.extend(self.end.to_list())
encoded.extend(self.domain.to_list())
return encoded
domain = self.domain.to_list() if self.domain else [0, 1]
return [
CurveTypeEncoding.Line.value,
*self.start.to_list(),
*self.end.to_list(),
*domain,
get_encoding_from_units(self.units),
]
class Arc(Base, speckle_type=GEOMETRY + "Arc"):
@@ -134,20 +140,26 @@ class Arc(Base, speckle_type=GEOMETRY + "Arc"):
angleRadians=args[4],
domain=Interval.from_list(args[5:7]),
plane=Plane.from_list(args[7:20]),
startPoint=Point.from_list(args[20:23]),
midPoint=Point.from_list(args[23:26]),
endPoint=Point.from_list(args[26:29]),
units=get_units_from_encoding(args[-1]),
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(CurveTypeEncoding.Arc.value)
encoded.append(self.radius)
encoded.append(self.startAngle)
encoded.append(self.endAngle)
encoded.append(self.angleRadians)
encoded.extend(self.domain.to_list())
encoded.extend(self.plane.to_list())
encoded.append(get_encoding_from_units(self.units))
return encoded
return [
CurveTypeEncoding.Arc.value,
self.radius,
self.startAngle,
self.endAngle,
self.angleRadians,
*self.domain.to_list(),
*self.plane.to_list(),
*self.startPoint.to_list(),
*self.midPoint.to_list(),
*self.endPoint.to_list(),
get_encoding_from_units(self.units),
]
class Circle(Base, speckle_type=GEOMETRY + "Circle"):
@@ -168,13 +180,13 @@ class Circle(Base, speckle_type=GEOMETRY + "Circle"):
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(CurveTypeEncoding.Circle.value)
encoded.append(self.radius),
encoded.extend(self.domain.to_list())
encoded.extend(self.plane.to_list())
encoded.append(get_encoding_from_units(self.units))
return encoded
return [
CurveTypeEncoding.Circle.value,
self.radius,
*self.domain.to_list(),
*self.plane.to_list(),
get_encoding_from_units(self.units),
]
class Ellipse(Base, speckle_type=GEOMETRY + "Ellipse"):
@@ -198,14 +210,14 @@ class Ellipse(Base, speckle_type=GEOMETRY + "Ellipse"):
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(CurveTypeEncoding.Ellipse.value)
encoded.append(self.firstRadius)
encoded.append(self.secondRadius)
encoded.extend(self.domain.to_list())
encoded.extend(self.plane.to_list())
encoded.append(get_encoding_from_units(self.units))
return encoded
return [
CurveTypeEncoding.Ellipse.value,
self.firstRadius,
self.secondRadius,
*self.domain.to_list(),
*self.plane.to_list(),
get_encoding_from_units(self.units),
]
class Polyline(Base, speckle_type=GEOMETRY + "Polyline", chunkable={"value": 20000}):
@@ -237,14 +249,14 @@ class Polyline(Base, speckle_type=GEOMETRY + "Polyline", chunkable={"value": 200
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(CurveTypeEncoding.Polyline.value)
encoded.append(int(self.closed))
encoded.extend(self.domain.to_list())
encoded.append(len(self.value))
encoded.extend(self.value)
encoded.append(get_encoding_from_units(self.units))
return encoded
return [
CurveTypeEncoding.Polyline.value,
int(self.closed),
*self.domain.to_list(),
len(self.value),
*self.value,
get_encoding_from_units(self.units),
]
def as_points(self) -> List[Point]:
"""Converts the `value` attribute to a list of Points"""
@@ -293,9 +305,9 @@ class Curve(
@classmethod
def from_list(cls, args: List[Any]) -> "Curve":
point_count = args[7]
weights_count = args[8]
knots_count = args[9]
point_count = int(args[7])
weights_count = int(args[8])
knots_count = int(args[9])
points_start = 10
weights_start = 10 + point_count
@@ -303,7 +315,7 @@ class Curve(
knots_end = knots_start + knots_count
return cls(
degree=args[1],
degree=int(args[1]),
periodic=bool(args[2]),
rational=bool(args[3]),
closed=bool(args[4]),
@@ -315,21 +327,21 @@ class Curve(
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(CurveTypeEncoding.Curve.value)
encoded.append(self.degree)
encoded.append(int(self.periodic))
encoded.append(int(self.rational))
encoded.append(int(self.closed))
encoded.extend(self.domain.to_list())
encoded.append(len(self.points))
encoded.append(len(self.weights))
encoded.append(len(self.knots))
encoded.extend(self.points)
encoded.extend(self.weights)
encoded.extend(self.knots)
encoded.append(get_encoding_from_units(self.units))
return encoded
return [
CurveTypeEncoding.Curve.value,
self.degree,
int(self.periodic),
int(self.rational),
int(self.closed),
*self.domain.to_list(),
len(self.points),
len(self.weights),
len(self.knots),
*self.points,
*self.weights,
*self.knots,
get_encoding_from_units(self.units),
]
class Polycurve(Base, speckle_type=GEOMETRY + "Polycurve"):
@@ -342,8 +354,7 @@ class Polycurve(Base, speckle_type=GEOMETRY + "Polycurve"):
@classmethod
def from_list(cls, args: List[Any]) -> "Polycurve":
curve_arrays = CurveArray()
curve_arrays.data = args[4:-1]
curve_arrays = CurveArray(args[5:-1])
return cls(
closed=bool(args[1]),
domain=Interval.from_list(args[2:4]),
@@ -352,14 +363,15 @@ class Polycurve(Base, speckle_type=GEOMETRY + "Polycurve"):
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(CurveTypeEncoding.Polycurve.value)
encoded.append(int(self.closed))
encoded.extend(self.domain.to_list())
curve_array = CurveArray.from_curves(self.segments)
encoded.extend(curve_array.data)
encoded.append(get_encoding_from_units(self.units))
return encoded
curve_array = CurveArray.from_curves(self.segments).data
return [
CurveTypeEncoding.Polycurve.value,
int(self.closed),
*self.domain.to_list(),
len(curve_array),
*curve_array,
get_encoding_from_units(self.units),
]
class Extrusion(Base, speckle_type=GEOMETRY + "Extrusion"):
@@ -460,46 +472,65 @@ class Surface(Base, speckle_type=GEOMETRY + "Surface"):
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(self.degreeU)
encoded.append(self.degreeV)
encoded.append(self.countU)
encoded.append(self.countV)
encoded.append(int(self.rational))
encoded.append(int(self.closedU))
encoded.append(int(self.closedV))
encoded.extend(self.domainU.to_list())
encoded.extend(self.domainV.to_list())
encoded.append(len(self.pointData))
encoded.append(len(self.knotsU))
encoded.append(len(self.knotsV))
encoded.extend(self.pointData)
encoded.extend(self.knotsU)
encoded.extend(self.knotsV)
encoded.append(get_encoding_from_units(self.units))
return encoded
return [
self.degreeU,
self.degreeV,
self.countU,
self.countV,
int(self.rational),
int(self.closedU),
int(self.closedV),
*self.domainU.to_list(),
*self.domainV.to_list(),
len(self.pointData),
len(self.knotsU),
len(self.knotsV),
*self.pointData,
*self.knotsU,
*self.knotsV,
get_encoding_from_units(self.units),
]
class BrepFace(Base, speckle_type=GEOMETRY + "BrepFace"):
_Brep: "Brep" = None
SurfaceIndex: int = None
LoopIndices: List[int] = None
OuterLoopIndex: int = None
OrientationReversed: bool = None
LoopIndices: List[int] = None
@property
def _outer_loop(self):
return self._Brep.Loops[self.OuterLoopIndex]
return self._Brep.Loops[self.OuterLoopIndex] # pylint: disable=no-member
@property
def _surface(self):
return self._Brep.Surfaces[self.SurfaceIndex]
return self._Brep.Surfaces[self.SurfaceIndex] # pylint: disable=no-member
@property
def _loops(self):
if self.LoopIndices:
# pylint: disable=not-an-iterable, no-member
return [self._Brep.Loops[i] for i in self.LoopIndices]
@classmethod
def from_list(cls, args: List[Any], brep: "Brep" = None) -> "BrepFace":
return cls(
_Brep=brep,
SurfaceIndex=args[0],
OuterLoopIndex=args[1],
OrientationReversed=bool(args[2]),
LoopIndices=args[3:],
)
def to_list(self) -> List[Any]:
return [
self.SurfaceIndex,
self.OuterLoopIndex,
int(self.OrientationReversed),
*self.LoopIndices,
]
class BrepEdge(Base, speckle_type=GEOMETRY + "BrepEdge"):
_Brep: "Brep" = None
@@ -521,18 +552,59 @@ class BrepEdge(Base, speckle_type=GEOMETRY + "BrepEdge"):
@property
def _trims(self):
if self.TrimIndices:
# pylint: disable=not-an-iterable
return [self._Brep.Trims[i] for i in self.TrimIndices]
@property
def _curve(self):
return self._Brep.Curve3D[self.Curve3dIndex]
@classmethod
def from_list(cls, args: List[Any], brep: "Brep" = None) -> "BrepEdge":
domain_start = args[4]
domain_end = args[5]
domain = (
Interval(start=domain_start, end=domain_end)
if None not in (domain_start, domain_end)
else None
)
return cls(
_Brep=brep,
Curve3dIndex=int(args[0]),
TrimIndices=[int(t) for t in args[6:]],
StartIndex=int(args[1]),
EndIndex=int(args[2]),
ProxyCurveIsReversed=bool(args[3]),
Domain=domain,
)
def to_list(self) -> List[Any]:
return [
self.Curve3dIndex,
self.StartIndex,
self.EndIndex,
int(self.ProxyCurveIsReversed),
self.Domain.start,
self.Domain.end,
*self.TrimIndices,
]
class BrepLoopType(int, Enum):
Unknown = 0
Outer = 1
Inner = 2
Slit = 3
CurveOnSurface = 4
PointOnSurface = 5
class BrepLoop(Base, speckle_type=GEOMETRY + "BrepLoop"):
_Brep: "Brep" = None
FaceIndex: int = None
TrimIndices: List[int] = None
Type: str = None
Type: BrepLoopType = None
@property
def _face(self):
@@ -541,10 +613,27 @@ class BrepLoop(Base, speckle_type=GEOMETRY + "BrepLoop"):
@property
def _trims(self):
if self.TrimIndices:
# pylint: disable=not-an-iterable
return [self._Brep.Trims[i] for i in self.TrimIndices]
@classmethod
def from_list(cls, args: List[any], brep: "Brep" = None):
return cls(
_Brep=brep,
FaceIndex=args[0],
Type=BrepLoopType(args[1]),
TrimIndices=args[2:],
)
class BrepTrimTypeEnum(int, Enum):
def to_list(self) -> List[int]:
return [
self.FaceIndex,
self.Type.value,
*self.TrimIndices,
]
class BrepTrimType(int, Enum):
Unknown = 0
Boundary = 1
Mated = 2
@@ -564,29 +653,35 @@ class BrepTrim(Base, speckle_type=GEOMETRY + "BrepTrim"):
LoopIndex: int = None
CurveIndex: int = None
IsoStatus: int = None
TrimType: str = None
TrimType: BrepTrimType = None
IsReversed: bool = None
Domain: Interval = None
@property
def _face(self):
return self._Brep.Faces[self.FaceIndex]
if self._Brep:
return self._Brep.Faces[self.FaceIndex] # pylint: disable=no-member
@property
def _loop(self):
return self._Brep.Loops[self.LoopIndex]
if self._Brep:
return self._Brep.Loops[self.LoopIndex] # pylint: disable=no-member
@property
def _edge(self):
return self._Brep.Edges[self.EdgeIndex] if self.EdgeIndex != -1 else None
if self._Brep:
# pylint: disable=no-member
return self._Brep.Edges[self.EdgeIndex] if self.EdgeIndex != -1 else None
@property
def _curve_2d(self):
return self._Brep.Curve2D[self.CurveIndex]
if self._Brep:
return self._Brep.Curve2D[self.CurveIndex] # pylint: disable=no-member
@classmethod
def from_list(cls, args: List[Any]) -> "BrepTrim":
def from_list(cls, args: List[Any], brep: "Brep" = None) -> "BrepTrim":
return cls(
_Brep=brep,
EdgeIndex=args[0],
StartIndex=args[1],
EndIndex=args[2],
@@ -594,49 +689,62 @@ class BrepTrim(Base, speckle_type=GEOMETRY + "BrepTrim"):
LoopIndex=args[4],
CurveIndex=args[5],
IsoStatus=args[6],
TrimType=BrepTrimTypeEnum(args[7]).name,
TrimType=BrepTrimType(args[7]),
IsReversed=bool(args[8]),
)
def to_list(self) -> List[Any]:
encoded = []
encoded.append(self.EdgeIndex)
encoded.append(self.StartIndex)
encoded.append(self.EndIndex)
encoded.append(self.FaceIndex)
encoded.append(self.LoopIndex)
encoded.append(self.CurveIndex)
encoded.append(self.IsoStatus)
encoded.append(getattr(BrepTrimTypeEnum, self.TrimType).value)
encoded.append(self.IsReversed)
return encoded
return [
self.EdgeIndex,
self.StartIndex,
self.EndIndex,
self.FaceIndex,
self.LoopIndex,
self.CurveIndex,
self.IsoStatus,
self.TrimType.value,
int(self.IsReversed),
]
class Brep(
Base,
speckle_type=GEOMETRY + "Brep",
chunkable={
"SurfacesValue": 200,
"Curve3DValues": 200,
"Curve2DValues": 200,
"VerticesValue": 5000,
"Edges": 5000,
"Loops": 5000,
"TrimsValue": 5000,
"Faces": 5000,
"SurfacesValue": 31250,
"Curve3DValues": 31250,
"Curve2DValues": 31250,
"VerticesValue": 31250,
"EdgesValue": 62500,
"LoopsValue": 62500,
"FacesValue": 62500,
"TrimsValue": 62500,
},
detachable={"displayValue"},
serialize_ignore={"Surfaces", "Curve3D", "Curve2D", "Vertices", "Trims"},
serialize_ignore={
"Surfaces",
"Curve3D",
"Curve2D",
"Vertices",
"Trims",
"Edges",
"Loops",
"Faces",
},
):
provenance: str = None
bbox: Box = None
area: float = None
volume: float = None
displayValue: Mesh = None
_displayValue: List[Mesh] = None
Surfaces: List[Surface] = None
Curve3D: List[Base] = None
Curve2D: List[Base] = None
Vertices: List[Point] = None
Edges: List[BrepEdge] = None
Loops: List[BrepLoop] = None
Faces: List[BrepFace] = None
Trims: List[BrepTrim] = None
IsClosed: bool = None
Orientation: int = None
@@ -645,65 +753,89 @@ class Brep(
return children
for child in children:
child._Brep = self
child._Brep = self # pylint: disable=protected-access
return children
# set as prop for now for backwards compatibility
@property
def Edges(self) -> List[BrepEdge]:
return self._inject_self_into_children(self._Edges)
def displayValue(self) -> List[Mesh]:
return self._displayValue
@Edges.setter
def Edges(self, value: List[BrepEdge]):
self._Edges = value
@displayValue.setter
def displayValue(self, value):
if isinstance(value, Mesh):
self._displayValue = [value]
elif isinstance(value, list):
self._displayValue = value
@property
def Loops(self) -> List[BrepLoop]:
return self._inject_self_into_children(self._Loops)
def EdgesValue(self) -> List[BrepEdge]:
return None if self.Edges is None else ObjectArray.from_objects(self.Edges).data
@Loops.setter
def Loops(self, value: List[BrepLoop]):
self._Loops = value
@EdgesValue.setter
def EdgesValue(self, value: List[float]):
if not value:
return
self.Edges = ObjectArray.decode_data(value, BrepEdge.from_list, brep=self)
@property
def Faces(self) -> List[BrepFace]:
return self._inject_self_into_children(self._Faces)
def LoopsValue(self) -> List[BrepLoop]:
return None if self.Loops is None else ObjectArray.from_objects(self.Loops).data
@Faces.setter
def Faces(self, value: List[BrepFace]):
self._Faces = value
@LoopsValue.setter
def LoopsValue(self, value: List[int]):
if not value:
return
self.Loops = ObjectArray.decode_data(value, BrepLoop.from_list, brep=self)
@property
def FacesValue(self) -> List[int]:
return None if self.Faces is None else ObjectArray.from_objects(self.Faces).data
@FacesValue.setter
def FacesValue(self, value: List[int]):
if not value:
return
self.Faces = ObjectArray.decode_data(value, BrepFace.from_list, brep=self)
@property
def SurfacesValue(self) -> List[float]:
if self.Surfaces is None:
return None
return ObjectArray.from_objects(self.Surfaces).data
return (
None
if self.Surfaces is None
else ObjectArray.from_objects(self.Surfaces).data
)
@SurfacesValue.setter
def SurfacesValue(self, value: List[float]):
if not value:
return
self.Surfaces = ObjectArray.decode_data(value, Surface.from_list)
@property
def Curve3DValues(self) -> List[float]:
if self.Curve3D is None:
return None
return CurveArray.from_curves(self.Curve3D).data
return (
None if self.Curve3D is None else CurveArray.from_curves(self.Curve3D).data
)
@Curve3DValues.setter
def Curve3DValues(self, value: List[float]):
crv_array = CurveArray()
crv_array.data = value
crv_array = CurveArray(value)
self.Curve3D = crv_array.to_curves()
@property
def Curve2DValues(self) -> List[Base]:
if self.Curve2D is None:
return None
return CurveArray.from_curves(self.Curve2D).data
return (
None if self.Curve2D is None else CurveArray.from_curves(self.Curve2D).data
)
@Curve2DValues.setter
def Curve2DValues(self, value: List[float]):
crv_array = CurveArray()
crv_array.data = value
crv_array = CurveArray(value)
self.Curve2D = crv_array.to_curves()
@property
@@ -730,27 +862,25 @@ class Brep(
self.Vertices = vertices
@property
def Trims(self) -> List[BrepTrim]:
return self._inject_self_into_children(self._Trims)
@Trims.setter
def Trims(self, value: List[BrepTrim]):
self._Trims = value
# TODO: can this be consistent with loops, edges, faces, curves, etc and prepend with the chunk list? needs to happen in sharp first
@property
def TrimsValue(self) -> List[float]:
if self.Trims is None:
return None
values = []
# return None if self.Trims is None else ObjectArray.from_objects(self.Trims).data
if not self.Trims:
return
value = []
for trim in self.Trims:
values.extend(trim.to_list())
return values
value.extend(trim.to_list())
return value
@TrimsValue.setter
def TrimsValue(self, value: List[float]):
if not value:
return
# self.Trims = ObjectArray.decode_data(value, BrepTrim.from_list, brep=self)
self.Trims = [
BrepTrim.from_list(value[i : i + 9]) for i in range(0, len(value), 9)
BrepTrim.from_list(value[i : i + 9], self) for i in range(0, len(value), 9)
]
+19 -4
View File
@@ -1,4 +1,4 @@
from typing import List
from typing import Any, List
from specklepy.objects.geometry import Point, Vector
from .base import Base
@@ -55,10 +55,11 @@ class Transform(
def value(self, value: List[float]) -> None:
try:
value = [float(x) for x in value]
except (ValueError, TypeError):
except (ValueError, TypeError) as error:
raise ValueError(
f"Could not create a Transform object with the requested value. Input must be a 16 element list of numbers. Value provided: {value}"
)
) from error
if len(value) != 16:
raise ValueError(
f"Could not create a Transform object: input list should be 16 floats long, but was {len(value)} long"
@@ -196,4 +197,18 @@ class BlockInstance(
Base, speckle_type=OTHER + "BlockInstance", detachable={"blockDefinition"}
):
blockDefinition: BlockDefinition = None
transform: Transform = None
transform: Transform = None
# TODO: prob move this into a built elements module, but just trialling this for now
class RevitParameter(Base, speckle_type="Objects.BuiltElements.Revit.Parameter"):
name: str = None
value: Any = None
applicationUnitType: str = None # eg UnitType UT_Length
applicationUnit: str = None # DisplayUnitType eg DUT_MILLIMITERS
applicationInternalName: str = (
None # BuiltInParameterName or GUID for shared parameter
)
isShared: bool = False
isReadOnly: bool = False
isTypeParameter: bool = False
+3 -4
View File
@@ -17,6 +17,7 @@ UNITS_STRINGS = {
UNITS_ENCODINGS = {
"none": 0,
None: 0,
"mm": 1,
"cm": 2,
"m": 3,
@@ -58,7 +59,5 @@ def get_units_from_encoding(unit: int):
def get_encoding_from_units(unit: str):
try:
return UNITS_ENCODINGS[unit]
except KeyError:
raise SpeckleException(
message=f"No encoding exists for unit {unit}. Please enter a valid unit to encode (eg {UNITS_ENCODINGS})."
)
except KeyError as e:
raise SpeckleException(message=f"No encoding exists for unit {unit}. Please enter a valid unit to encode (eg {UNITS_ENCODINGS}).") from e
+101 -55
View File
@@ -1,16 +1,19 @@
import re
import ujson
import hashlib
import re
import warnings
from uuid import uuid4
from enum import Enum
from warnings import warn
from typing import Any, Dict, List, Tuple
from specklepy.objects.base import Base, DataChunk
from specklepy.logging.exceptions import (
SerializationException,
SpeckleException,
SpeckleWarning,
)
from specklepy.transports.abstract_transport import AbstractTransport
# import for serialization
import specklepy.objects.geometry
import specklepy.objects.other
@@ -37,22 +40,33 @@ def safe_json_loads(obj: str, obj_id=None) -> Any:
class BaseObjectSerializer:
read_transport: AbstractTransport
write_transports: List[AbstractTransport]
detach_lineage: List[bool] = [] # tracks depth and whether or not to detach
lineage: List[str] = [] # keeps track of hash chain through the object tree
family_tree: Dict[str, Dict[str, int]] = {}
closure_table: Dict[str, Dict[str, int]] = {}
detach_lineage: List[bool] # tracks depth and whether or not to detach
lineage: List[str] # keeps track of hash chain through the object tree
family_tree: Dict[str, Dict[str, int]]
closure_table: Dict[str, Dict[str, int]]
deserialized: Dict[str, Base] # holds deserialized objects so objects with same id return the same instance
def __init__(
self, write_transports: List[AbstractTransport] = [], read_transport=None
) -> None:
self.write_transports = write_transports
def __init__(self, write_transports: List[AbstractTransport] = None, read_transport=None) -> None:
self.write_transports = write_transports or []
self.read_transport = read_transport
self.detach_lineage = []
self.lineage = []
self.family_tree = {}
self.closure_table = {}
self.deserialized = {}
def write_json(self, base: Base):
self.__reset_writer()
self.detach_lineage = [True]
hash, obj = self.traverse_base(base)
return hash, ujson.dumps(obj)
"""Serializes a given base object into a json string
Arguments:
base {Base} -- the base object to be decomposed and serialized
Returns:
(str, str) -- a tuple containing the object id of the base object and the serialized object string
"""
obj_id, obj = self.traverse_base(base)
return obj_id, ujson.dumps(obj)
def traverse_base(self, base: Base) -> Tuple[str, Dict]:
"""Decomposes the given base object and builds a serializable dictionary
@@ -61,8 +75,23 @@ class BaseObjectSerializer:
base {Base} -- the base object to be decomposed and serialized
Returns:
(str, dict) -- a tuple containing the hash (id) of the base object and the constructed serializable dictionary
(str, dict) -- a tuple containing the object id of the base object and the constructed serializable dictionary
"""
self.__reset_writer()
if self.write_transports:
for wt in self.write_transports:
wt.begin_write()
obj_id, obj = self._traverse_base(base)
if self.write_transports:
for wt in self.write_transports:
wt.end_write()
return obj_id, obj
def _traverse_base(self, base: Base) -> Tuple[str, Dict]:
if not self.detach_lineage:
self.detach_lineage = [True]
@@ -85,11 +114,6 @@ class BaseObjectSerializer:
if prop == "id":
continue
# allow serialisation of nulls
if value is None:
object_builder[prop] = value
continue
# only bother with chunking and detaching if there is a write transport
if self.write_transports:
dynamic_chunk_match = prop.startswith("@") and re.match(
@@ -106,17 +130,22 @@ class BaseObjectSerializer:
prop.startswith("@") or prop in base._detachable or chunkable
)
# 1. handle primitives (ints, floats, strings, and bools)
if isinstance(value, PRIMITIVES):
# 1. handle None and primitives (ints, floats, strings, and bools)
if value is None or isinstance(value, PRIMITIVES):
object_builder[prop] = value
continue
# NOTE: for dynamic props, this won't be re-serialised as an enum but as an int
if isinstance(value, Enum):
object_builder[prop] = value.value
continue
# 2. handle Base objects
elif isinstance(value, Base):
child_obj = self.traverse_value(value, detach=detach)
if detach and self.write_transports:
ref_hash = child_obj["id"]
object_builder[prop] = self.detach_helper(ref_hash=ref_hash)
ref_id = child_obj["id"]
object_builder[prop] = self.detach_helper(ref_id=ref_id)
else:
object_builder[prop] = child_obj
@@ -135,8 +164,8 @@ class BaseObjectSerializer:
chunk_refs = []
for c in chunks:
self.detach_lineage.append(detach)
ref_hash, _ = self.traverse_base(c)
ref_obj = self.detach_helper(ref_hash=ref_hash)
ref_id, _ = self._traverse_base(c)
ref_obj = self.detach_helper(ref_id=ref_id)
chunk_refs.append(ref_obj)
object_builder[prop] = chunk_refs
@@ -155,20 +184,20 @@ class BaseObjectSerializer:
}
object_builder["totalChildrenCount"] = len(closure)
hash = hash_obj(object_builder)
obj_id = hash_obj(object_builder)
object_builder["id"] = hash
object_builder["id"] = obj_id
if closure:
object_builder["__closure"] = self.closure_table[hash] = closure
object_builder["__closure"] = self.closure_table[obj_id] = closure
# write detached or root objects to transports
if detached and self.write_transports:
for t in self.write_transports:
t.save_object(id=hash, serialized_object=ujson.dumps(object_builder))
t.save_object(id=obj_id, serialized_object=ujson.dumps(object_builder))
del self.lineage[-1]
return hash, object_builder
return obj_id, object_builder
def traverse_value(self, obj: Any, detach: bool = False) -> Any:
"""Decomposes a given object and constructs a serializable object or dictionary
@@ -182,6 +211,10 @@ class BaseObjectSerializer:
if isinstance(obj, PRIMITIVES):
return obj
# NOTE: for dynamic props, this won't be re-serialised as an enum but as an int
if isinstance(obj, Enum):
return obj.value
elif isinstance(obj, (list, tuple, set)):
if not detach:
return [self.traverse_value(o) for o in obj]
@@ -190,15 +223,15 @@ class BaseObjectSerializer:
for o in obj:
if isinstance(o, Base):
self.detach_lineage.append(detach)
hash, _ = self.traverse_base(o)
detached_list.append(self.detach_helper(ref_hash=hash))
ref_id, _ = self._traverse_base(o)
detached_list.append(self.detach_helper(ref_id=ref_id))
else:
detached_list.append(self.traverse_value(o, detach))
return detached_list
elif isinstance(obj, dict):
for k, v in obj.items():
if isinstance(v, PRIMITIVES):
if isinstance(v, PRIMITIVES) or v is None:
continue
else:
obj[k] = self.traverse_value(v)
@@ -206,7 +239,7 @@ class BaseObjectSerializer:
elif isinstance(obj, Base):
self.detach_lineage.append(detach)
_, base_obj = self.traverse_base(obj)
_, base_obj = self._traverse_base(obj)
return base_obj
else:
@@ -215,16 +248,16 @@ class BaseObjectSerializer:
except:
warn(
f"Failed to handle {type(obj)} in `BaseObjectSerializer.traverse_value`",
SerializationException,
SpeckleWarning,
)
return str(obj)
def detach_helper(self, ref_hash: str) -> Dict[str, str]:
def detach_helper(self, ref_id: str) -> Dict[str, str]:
"""Helper to keep track of detached objects and their depth in the family tree and create reference objects to place in the parent object
Arguments:
ref_hash {str} -- the hash of the fully traversed object
ref_id {str} -- the id of the fully traversed object
Returns:
dict -- a reference object to be inserted into the given object's parent
@@ -233,19 +266,19 @@ class BaseObjectSerializer:
for parent in self.lineage:
if parent not in self.family_tree:
self.family_tree[parent] = {}
if ref_hash not in self.family_tree[parent] or self.family_tree[parent][
ref_hash
if ref_id not in self.family_tree[parent] or self.family_tree[parent][
ref_id
] > len(self.detach_lineage):
self.family_tree[parent][ref_hash] = len(self.detach_lineage)
self.family_tree[parent][ref_id] = len(self.detach_lineage)
return {
"referencedId": ref_hash,
"referencedId": ref_id,
"speckle_type": "reference",
}
def __reset_writer(self) -> None:
"""Reinitializes the lineage, and other variables that get used during the json writing process"""
self.detach_lineage = []
self.detach_lineage = [True]
self.lineage = []
self.family_tree = {}
self.closure_table = {}
@@ -261,6 +294,8 @@ class BaseObjectSerializer:
"""
if not obj_string:
return None
self.deserialized = {}
obj = safe_json_loads(obj_string)
return self.recompose_base(obj=obj)
@@ -279,6 +314,9 @@ class BaseObjectSerializer:
if isinstance(obj, str):
obj = safe_json_loads(obj)
if "id" in obj and obj["id"] in self.deserialized:
return self.deserialized[obj["id"]]
if "speckle_type" in obj and obj["speckle_type"] == "reference":
obj = self.get_child(obj=obj)
@@ -309,19 +347,25 @@ class BaseObjectSerializer:
# 2. handle referenced child objects
elif "referencedId" in value:
ref_hash = value["referencedId"]
ref_obj_str = self.read_transport.get_object(id=ref_hash)
if not ref_obj_str:
raise SpeckleException(
f"Could not find the referenced child object of id `{ref_hash}` in the given read transport: {self.read_transport.name}"
ref_id = value["referencedId"]
ref_obj_str = self.read_transport.get_object(id=ref_id)
if ref_obj_str:
ref_obj = safe_json_loads(ref_obj_str, ref_id)
base.__setattr__(prop, self.recompose_base(obj=ref_obj))
else:
warnings.warn(
f"Could not find the referenced child object of id `{ref_id}` in the given read transport: {self.read_transport.name}",
SpeckleWarning,
)
ref_obj = safe_json_loads(ref_obj_str, ref_hash)
base.__setattr__(prop, self.recompose_base(obj=ref_obj))
base.__setattr__(prop, self.handle_value(value))
# 3. handle all other cases (base objects, lists, and dicts)
else:
base.__setattr__(prop, self.handle_value(value))
if "id" in obj:
self.deserialized[obj["id"]] = base
return base
def handle_value(self, obj: Any):
@@ -367,11 +411,13 @@ class BaseObjectSerializer:
return obj
def get_child(self, obj: Dict):
ref_hash = obj["referencedId"]
ref_obj_str = self.read_transport.get_object(id=ref_hash)
ref_id = obj["referencedId"]
ref_obj_str = self.read_transport.get_object(id=ref_id)
if not ref_obj_str:
raise SpeckleException(
f"Could not find the referenced child object of id `{ref_hash}` in the given read transport: {self.read_transport.name}"
warnings.warn(
f"Could not find the referenced child object of id `{ref_id}` in the given read transport: {self.read_transport.name}",
SpeckleWarning,
)
return obj
return safe_json_loads(ref_obj_str, ref_hash)
return safe_json_loads(ref_obj_str, ref_id)
+2 -18
View File
@@ -156,29 +156,13 @@ class ServerTransport(AbstractTransport):
lines = r.iter_lines(decode_unicode=True)
# iter through returned objects saving them as we go
target_transport.begin_write()
for line in lines:
if line:
hash, obj = line.split("\t")
target_transport.save_object(hash, obj)
target_transport.save_object(id, root_obj_serialized)
target_transport.end_write()
return root_obj_serialized
# async def stream_res(self, endpoint: str) -> str:
# data = b""
# async with aiohttp.ClientSession() as session:
# session.headers.update(
# {
# "Authorization": f"{self.session.headers['Authorization']}",
# "Accept": "text/plain",
# }
# )
# async with session.get(endpoint) as res:
# while True:
# chunk = await res.content.read(self.chunk_size)
# if not chunk:
# break
# data += chunk
# return data.decode("utf-8")
+36 -62
View File
@@ -3,7 +3,7 @@ import sys
import time
import sched
import sqlite3
from typing import Any, List, Dict
from typing import Any, List, Dict, Tuple
from appdirs import user_data_dir
from contextlib import closing
from specklepy.transports.abstract_transport import AbstractTransport
@@ -14,25 +14,29 @@ class SQLiteTransport(AbstractTransport):
_name = "SQLite"
_base_path: str = None
_root_path: str = None
_is_writing: bool = False
_scheduler = sched.scheduler(time.time, time.sleep)
_polling_interval = 0.5 # seconds
__connection: sqlite3.Connection = None
app_name: str = ""
scope: str = ""
saved_obj_count: int = 0
max_size: int = None
_current_batch: List[Tuple[str, str]] = None
_current_batch_size: int = None
def __init__(
self,
base_path: str = None,
app_name: str = None,
scope: str = None,
max_batch_size_mb: float = 10.0,
**data: Any,
) -> None:
super().__init__(**data)
self.app_name = app_name or "Speckle"
self.scope = scope or "Objects"
self._base_path = base_path or self.get_base_path(self.app_name)
self.max_size = int(max_batch_size_mb * 1000 * 1000)
self._current_batch = []
self._current_batch_size = 0
try:
os.makedirs(self._base_path, exist_ok=True)
@@ -50,12 +54,6 @@ class SQLiteTransport(AbstractTransport):
def __repr__(self) -> str:
return f"SQLiteTransport(app: '{self.app_name}', scope: '{self.scope}')"
# def __write_timer_elapsed(self):
# print("WRITE TIMER ELAPSED")
# proc = Process(target=_run_queue, args=(self.__queue, self._root_path))
# proc.start()
# proc.join()
@staticmethod
def get_base_path(app_name):
# from appdirs https://github.com/ActiveState/appdirs/blob/master/appdirs.py
@@ -74,36 +72,6 @@ class SQLiteTransport(AbstractTransport):
path = os.path.expanduser("~/.config/")
return os.path.join(path, app_name)
# def __consume_queue(self):
# if self._is_writing or self.__queue.empty():
# return
# print("CONSUME QUEUE")
# self._is_writing = True
# while not self.__queue.empty():
# data = self.__queue.get()
# self.save_object(data[0], data[1])
# self._is_writing = False
# self._scheduler.enter(
# delay=self._polling_interval, priority=1, action=self.__consume_queue
# )
# self._scheduler.run(blocking=True)
# def save_object(self, id: str, serialized_object: str) -> None:
# """Adds an object to the queue and schedules it to be saved.
# Arguments:
# id {str} -- the object id
# serialized_object {str} -- the full string representation of the object
# """
# print("SAVE OBJECT")
# self.__queue.put((id, serialized_object))
# self._scheduler.enter(
# delay=self._polling_interval, priority=1, action=self.__consume_queue
# )
# self._scheduler.run(blocking=True)
def save_object_from_transport(
self, id: str, source_transport: AbstractTransport
) -> None:
@@ -117,23 +85,41 @@ class SQLiteTransport(AbstractTransport):
self.save_object(id, serialized_object)
def save_object(self, id: str, serialized_object: str) -> None:
"""Directly saves an object into the database.
"""
Adds an object to the current batch to be written to the db. If the current batch is full,
the batch is written to the db and the current batch is reset.
Arguments:
id {str} -- the object id
serialized_object {str} -- the full string representation of the object
"""
obj_size = len(serialized_object)
if (
not self._current_batch
or self._current_batch_size + obj_size < self.max_size
):
self._current_batch.append((id, serialized_object))
self._current_batch_size += obj_size
return
self.save_current_batch()
self._current_batch = [(id, serialized_object)]
self._current_batch_size = obj_size
def save_current_batch(self) -> None:
"""Save the current batch of objects to the local db"""
self.__check_connection()
try:
with closing(self.__connection.cursor()) as c:
c.execute(
c.executemany(
"INSERT OR IGNORE INTO objects(hash, content) VALUES(?,?)",
(id, serialized_object),
self._current_batch,
)
self.__connection.commit()
except Exception as ex:
raise SpeckleException(
f"Could not save the object to the local db. Inner exception: {ex}", ex
f"Could not save the batch of objects to the local db. Inner exception: {ex}",
ex,
)
def get_object(self, id: str) -> str or None:
@@ -156,10 +142,14 @@ class SQLiteTransport(AbstractTransport):
return ret
def begin_write(self):
self._object_cache = []
self.saved_obj_count = 0
def end_write(self):
pass
if self._current_batch:
self.save_current_batch()
self._current_batch = []
self._current_batch_size = 0
def copy_object_and_children(
self, id: str, target_transport: AbstractTransport
@@ -198,20 +188,4 @@ class SQLiteTransport(AbstractTransport):
self.__connection = sqlite3.connect(self._root_path)
def __del__(self):
self.__connection.close()
# def _run_queue(queue: Queue, root_path: str):
# if queue.empty():
# return
# print("RUN QUEUE")
# conn = sqlite3.connect(root_path)
# while not queue.empty():
# data = queue.get()
# with closing(conn.cursor()) as c:
# c.execute(
# "INSERT OR IGNORE INTO objects(hash, content) VALUES(?,?)",
# (data[0], data[1]),
# )
# conn.commit()
# conn.close()
self.close()
+15 -3
View File
@@ -6,7 +6,7 @@ from specklepy.api.models import Stream
from specklepy.api.client import SpeckleClient
from specklepy.objects.base import Base
from specklepy.objects.geometry import Point
from specklepy.objects.fakemesh import FakeMesh
from specklepy.objects.fakemesh import FakeDirection, FakeMesh
from specklepy.logging import metrics
metrics.disable()
@@ -61,7 +61,14 @@ def second_user_dict(host):
@pytest.fixture(scope="session")
def client(host, user_dict):
client = SpeckleClient(host=host, use_ssl=False)
client.authenticate(user_dict["token"])
client.authenticate_with_token(user_dict["token"])
return client
@pytest.fixture(scope="session")
def second_client(host, second_user_dict):
client = SpeckleClient(host=host, use_ssl=False)
client.authenticate_with_token(second_user_dict["token"])
return client
@@ -81,9 +88,10 @@ def mesh():
mesh = FakeMesh()
mesh.name = "my_mesh"
mesh.vertices = [random.uniform(0, 10) for _ in range(1, 210)]
mesh.faces = [i for i in range(1, 210)]
mesh.faces = list(range(1, 210))
mesh["@(100)colours"] = [random.uniform(0, 10) for _ in range(1, 210)]
mesh["@()default_chunk"] = [random.uniform(0, 10) for _ in range(1, 210)]
mesh.cardinal_dir = FakeDirection.WEST
mesh.test_bases = [Base(name=f"test {i}") for i in range(1, 22)]
mesh.detach_this = Base(name="predefined detached base")
mesh["@detach"] = Base(name="detached base")
@@ -102,6 +110,10 @@ def base():
base = Base()
base.name = "my_base"
base.units = "millimetres"
base.null_val = None
base.null_dict = {"a null val": None}
base.tuple = (1, 2, "3")
base.set = {1, 2, "3"}
base.vertices = [random.uniform(0, 10) for _ in range(1, 120)]
base.test_bases = [Base(name=i) for i in range(1, 22)]
base["@detach"] = Base(name="detached base")
+33 -1
View File
@@ -1,5 +1,6 @@
from enum import Enum
from typing import Dict, List, Optional, Union
from contextlib import ExitStack as does_not_raise
from typing import Dict, List, Optional
import pytest
from specklepy.api import operations
@@ -95,6 +96,12 @@ def test_base_of_custom_speckle_type() -> None:
assert b1.name == "Tweety's Crib"
class DietaryRestrictions(Enum):
VEGAN = 1
GLUTEN_FREE = 2
NUT_FREE = 3
class FrozenYoghurt(Base):
"""Testing type checking"""
@@ -103,6 +110,8 @@ class FrozenYoghurt(Base):
customer: str
add_ons: Optional[Dict[str, float]] # dict item types won't be checked
price: float = 0.0
dietary: DietaryRestrictions
tag: Union[int, str]
def test_type_checking() -> None:
@@ -111,6 +120,9 @@ def test_type_checking() -> None:
order.servings = 2
order.price = "7" # will get converted
order.customer = "izzy"
order.dietary = DietaryRestrictions.VEGAN
order.tag = "preorder"
order.tag = 4411
with pytest.raises(SpeckleException):
order.flavours = "not a list"
@@ -118,8 +130,28 @@ def test_type_checking() -> None:
order.servings = "five"
with pytest.raises(SpeckleException):
order.add_ons = ["sprinkles"]
with pytest.raises(SpeckleException):
order.dietary = "no nuts plz"
with pytest.raises(SpeckleException):
order.tag = ["tag01", "tag02"]
order.add_ons = {"sprinkles": 0.2, "chocolate": 1.0}
order.flavours = ["strawberry", "lychee", "peach", "pineapple"]
assert order.price == 7.0
def test_cached_deserialization() -> None:
material = Base(color="blue", opacity=0.5)
a = Base(name="a")
a["@material"] = material
b = Base(name="b")
b["@material"] = material
root = Base(a=a, b=b)
serialized = operations.serialize(root)
deserialized = operations.deserialize(serialized)
assert deserialized["a"]["@material"] is deserialized["b"]["@material"]
+111 -54
View File
@@ -1,5 +1,5 @@
# pylint: disable=redefined-outer-name
import json
from typing import Callable
import pytest
from specklepy.api import operations
@@ -13,8 +13,9 @@ from specklepy.objects.geometry import (
BrepEdge,
BrepFace,
BrepLoop,
BrepLoopType,
BrepTrim,
BrepTrimTypeEnum,
BrepTrimType,
Circle,
Curve,
Ellipse,
@@ -48,12 +49,7 @@ def vector():
@pytest.fixture()
def plane(point, vector):
return Plane(
origin=point,
normal=vector,
xdir=vector,
ydir=vector,
)
return Plane(origin=point, normal=vector, xdir=vector, ydir=vector, units="m")
@pytest.fixture()
@@ -74,6 +70,7 @@ def line(point, interval):
start=point,
end=point,
domain=interval,
units="none"
# These attributes are not handled in C#
# bbox=None,
# length=None
@@ -81,7 +78,7 @@ def line(point, interval):
@pytest.fixture()
def arc(plane, interval):
def arc(plane, interval, point):
return Arc(
radius=2.3,
startAngle=22.1,
@@ -90,13 +87,13 @@ def arc(plane, interval):
plane=plane,
domain=interval,
units="m",
startPoint=point,
midPoint=point,
endPoint=point,
# These attributes are not handled in C#
# bbox=None,
# area=None,
# length=None,
# startPoint=None,
# midPoint=None,
# endPoint=None,
)
@@ -236,7 +233,7 @@ def brep_edge(interval):
@pytest.fixture()
def brep_loop():
return BrepLoop(FaceIndex=5, TrimIndices=[3, 4, 5], Type="unknown")
return BrepLoop(FaceIndex=5, TrimIndices=[3, 4, 5], Type=BrepLoopType.Unknown)
@pytest.fixture()
@@ -249,7 +246,7 @@ def brep_trim():
LoopIndex=4,
CurveIndex=7,
IsoStatus=6,
TrimType="Mated",
TrimType=BrepTrimType.Mated,
IsReversed=False,
# These attributes are not handled in C#
# Domain=None,
@@ -338,22 +335,22 @@ def geometry_objects_dict(
],
)
def test_to_and_from_list(object_name: str, geometry_objects_dict):
object = geometry_objects_dict[object_name]
assert hasattr(object, "to_list")
assert hasattr(object, "from_list")
obj = geometry_objects_dict[object_name]
assert hasattr(obj, "to_list")
assert hasattr(obj, "from_list")
chunks = object.to_list()
chunks = obj.to_list()
assert isinstance(chunks, list)
object_class = object.__class__
object_class = obj.__class__
decoded_object: Base = object_class.from_list(chunks)
assert decoded_object.get_id() == object.get_id()
assert decoded_object.get_id() == obj.get_id()
def test_brep_surfaces_value_serialization(surface):
brep = Brep()
assert brep.Surfaces == None
assert brep.SurfacesValue == None
assert brep.Surfaces is None
assert brep.SurfacesValue is None
brep.Surfaces = [surface, surface]
assert brep.SurfacesValue == ObjectArray.from_objects([surface, surface]).data
@@ -364,8 +361,8 @@ def test_brep_surfaces_value_serialization(surface):
def test_brep_curve2d_values_serialization(curve, polyline, circle):
brep = Brep()
assert brep.Curve2D == None
assert brep.Curve2DValues == None
assert brep.Curve2D is None
assert brep.Curve2DValues is None
brep.Curve2D = [curve, polyline]
assert brep.Curve2DValues == CurveArray.from_curves([curve, polyline]).data
@@ -376,8 +373,8 @@ def test_brep_curve2d_values_serialization(curve, polyline, circle):
def test_brep_curve3d_values_serialization(curve, polyline, circle):
brep = Brep()
assert brep.Curve3D == None
assert brep.Curve3DValues == None
assert brep.Curve3D is None
assert brep.Curve3DValues is None
brep.Curve3D = [curve, polyline]
assert brep.Curve3DValues == CurveArray.from_curves([curve, polyline]).data
@@ -389,9 +386,9 @@ def test_brep_curve3d_values_serialization(curve, polyline, circle):
def test_brep_vertices_values_serialization():
brep = Brep()
brep.VerticesValue = [1, 1, 1, 1, 2, 2, 2, 3, 3, 3]
brep.Vertices[0].get_id() == Point(x=1, y=1, z=1, _units="mm").get_id()
brep.Vertices[1].get_id() == Point(x=2, y=2, z=2, _units="mm").get_id()
brep.Vertices[2].get_id() == Point(x=3, y=3, z=3, _units="mm").get_id()
assert brep.Vertices[0].get_id() == Point(x=1, y=1, z=1, _units="mm").get_id()
assert brep.Vertices[1].get_id() == Point(x=2, y=2, z=2, _units="mm").get_id()
assert brep.Vertices[2].get_id() == Point(x=3, y=3, z=3, _units="mm").get_id()
def test_trims_value_serialization():
@@ -405,7 +402,7 @@ def test_trims_value_serialization():
0,
1,
1,
1,
0,
1,
0,
0,
@@ -414,32 +411,82 @@ def test_trims_value_serialization():
1,
2,
1,
0,
1,
]
brep.Trims[0].get_id() == BrepTrim(
EdgeIndex=0,
StartIndex=0,
EndIndex=0,
FaceIndex=0,
LoopIndex=0,
CurveIndex=0,
IsoStatus=1,
TrimType=BrepTrimTypeEnum.Boundary,
IsReversed=False,
).get_id()
assert (
brep.Trims[0].get_id()
== BrepTrim(
EdgeIndex=0,
StartIndex=0,
EndIndex=0,
FaceIndex=0,
LoopIndex=0,
CurveIndex=0,
IsoStatus=1,
TrimType=BrepTrimType.Boundary,
IsReversed=False,
).get_id()
)
brep.Trims[1].get_id() == BrepTrim(
EdgeIndex=1,
StartIndex=0,
EndIndex=0,
FaceIndex=0,
LoopIndex=0,
CurveIndex=1,
IsoStatus=2,
TrimType=BrepTrimTypeEnum.Boundary,
IsReversed=True,
).get_id()
assert (
brep.Trims[1].get_id()
== BrepTrim(
EdgeIndex=1,
StartIndex=0,
EndIndex=0,
FaceIndex=0,
LoopIndex=0,
CurveIndex=1,
IsoStatus=2,
TrimType=BrepTrimType.Boundary,
IsReversed=True,
).get_id()
)
def test_loops_value_serialization():
brep = Brep()
brep.LoopsValue = [6, 0, 1, 0, 1, 2, 3]
assert brep == brep.Loops[0]._Brep # pylint: disable=protected-access
assert (
brep.Loops[0].get_id()
== BrepLoop(
FaceIndex=0, Type=BrepLoopType(1), TrimIndices=[0, 1, 2, 3]
).get_id()
)
def test_edges_value_serialization():
brep = Brep()
brep.EdgesValue = [8, 0, 0, 1, 0, -8.13345756858629, 8.13345756858629, 1, 3]
assert brep == brep.Edges[0]._Brep # pylint: disable=protected-access
assert (
brep.Edges[0].get_id()
== BrepEdge(
Curve3dIndex=0,
StartIndex=0,
EndIndex=1,
ProxyCurveIsReversed=False,
Domain=Interval(start=-8.13345756858629, end=8.13345756858629),
TrimIndices=[1, 3],
).get_id()
)
def test_faces_value_serialization():
brep = Brep()
brep.FacesValue = [4, 0, 0, 1, 0]
assert brep == brep.Faces[0]._Brep # pylint: disable=protected-access
assert (
brep.Faces[0].get_id()
== BrepFace(
SurfaceIndex=0, OuterLoopIndex=0, OrientationReversed=True, LoopIndices=[0]
).get_id()
)
def test_serialized_brep_attributes(brep: Brep):
@@ -447,7 +494,16 @@ def test_serialized_brep_attributes(brep: Brep):
serialized = operations.serialize(brep, [transport])
serialized_dict = json.loads(serialized)
removed_keys = ["Surfaces", "Curve3D", "Curve2D", "Vertices", "Trims"]
removed_keys = [
"Surfaces",
"Curve3D",
"Curve2D",
"Vertices",
"Trims",
"Loops",
"Edges",
"Faces",
]
for k in removed_keys:
assert k not in serialized_dict.keys()
@@ -459,6 +515,7 @@ def test_mesh_create():
mesh = Mesh.create(vertices, faces)
with pytest.raises(SpeckleException):
# pylint: disable=unused-variable
bad_mesh = Mesh.create(vertices=7, faces=faces)
assert mesh.vertices == vertices
+10 -2
View File
@@ -1,5 +1,6 @@
import pytest
from specklepy.api.models import ServerInfo
from specklepy.api.client import SpeckleClient
class TestServer:
@@ -12,12 +13,19 @@ class TestServer:
"lifespan": 9001,
}
def test_server_get(self, client):
def test_server_get(self, client: SpeckleClient):
server = client.server.get()
assert isinstance(server, ServerInfo)
def test_server_apps(self, client):
def test_server_version(self, client: SpeckleClient):
version = client.server.version()
assert isinstance(version, tuple)
assert isinstance(version[0], int)
assert len(version) >= 3
def test_server_apps(self, client: SpeckleClient):
apps = client.server.apps()
assert isinstance(apps, list)
+143 -13
View File
@@ -1,6 +1,18 @@
import pytest
from specklepy.api.models import Stream
from specklepy.logging.exceptions import GraphQLException
from datetime import datetime
from specklepy.api.models import (
ActivityCollection,
Activity,
PendingStreamCollaborator,
Stream,
User,
)
from specklepy.api.client import SpeckleClient
from specklepy.logging.exceptions import (
GraphQLException,
SpeckleException,
UnsupportedException,
)
@pytest.mark.run(order=2)
@@ -23,6 +35,10 @@ class TestStream:
isPublic=False,
)
@pytest.fixture(scope="module")
def second_user(self, second_client: SpeckleClient):
return second_client.user.get()
def test_stream_create(self, client, stream, updated_stream):
stream.id = updated_stream.id = client.stream.create(
name=stream.name,
@@ -67,29 +83,143 @@ class TestStream:
assert len(search_results) == 1
assert search_results[0].name == updated_stream.name
def test_stream_grant_permission(self, client, stream, second_user_dict):
granted = client.stream.grant_permission(
def test_stream_favorite(self, client, stream):
favorited = client.stream.favorite(stream.id)
assert isinstance(favorited, Stream)
assert favorited.favoritedDate is not None
unfavorited = client.stream.favorite(stream.id, False)
assert isinstance(favorited, Stream)
assert unfavorited.favoritedDate is None
def test_stream_grant_permission(self, client, stream, second_user):
# deprecated as of Speckle Server 2.6.4
with pytest.raises(UnsupportedException):
client.stream.grant_permission(
stream_id=stream.id,
user_id=second_user.id,
role="stream:contributor",
)
def test_stream_invite(
self, client: SpeckleClient, stream: Stream, second_user_dict: dict
):
invited = client.stream.invite(
stream_id=stream.id,
user_id=second_user_dict["id"],
role="stream:contributor",
email=second_user_dict["email"],
role="stream:reviewer",
message="welcome to my stream!",
)
fetched_stream = client.stream.get(stream.id)
assert invited is True
assert granted is True
assert len(fetched_stream.collaborators) == 2
assert fetched_stream.collaborators[0].name == second_user_dict["name"]
# fail if no email or id
with pytest.raises(SpeckleException):
client.stream.invite(stream_id=stream.id)
def test_stream_revoke_permission(self, client, stream, second_user_dict):
def test_stream_invite_get_all_for_user(
self, second_client: SpeckleClient, stream: Stream
):
# NOTE: these are user queries, but testing here to contain the flow
invites = second_client.user.get_all_pending_invites()
assert isinstance(invites, list)
assert isinstance(invites[0], PendingStreamCollaborator)
assert len(invites) == 1
invite = second_client.user.get_pending_invite(stream_id=stream.id)
assert isinstance(invite, PendingStreamCollaborator)
def test_stream_invite_use(self, second_client: SpeckleClient, stream: Stream):
invite: PendingStreamCollaborator = (
second_client.user.get_all_pending_invites()[0]
)
accepted = second_client.stream.invite_use(
stream_id=stream.id, token=invite.token
)
assert accepted is True
def test_stream_update_permission(
self, client: SpeckleClient, stream: Stream, second_user: User
):
updated = client.stream.update_permission(
stream_id=stream.id, user_id=second_user.id, role="stream:contributor"
)
assert updated is True
def test_stream_revoke_permission(self, client, stream, second_user):
revoked = client.stream.revoke_permission(
stream_id=stream.id, user_id=second_user_dict["id"]
stream_id=stream.id, user_id=second_user.id
)
fetched_stream = client.stream.get(stream.id)
assert revoked == True
assert revoked is True
assert len(fetched_stream.collaborators) == 1
def test_stream_invite_cancel(
self,
client: SpeckleClient,
stream: Stream,
second_user: User,
):
invited = client.stream.invite(
stream_id=stream.id,
user_id=second_user.id,
message="welcome to my stream!",
)
assert invited is True
invites = client.stream.get_all_pending_invites(stream_id=stream.id)
cancelled = client.stream.invite_cancel(
invite_id=invites[0].inviteId, stream_id=stream.id
)
assert cancelled is True
def test_stream_invite_batch(
self, client: SpeckleClient, stream: Stream, second_user: User
):
# NOTE: only works for server admins
# invited = client.stream.invite_batch(
# stream_id=stream.id,
# emails=["userA@speckle.xyz", "userB@speckle.xyz"],
# user_ids=[second_user.id],
# message="yeehaw 🤠",
# )
# assert invited is True
# invited_only_email = client.stream.invite_batch(
# stream_id=stream.id,
# emails=["userC@speckle.xyz"],
# message="yeehaw 🤠",
# )
# assert invited_only_email is True
# fail if no emails or user ids
with pytest.raises(SpeckleException):
client.stream.invite_batch(stream_id=stream.id)
def test_stream_activity(self, client: SpeckleClient, stream: Stream):
activity = client.stream.activity(stream.id)
older_activity = client.stream.activity(
stream.id, before=activity.items[0].time
)
assert isinstance(activity, ActivityCollection)
assert isinstance(older_activity, ActivityCollection)
assert older_activity.totalCount < activity.totalCount
assert activity.items is not None
assert isinstance(activity.items[0], Activity)
def test_stream_delete(self, client, stream):
deleted = client.stream.delete(stream.id)
+158
View File
@@ -0,0 +1,158 @@
import json
from typing import Callable
import pytest
from specklepy.api import operations
from specklepy.logging.exceptions import SpeckleException
from specklepy.objects.base import Base
from specklepy.objects.encoding import CurveArray, ObjectArray
from specklepy.objects.geometry import (
Line,
Mesh,
Point,
Vector,
)
from specklepy.transports.memory import MemoryTransport
from specklepy.objects.structural.geometry import (
Node,
Element1D,
Element2D,
Restraint,
ElementType1D,
ElementType2D,
)
from specklepy.objects.structural.properties import (
Property1D,
Property2D,
SectionProfile,
MemberType,
ShapeType,
)
from specklepy.objects.structural.material import (
Material,
)
from specklepy.objects.structural.analysis import Model
from specklepy.objects.structural.loading import LoadGravity
@pytest.fixture()
def point():
return Point(x=1, y=10, z=0)
@pytest.fixture()
def vector():
return Vector(x=0, y=0, z=-1)
@pytest.fixture()
def line(point, interval):
return Line(
start=point,
end=point,
domain=interval,
# These attributes are not handled in C#
# bbox=None,
# length=None
)
@pytest.fixture()
def mesh(box):
return Mesh(
vertices=[2, 1, 2, 4, 77.3, 5, 33, 4, 2],
faces=[1, 2, 3, 4, 5, 6, 7],
colors=[111, 222, 333, 444, 555, 666, 777],
bbox=box,
area=233,
volume=232.2,
)
@pytest.fixture()
def restraint():
return Restraint(code="FFFFFF")
@pytest.fixture()
def node(restraint, point):
return Node(basePoint=point, restraint=restraint, name="node1")
@pytest.fixture()
def material():
return Material(name="TestMaterial")
@pytest.fixture()
def memberType():
return MemberType(0)
@pytest.fixture()
def shapeType():
return ShapeType(8)
@pytest.fixture()
def sectionProfile(shapeType):
return SectionProfile(name="Test", shapeType=shapeType)
@pytest.fixture()
def property1D(memberType, sectionProfile, material):
return Property1D(
Material=material,
SectionProfile=sectionProfile,
memberType=memberType,
)
@pytest.fixture()
def elementType1D():
return ElementType1D(0)
@pytest.fixture()
def element1D(line, restraint, elementType1D, property1D):
return Element1D(
baseLine=line,
end1Releases=restraint,
end2Releases=restraint,
type=elementType1D,
property=property1D,
)
@pytest.fixture()
def property2D(material):
return Property2D(Material=material)
@pytest.fixture()
def elementType2D():
return ElementType2D(0)
@pytest.fixture()
def element2D(point, elementType2D):
return Element2D(
topology=[point],
type=elementType2D,
)
@pytest.fixture()
def loadGravity(element1D, element2D, vector):
return LoadGravity(elements=[element1D, element2D], gravityFactors=vector)
@pytest.fixture()
def model(loadGravity, element1D, element2D, material, property1D, property2D):
return Model(
loads=[loadGravity],
elements=[element1D, element2D],
materials=[material],
properties=[property1D, property2D],
)
+17 -2
View File
@@ -1,6 +1,7 @@
from specklepy.logging.exceptions import SpeckleException
from specklepy.api.models import User
import pytest
from specklepy.api.client import SpeckleClient
from specklepy.api.models import Activity, ActivityCollection, User
from specklepy.logging.exceptions import SpeckleException
@pytest.mark.run(order=1)
@@ -43,3 +44,17 @@ class TestUser:
assert isinstance(failed_update, SpeckleException)
assert updated is True
assert me.bio == bio
def test_user_activity(self, client: SpeckleClient, second_user_dict):
my_activity = client.user.activity(limit=10)
their_activity = client.user.activity(second_user_dict["id"])
assert isinstance(my_activity, ActivityCollection)
assert isinstance(my_activity.items[0], Activity)
assert my_activity.totalCount > 0
assert isinstance(their_activity, ActivityCollection)
older_activity = client.user.activity(before=my_activity.items[0].time)
assert isinstance(older_activity, ActivityCollection)
assert older_activity.totalCount < my_activity.totalCount