Compare commits

..

1 Commits

Author SHA1 Message Date
izzy lyseggen a2261c7cd9 feat(commits): add new fields 2021-01-04 10:18:41 +00:00
267 changed files with 3045 additions and 21730 deletions
-17
View File
@@ -1,17 +0,0 @@
version: 2.1
# Define the jobs we want to run for this project
jobs:
build:
docker:
- image: cimg/base:2023.03
steps:
- run: echo "so long and thanks for all the fish"
# Orchestrate our job run sequence
workflows:
build_and_test:
when:
false
jobs:
- build
-3
View File
@@ -1,3 +0,0 @@
* text=auto eol=lf
*.{cmd,[cC][mM][dD]} text eol=crlf
*.{bat,[bB][aA][tT]} text eol=crlf
+11 -12
View File
@@ -1,5 +1,4 @@
# Speckle Contribution Guidelines
## Introduction
Thank you for reading this! Speckle's a rather wide network of parts that depend on each other, either directly, indirectly or even just cosmetically.
@@ -10,41 +9,41 @@ This means that what might look like a simple quick change in one repo may have
## Bugs & Issues 🐞
### Found a new bug?
### Found a new bug?
- First step is to check whether this is a new bug! We encourage you to search through the issues of the project in question **and** associated repos!
- If you come up with nothing, **open a new issue with a clear title and description**, as much relevant information as possible: system configuration, code samples & steps to reproduce the problem.
- If you come up with nothing, **open a new issue with a clear title and description**, as much relevant information as possible: system configuration, code samples & steps to reproduce the problem.
- Can't mention this often enough: tells us how to reproduce the problem! We will ignore or flag as such issues without reproduction steps.
- Can't mention this often enough: tells us how to reproduce the problem! We will ignore or flag as such issues without reproduction steps.
- Try to reference & note all potentially affected projects.
### Sending a PR for Bug Fixes
You fixed something! Great! We hope you logged it first :) Make sure though that you've covered the lateral thinking needed for a bug report, as described above, also in your implementation! If there any tests, make sure they all pass. If there are none, it means they're missing - so add them!
You fixed something! Great! We hope you logged it first :) Make sure though that you've covered the lateral thinking needed for a bug report, as described above, also in your implementation! If there any tests, make sure they all pass. If there are none, it means they're missing - so add them!
## New Features 🎉
The golden rule is to Discuss First!
- Before embarking on adding a new feature, suggest it first as an issue with the `enhancement` label and/or title - this will allow relevant people to pitch in
- We'll now discuss your requirements and see how and if they fit within the Speckle ecosystem.
- The last step is to actually start writing code & submit a PR so we can follow along!
- All new features should, if and where possible, come with tests. We won't merge without!
- We'll now discuss your requirements and see how and if they fit within the Speckle ecosystem.
- The last step is to actually start writing code & submit a PR so we can follow along!
- All new features should, if and where possible, come with tests. We won't merge without!
> Many clients may potentially have overlapping scopes, some features might already be in dev somewhere else, or might have been postponed to the next major release due to api instability in that area. For example, adding a delete stream button in the accounts panel in rhino: this feature was planned for speckle admin, and the whole functionality of the accounts panel in rhino is to be greatly reduced!
## Cosmetic Patches ✨
Changes that are cosmetic in nature and do not add anything substantial to the stability or functionality of Speckle **will generally not be accepted**.
Changes that are cosmetic in nature and do not add anything substantial to the stability or functionality of Speckle **will generally not be accepted**.
Why? However trivial the changes might seem, there might be subtle reasons for the original code to be as it is. Furthermore, there are a lot of potential hidden costs (that even maintainers themselves are not aware of fully!) and they eat up review time unncessarily.
> **Examples**: modifying the colour of an UI element in one client may have a big hidden cost and need propagation in several other clients that implement a similar ui element. Changing the default port or specifiying `localhost` instead of `0.0.0.0` breaks cross-vm debugging and developing.
> **Examples**: modifying the colour of an UI element in one client may have a big hidden cost and need propagation in several other clients that implement a similar ui element. Changing the default port or specifiying `localhost` instead of `0.0.0.0` breaks cross-vm debugging and developing.
## Wrap up
Don't worry if you get things wrong. We all do, including project owners: this document should've been here a long time ago. There's plenty of room for discussion on our community [forum](https://discourse.speckle.works).
Don't worry if you get things wrong. We all do, including project owners: this document should've been here a long time ago. There's plenty of room for discussion either on our community [forum](https://discourse.speckle.works) or [chat](https://speckle-works.slack.com/join/shared_invite/enQtNjY5Mzk2NTYxNTA4LTU4MWI5ZjdhMjFmMTIxZDIzOTAzMzRmMTZhY2QxMmM1ZjVmNzJmZGMzMDVlZmJjYWQxYWU0MWJkYmY3N2JjNGI).
🙌❤️💙💚💜🙌
+25
View File
@@ -0,0 +1,25 @@
---
name: New issue
about: Create a report to help us improve
title:
labels:
assignees:
---
If it's your first time here - or you forgot about them - make sure you read the [contribution guidelines](CONTRIBUTING.md), and then feel free to delete this line!
### Expected vs. Actual Behavior
Describe the problem here.
### Reproduction Steps & System Config (win, osx, web, etc.)
Let us know how we can reproduce this, and attach relevant files (if any).
### Proposed Solution (if any)
Let us know what how you would solve this.
#### Optional: Affected Projects
Does this issue propagate to other dependencies or dependents? If so, list them here!
-113
View File
@@ -1,113 +0,0 @@
---
name: Bug report
about: Help improve Speckle!
title: ''
labels: bug
assignees: ''
---
<!---
Provide a short summary in the Title above. Examples of good Issue titles:
* "Bug: Error from server when reticulating splines"
* "Bug: Revit crashes when installing connector"
-->
## Prerequisites
<!---
Please answer the following questions before submitting an issue.
-->
- [ ] I read the [contribution guidelines](https://github.com/specklesystems/speckle-server/blob/main/CONTRIBUTING.md)
- [ ] I checked the [documentation](https://speckle.guide/) and found no answer.
- [ ] I checked [existing issues](../issues?q=is%3Aissue) and found no similar issue. <!-- If you do find an existing issue, please show your support by liking it :+1: instead of creating a new issue -->
- [ ] I checked the [community forum](https://speckle.community/) for related discussions and found no answer.
- [ ] I'm reporting the issue to the correct repository (see also [speckle-server](https://github.com/specklesystems/speckle-server), [speckle-sharp](https://github.com/specklesystems/speckle-sharp), [specklepy](https://github.com/specklesystems/specklepy), [speckle-docs](https://github.com/specklesystems/speckle-docs), and [others](https://github.com/orgs/specklesystems/repositories))
## What package are you referring to?
<!---
Is it related to the server (backend) only, or does this bug relate to the frontend, viewer, objectloader or any other package?
-->
## Describe the bug
<!---
A clear and concise description of what the bug is.
-->
## To Reproduce
<!---
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
-->
## Expected behavior
<!---
A clear and concise description of what you expected to happen.
-->
## Screenshots
<!---
If applicable, add screenshots to help explain your problem.
-->
## System Info
If applicable, please fill in the below details - they help a lot!
### Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
### Smartphone (please complete the following information):
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
## Failure Logs
<!---
Please include any relevant log snippets or files here, or upload as a file.
If including inline, please use markdown code block syntax. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks
For example:
```
your log output here
```
-->
## Additional context
<!---
Add any other context about the problem here.
-->
## Proposed Solution (if any)
<!---
Let us know what how you would solve this.
-->
#### Optional: Affected Projects
<!---
Does this issue propagate to other dependencies or dependents? If so, list them here with links!
-->
-71
View File
@@ -1,71 +0,0 @@
---
name: Feature request
about: Suggest an idea for Speckle!
title: ''
labels: enhancement, question
assignees: ''
---
<!---
Provide a short summary in the Title above. Examples of good Issue titles:
* "Enhancement: Connector for Minecraft"
* "Enhancement: Web viewer should support tesseracts"
-->
## Prerequisites
<!---
Please answer the following questions before submitting an issue.
-->
- [ ] I read the [contribution guidelines](https://github.com/specklesystems/speckle-server/blob/main/CONTRIBUTING.md)
- [ ] I checked the [documentation](https://speckle.guide/) and found no answer.
- [ ] I checked [existing issues](../issues?q=is%3Aissue) and found no similar issue. <!-- If you do find an existing issue, please show your support by liking it :+1: instead of creating a new issue -->
- [ ] I checked the [community forum](https://speckle.community/) for related discussions and found no answer.
- [ ] I'm requesting the feature to the correct repository (see also [speckle-server](https://github.com/specklesystems/speckle-server), [speckle-sharp](https://github.com/specklesystems/speckle-sharp), [specklepy](https://github.com/specklesystems/specklepy), [speckle-docs](https://github.com/specklesystems/speckle-docs), and [others](https://github.com/orgs/specklesystems/repositories))
## What package are you referring to?
<!---
Is it related to the server (backend) only, or does this feature request relate to the frontend, viewer, objectloader or any other package?
-->
## Is your feature request related to a problem? Please describe.
<!---
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
-->
## Describe the solution you'd like
<!---
A clear and concise description of what you want to happen.
-->
## Describe alternatives you've considered
<!---
A clear and concise description of any alternative solutions or features you've considered.
-->
## Additional context
<!---
Add any other context or screenshots about the feature request here.
Have you seen this feature implemented in any other software? Can you provide screenshots or links to video or documentation?
What works well about these existing features in other software? What doesn't work well?
-->
## Related issues or community discussions
<!---
Is this feature request related to (but sufficiently distinct from) any existing issues?
Does this feature request require other features to be available beforehand?
Has this feature been discussed in the community forum, please link here? https://speckle.community/
-->
@@ -0,0 +1,25 @@
Description of PR...
## Changes
- Item 1
- Item 2
## Checklist
- [ ] Unit tests
- [ ] Documentation
## References
(optional)
Include **important** links regarding the implementation of this PR.
This usually includes and RFC or an aggregation of issues and/or individual conversations
that helped put this solution together. This helps ensure there is a good aggregation
of resources regarding the implementation.
```text
Fixes #85, Fixes #22, Fixes username/repo#123
Connects #123
```
-102
View File
@@ -1,102 +0,0 @@
<!---
Provide a short summary in the Title above. Examples of good PR titles:
* "Feature: adds metrics to component"
* "Fix: resolves duplication in comment thread"
* "Update: apollo v2.34.0"
-->
## Description & motivation
<!---
Describe your changes, and why you're making them. What benefit will this have to others?
Is this linked to an open Github issue, a thread in Speckle community,
or another pull request? Link it here.
If it is related to a Github issue, and resolves it, please link to the issue number, e.g.:
Fixes #85, Fixes #22, Fixes username/repo#123
Connects #123
-->
## Changes:
<!---
- Item 1
- Item 2
-->
## To-do before merge:
<!---
(Optional -- remove this section if not needed)
Include any notes about things that need to happen before this PR is merged, e.g.:
- [ ] Change the base branch
- [ ] Ensure PR #56 is merged
-->
## Screenshots:
<!---
Include a screenshot the before and after. This can be a screenshot of a plugin, web frontend, or output in a terminal.
-->
## Validation of changes:
<!---
Describe what tests have been added or amended, and why these demonstrate it works and will prevent this feature being accidentally broken by future changes.
-->
## Checklist:
<!---
This checklist is mostly useful as a reminder of small things that can easily be
forgotten it is meant as a helpful tool rather than hoops to jump through.
Put an `x` between the square brackets, e.g. [x], for all the items that apply,
make notes next to any that haven't been addressed, and remove any items that are not relevant to this PR.
-->
- [ ] My pull request follows the guidelines in the [Contributing guide](https://github.com/specklesystems/speckle-server/blob/main/CONTRIBUTING.md)?
- [ ] My pull request does not duplicate any other open [Pull Requests](../../pulls) for the same update/change?
- [ ] My commits are related to the pull request and do not amend unrelated code or documentation.
- [ ] My code follows a similar style to existing code.
- [ ] I have added appropriate tests.
- [ ] I have updated or added relevant documentation.
## References
<!---
(Optional -- remove this section if not needed )
Include **important** links regarding the implementation of this PR.
This usually includes a RFC or an aggregation of issues and/or individual conversations
that helped put this solution together. This helps ensure we retain and share knowledge
regarding the implementation, and may help others understand motivation and design decisions etc..
-->
-98
View File
@@ -1,98 +0,0 @@
name: "Specklepy test"
on:
workflow_call:
secrets:
CODECOV_TOKEN:
required: true
pull_request:
branches:
- "main"
jobs:
test-internal: # Run integration tests against the internal server image
name: Test (internal)
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
steps:
- uses: actions/checkout@v4
- name: Install uv and set the python version
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: Install the project
run: uv sync --all-extras --dev
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: "ghcr.io"
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Run Speckle Server
run: docker compose --file docker-compose-internal.yml up --detach --wait
- name: Run tests
run: uv run pytest --cov --cov-report xml:reports/coverage.xml --junitxml=reports/test-results.xml
- uses: codecov/codecov-action@v5
if: matrix.python-version == 3.12
with:
fail_ci_if_error: true # optional (default = false)
files: ./reports/test-results.xml # optional
token: ${{ secrets.CODECOV_TOKEN }}
- name: Minimize uv cache
run: uv cache prune --ci
test-public: # Run integration tests against the public server image
name: Test (public)
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
steps:
- uses: actions/checkout@v4
- name: Install uv and set the python version
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-dependency-glob: "uv.lock"
- name: Install the project
run: uv sync --all-extras --dev
- uses: actions/cache@v3
with:
path: ~/.cache/pre-commit/
key: ${{ hashFiles('.pre-commit-config.yaml') }}
- name: Run pre-commit
run: uv run pre-commit run --all-files
- name: Run Speckle Server
run: docker compose --file docker-compose.yml up --detach --wait
- name: Run tests
run: uv run pytest --cov --cov-report xml:reports/coverage.xml --junitxml=reports/test-results.xml
- name: Minimize uv cache
run: uv cache prune --ci
-55
View File
@@ -1,55 +0,0 @@
name: "Publish Python Package"
on:
push:
branches:
- "main"
tags:
- "3.*.*"
jobs:
test:
uses: "./.github/workflows/pr.yml"
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
publish-package:
name: "Build and Publish Python Package"
runs-on: ubuntu-latest
needs: test
# set the environment based on what triggered the workflow
environment:
name: ${{ github.ref_type == 'tag' && 'pypi' || 'testpypi' }}
permissions:
id-token: write
steps:
- name: "Install uv"
uses: astral-sh/setup-uv@v5
- name: "Checkout code"
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: "Build package artifacts"
run: uv build
# Logic for TestPyPI (on main branch push)
- name: "Publish to TestPyPI"
if: ${{ github.ref_type == 'branch' }}
run: uv publish --index test
- name: "Verify TestPyPI package installation"
if: ${{ github.ref_type == 'branch' }}
run: uv run --index test --with specklepy --no-project -- python -c "import specklepy"
# Logic for PyPI (on v3* tag creation)
- name: "Publish to PyPI"
if: ${{ github.ref_type == 'tag' }}
run: uv publish
- name: "Verify PyPI package installation"
if: ${{ github.ref_type == 'tag' }}
run: uv run --with specklepy --no-project -- python -c "import specklepy"
+1 -7
View File
@@ -1,9 +1,3 @@
.tool-versions
.envrc
reports/
.volumes/
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
@@ -114,4 +108,4 @@ venv.bak/
# other
scratch.py
settings.json
settings.json
-31
View File
@@ -1,31 +0,0 @@
repos:
- repo: local
hooks:
# Run the linter.
- id: ruff
name: ruff lint
entry: uv run ruff check --force-exclude
language: system
types_or: [python, pyi]
# Run the formatter.
- id: ruff-format
name: ruff format
entry: uv run ruff format --force-exclude
language: system
types_or: [python, pyi]
- repo: https://github.com/commitizen-tools/commitizen
hooks:
- id: commitizen
- id: commitizen-branch
stages:
- pre-push
rev: v3.13.0
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: end-of-file-fixer
- id: trailing-whitespace
-6
View File
@@ -1,6 +0,0 @@
{
"recommendations": [
"ms-python.python",
"charliermarsh.ruff"
]
}
+2 -11
View File
@@ -6,20 +6,11 @@
"configurations": [
{
"name": "Python: Current File",
"type": "debugpy",
"type": "python",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal",
"justMyCode": false
},
{
"name": "Pytest",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"args": [],
"console": "integratedTerminal",
"justMyCode": true
}
]
}
}
+116 -42
View File
@@ -1,68 +1,142 @@
<h1 align="center">
<img src="https://user-images.githubusercontent.com/2679513/131189167-18ea5fe1-c578-47f6-9785-3748178e4312.png" width="150px"/><br/>
Speckle | specklepy 🐍
</h1>
# speckle-py 🥧
<p align="center"><a href="https://twitter.com/SpeckleSystems"><img src="https://img.shields.io/twitter/follow/SpeckleSystems?style=social" alt="Twitter Follow"></a> <a href="https://speckle.community"><img src="https://img.shields.io/discourse/users?server=https%3A%2F%2Fspeckle.community&amp;style=flat-square&amp;logo=discourse&amp;logoColor=white" alt="Community forum users"></a> <a href="https://speckle.systems"><img src="https://img.shields.io/badge/https://-speckle.systems-royalblue?style=flat-square" alt="website"></a> <a href="https://speckle.guide/dev/"><img src="https://img.shields.io/badge/docs-speckle.guide-orange?style=flat-square&amp;logo=read-the-docs&amp;logoColor=white" alt="docs"></a></p>
[![Twitter Follow](https://img.shields.io/twitter/follow/SpeckleSystems?style=social)](https://twitter.com/SpeckleSystems) [![Discourse users](https://img.shields.io/discourse/users?server=https%3A%2F%2Fdiscourse.speckle.works&style=flat-square)](https://discourse.speckle.works)
[![Slack Invite](https://img.shields.io/badge/-slack-grey?style=flat-square&logo=slack)](https://speckle-works.slack.com/join/shared_invite/enQtNjY5Mzk2NTYxNTA4LTU4MWI5ZjdhMjFmMTIxZDIzOTAzMzRmMTZhY2QxMmM1ZjVmNzJmZGMzMDVlZmJjYWQxYWU0MWJkYmY3N2JjNGI) [![website](https://img.shields.io/badge/www-speckle.systems-royalblue?style=flat-square)](https://speckle.systems)
> Speckle is the first AEC data hub that connects with your favorite AEC tools. Speckle exists to overcome the challenges of working in a fragmented industry where communication, creative workflows, and the exchange of data are often hindered by siloed software and processes. It is here to make the industry better.
## Introduction
<h3 align="center">
The Python SDK
</h3>
<p align="center">
<a href="https://pypi.org/project/specklepy/"><img alt="PyPI - Version" src="https://img.shields.io/pypi/v/specklepy"></a>
<a href="https://codecov.io/gh/specklesystems/specklepy"><img src="https://codecov.io/gh/specklesystems/specklepy/branch/main/graph/badge.svg?token=8KQFL5N0YF" alt="Codecov"></a>
<a href="https://github.com/specklesystems/specklepy/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/github/license/specklesystems/specklepy"></a>
</p>
# Repo structure
## Usage
Send and receive data from a Speckle Server with `operations`, interact with the Speckle API with the `SpeckleClient`, create and extend your own custom Speckle Objects with `Base`, and more!
Head to the [**📚 specklepy docs**](https://speckle.guide/dev/python.html) for more information and usage examples.
> ⚠ This is the start of the Python client for Speckle 2.0. It is currently quite nebulous and may be trashed and rebuilt at any moment! It is compatible with Python 3.6+ ⚠
## Developing & Debugging
### Installation
To get started, create a virtual environment and pip install the requirements.
This project uses uv for dependency management, make sure you follow the official [docs](https://docs.astral.sh/uv/) to get it.
on windows:
```
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt
```
To create a new virtual environment with uv run `$ uv venv` and follow the instructions on the screen to activate the virtual environment.
To bootstrap the project environment run `$ uv sync`. This will install both the package and dev dependencies.
on mac:
To execute any python script run `$ uv run python my_script.py`
```
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
> Alternatively you may roll your own virtual-env with either venv, virtualenv, pyenv-virtualenv etc. Uv will play along an recognize if it is invoked from inside a virtual environment.
### Style guide
All our repo wide styling linting and other rules are checked and enforced by `pre-commit`, which is included in the dev dependencies.
It is recommended to set up `pre-commit` after installing the dependencies by running `$ pre-commit install`.
Commiting code that doesn't adhere to the given rules, will fail the checks in our CI system.
## Overview of functionality
### Local Data Paths
The `SpeckleClient` is the entry point for interacting with the GraphQL API. You'll need to have a running server to use this.
It may be helpful to know where the local accounts and object cache dbs are stored. Depending on on your OS, you can find the dbs at:
```py
from speckle.api.client import SpeckleClient
from speckle.api.credentials import get_default_account, get_local_accounts
- Windows: `APPDATA` or `<USER>\AppData\Roaming\Speckle`
- Linux: `$XDG_DATA_HOME` or by default `~/.local/share/Speckle`
- Mac: `~/.config/Speckle`
all_accounts = get_local_accounts() # get back a list
account = get_default_account()
client = SpeckleClient(host="localhost:3000", use_ssl=False)
# client = SpeckleClient(host="yourserver.com") or whatever your host is
client.authenticate(account.token)
```
Interacting with streams is meant to be intuitive and evocative of PySpeckle 1.0
```py
# get your streams
stream_list = client.stream.list()
# search your streams
results = client.user.search("mech")
# create a stream
new_stream_id = client.stream.create(name="a shiny new stream")
# get a stream
new_stream = client.stream.get(id=new_stream_id)
```
New in 2.0: commits! Here are some basic commit interactions.
```py
# get list of commits
commits = client.commit.list("stream id")
# get a specific commit
commit = client.commit.get("stream id", "commit id")
# create a commit
commit_id = client.commit.create("stream id", "object id", "this is a commit message to describe the commit")
# delete a commit
deleted = client.commit.delete("stream id", "commit id")
```
The `BaseObjectSerializer` is used for decomposing and serializing `Base` objects so they can be sent / received to the server. You can use it directly to get the id (hash) and a serializable object representation of the decomposed `Base`. You can learn more about the Speckle `Base` object [here](https://discourse.speckle.works/t/core-2-0-the-base-object/782) and the decomposition API [here](https://discourse.speckle.works/t/core-2-0-decomposition-api/911).
```py
detached_base = Base()
detached_base.name = "this will get detached"
base_obj = Base()
base_obj.name = "my base"
base_obj["@nested"] = detached_base
serializer = BaseObjectSerializer()
hash, obj_dict = serializer.traverse_base(base_obj)
```
If you use the `operations`, you will not need to interact with the serializer directly as this will be taken care of for you. You will just need to provide a transport to indicate where the objects should be sent / received from. At the moment, just the `MemoryTransport` and the `ServerTransport` are fully functional at the moment. If you'd like to learn more about Transports in Speckle 2.0, have a look [here](https://discourse.speckle.works/t/core-2-0-transports/919).
```py
transport = MemoryTransport()
# this serialises the object and sends it to the transport
hash = operations.send(base=base_obj, transports=[transport])
# if the object had detached objects, you can see these as well
saved_objects = transport.objects # a dict with the obj hash as the key
# this receives and object from the given transport, deserialises it, and recomposes it into a base object
received_base = operations.receive(obj_id=hash, remote_transport=transport)
```
You can also use the GraphQL API to send and receive objects.
```py
# create a test base object
test_base = Base()
test_base.testing = "a test base obj"
# run it through the serialiser
s = BaseObjectSerializer()
hash, obj = s.traverse_base(test_base)
# send it to the server
objCreate = client.object.create(stream_id="stream id", objects=[obj])
received_base = client.object.get(hash)
```
This doc is not complete - there's more to see so have a dive into the code and play around! Please feel free to provide feedback, submit issues, or discuss new features ✨
## Contributing
Please make sure you read the [contribution guidelines](.github/CONTRIBUTING.md) and [code of conduct](.github/CODE_OF_CONDUCT.md) for an overview of the practices we try to follow.
Please make sure you read the [contribution guidelines](.github/CONTRIBUTING.md) for an overview of the best practices we try to follow.
## Community
The Speckle Community hangs out on [the forum](https://discourse.speckle.works), do join and introduce yourself & feel free to ask us questions!
The Speckle Community hangs out in two main places, usually:
## Security
- on [the forum](https://discourse.speckle.works)
- on [the chat](https://speckle-works.slack.com/join/shared_invite/enQtNjY5Mzk2NTYxNTA4LTU4MWI5ZjdhMjFmMTIxZDIzOTAzMzRmMTZhY2QxMmM1ZjVmNzJmZGMzMDVlZmJjYWQxYWU0MWJkYmY3N2JjNGI)
For any security vulnerabilities or concerns, please contact us directly at security[at]speckle.systems.
Do join and introduce yourself!
## License
-12
View File
@@ -1,12 +0,0 @@
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 2.2.+ | :white_check_mark: |
| < 2.2 | :x: |
## Reporting a Vulnerability
Hi! If you've found something off, we'd be more than happy if you would report it via security@speckle.systems. We will work together with you to correctly identify the cause and implement a fix. Thanks for helping make Speckle safer!
-118
View File
@@ -1,118 +0,0 @@
name: "speckle-server"
services:
####
# Speckle Server dependencies
#######
postgres:
image: "postgres:16.4-alpine3.20@sha256:d898b0b78a2627cb4ee63464a14efc9d296884f1b28c841b0ab7d7c42f1fffdf"
restart: always
environment:
POSTGRES_DB: speckle
POSTGRES_USER: speckle
POSTGRES_PASSWORD: speckle
volumes:
- ./.volumes/postgres-data:/var/lib/postgresql/data/
healthcheck:
# the -U user has to match the POSTGRES_USER value
test: ["CMD-SHELL", "pg_isready -U speckle"]
interval: 5s
timeout: 5s
retries: 30
redis:
image: "valkey/valkey:8.1-alpine@sha256:0d27f0bca0249f61d060029a6aaf2e16b2c417d68d02a508e1dfb763fa2948b4"
restart: always
volumes:
- ./.volumes/redis-data:/data
healthcheck:
test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
interval: 5s
timeout: 5s
retries: 30
minio:
image: "minio/minio:RELEASE.2023-10-25T06-33-25Z"
command: server /data --console-address ":9001"
restart: always
volumes:
- ./.volumes/minio-data:/data
ports:
- '127.0.0.1:9000:9000'
- '127.0.0.1:9001:9001'
healthcheck:
test:
[
"CMD-SHELL",
"curl -s -o /dev/null http://127.0.0.1:9000/minio/index.html",
]
interval: 5s
timeout: 30s
retries: 30
start_period: 10s
speckle-server:
image: ghcr.io/specklesystems/speckle-server:latest
restart: always
healthcheck:
test:
- CMD
- /nodejs/bin/node
- -e
- "try { require('node:http').request({headers: {'Content-Type': 'application/json'}, port:3000, hostname:'127.0.0.1', path:'/readiness', method: 'GET', timeout: 2000 }, (res) => { body = ''; res.on('data', (chunk) => {body += chunk;}); res.on('end', () => {process.exit(Number(res.statusCode != 200 || body.toLowerCase().includes('error')));}); }).end(); } catch { process.exit(1); }"
interval: 10s
timeout: 10s
retries: 3
start_period: 90s
ports:
- "0.0.0.0:3000:3000"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
minio:
condition: service_healthy
environment:
# TODO: Change this to the URL of the speckle server, as accessed from the network
CANONICAL_URL: "http://127.0.0.1:8080"
SPECKLE_AUTOMATE_URL: "http://127.0.0.1:3030"
FRONTEND_ORIGIN: "http://127.0.0.1:8081"
# TODO: Change thvolumes:
REDIS_URL: "redis://redis"
S3_ENDPOINT: "http://minio:9000"
S3_PUBLIC_ENDPOINT: "http://127.0.0.1:9000"
S3_ACCESS_KEY: "minioadmin"
S3_SECRET_KEY: "minioadmin"
S3_BUCKET: "speckle-server"
S3_CREATE_BUCKET: "true"
FILE_SIZE_LIMIT_MB: 100
MAX_PROJECT_MODELS_PER_PAGE: 500
# TODO: Change this to a unique secret for this server
SESSION_SECRET: "TODO:ReplaceWithLongString"
STRATEGY_LOCAL: "true"
POSTGRES_URL: "postgres"
POSTGRES_USER: "speckle"
POSTGRES_PASSWORD: "speckle"
POSTGRES_DB: "speckle"
ENABLE_MP: "false"
LOG_PRETTY: "true"
FF_NEXT_GEN_FILE_IMPORTER_ENABLED: "true"
FF_LARGE_FILE_IMPORTS_ENABLED: "true"
networks:
default:
name: speckle-server
volumes:
postgres-data:
redis-data:
minio-data:
-118
View File
@@ -1,118 +0,0 @@
name: "speckle-server"
services:
####
# Speckle Server dependencies
#######
postgres:
image: "postgres:16.4-alpine3.20@sha256:d898b0b78a2627cb4ee63464a14efc9d296884f1b28c841b0ab7d7c42f1fffdf"
restart: always
environment:
POSTGRES_DB: speckle
POSTGRES_USER: speckle
POSTGRES_PASSWORD: speckle
volumes:
- ./.volumes/postgres-data:/var/lib/postgresql/data/
healthcheck:
# the -U user has to match the POSTGRES_USER value
test: ["CMD-SHELL", "pg_isready -U speckle"]
interval: 5s
timeout: 5s
retries: 30
redis:
image: "valkey/valkey:8.1-alpine@sha256:0d27f0bca0249f61d060029a6aaf2e16b2c417d68d02a508e1dfb763fa2948b4"
restart: always
volumes:
- ./.volumes/redis-data:/data
healthcheck:
test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
interval: 5s
timeout: 5s
retries: 30
minio:
image: "minio/minio:RELEASE.2023-10-25T06-33-25Z"
command: server /data --console-address ":9001"
restart: always
volumes:
- ./.volumes/minio-data:/data
ports:
- '127.0.0.1:9000:9000'
- '127.0.0.1:9001:9001'
healthcheck:
test:
[
"CMD-SHELL",
"curl -s -o /dev/null http://127.0.0.1:9000/minio/index.html",
]
interval: 5s
timeout: 30s
retries: 30
start_period: 10s
speckle-server:
image: speckle/speckle-server:latest
restart: always
healthcheck:
test:
- CMD
- /nodejs/bin/node
- -e
- "try { require('node:http').request({headers: {'Content-Type': 'application/json'}, port:3000, hostname:'127.0.0.1', path:'/readiness', method: 'GET', timeout: 2000 }, (res) => { body = ''; res.on('data', (chunk) => {body += chunk;}); res.on('end', () => {process.exit(Number(res.statusCode != 200 || body.toLowerCase().includes('error')));}); }).end(); } catch { process.exit(1); }"
interval: 10s
timeout: 10s
retries: 3
start_period: 90s
ports:
- "0.0.0.0:3000:3000"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
minio:
condition: service_healthy
environment:
# TODO: Change this to the URL of the speckle server, as accessed from the network
CANONICAL_URL: "http://127.0.0.1:8080"
SPECKLE_AUTOMATE_URL: "http://127.0.0.1:3030"
FRONTEND_ORIGIN: "http://127.0.0.1:8081"
# TODO: Change thvolumes:
REDIS_URL: "redis://redis"
S3_ENDPOINT: "http://minio:9000"
S3_PUBLIC_ENDPOINT: "http://127.0.0.1:9000"
S3_ACCESS_KEY: "minioadmin"
S3_SECRET_KEY: "minioadmin"
S3_BUCKET: "speckle-server"
S3_CREATE_BUCKET: "true"
FILE_SIZE_LIMIT_MB: 100
MAX_PROJECT_MODELS_PER_PAGE: 500
# TODO: Change this to a unique secret for this server
SESSION_SECRET: "TODO:ReplaceWithLongString"
STRATEGY_LOCAL: "true"
POSTGRES_URL: "postgres"
POSTGRES_USER: "speckle"
POSTGRES_PASSWORD: "speckle"
POSTGRES_DB: "speckle"
ENABLE_MP: "false"
LOG_PRETTY: "true"
FF_NEXT_GEN_FILE_IMPORTER_ENABLED: "true"
FF_LARGE_FILE_IMPORTS_ENABLED: "true"
networks:
default:
name: speckle-server
volumes:
postgres-data:
redis-data:
minio-data:
-63
View File
@@ -1,63 +0,0 @@
import os
import random
import string
import time
from pathlib import Path
from typing import List
from specklepy.api import operations
from specklepy.objects import Base
from specklepy.transports.sqlite import SQLiteTransport
class Sub(Base):
bar: List[str]
def random_string():
letters = string.ascii_lowercase
return "".join(random.choice(letters) for _ in range(10))
BASE_PATH = SQLiteTransport.get_base_path("Speckle")
def clean_db():
os.remove(Path(BASE_PATH, "Objects.db"))
def one_pass(clean: bool, randomize: bool, child_count: int):
foo = Base()
for i in range(child_count):
stuff = random_string() if randomize else "stuff"
foo[f"@child_{i}"] = Sub(bar=["asdf", "bar", i, stuff])
if clean:
clean_db()
transport = SQLiteTransport()
start = time.time()
hash = operations.send(base=foo, transports=[transport])
send_time = time.time() - start
receive_start = time.time()
operations.receive(hash, transport)
receive_time = time.time() - receive_start
return send_time, receive_time
if __name__ == "__main__":
sample_size = 4
test_permutations = [
(True, True),
(False, False),
(False, True),
(True, False),
]
for clean, randomize in test_permutations:
print(f"CLEAN: {clean}, RANDOMIZE: {randomize}")
for child_count in [10, 100, 1000, 10000]:
print(f"\tCHILD COUNT: {child_count}")
for _ in range(sample_size):
send_time, receive_time = one_pass(clean, randomize, child_count)
print(f"\t\tSend: {send_time} Receive: {receive_time}")
-15
View File
@@ -1,15 +0,0 @@
from devtools import debug
from specklepy.api import operations
from specklepy.api.wrapper import StreamWrapper
if __name__ == "__main__":
stream_url = "https://latest.speckle.dev/streams/7d051a6449"
wrapper = StreamWrapper(stream_url)
transport = wrapper.get_transport()
rec = operations.receive("98396753f8bf7fe1cb60c5193e9f9d86", transport)
# hash = operations.send(base=foo, transports=[transport], use_default_cache=False)
debug(rec)
-39
View File
@@ -1,39 +0,0 @@
import random
import string
from typing import List
from specklepy.api import operations
from specklepy.api.wrapper import StreamWrapper
from specklepy.objects import Base
class Sub(Base):
bar: List[str]
def random_string():
letters = string.ascii_lowercase
return "".join(random.choice(letters) for _ in range(10))
def create_object(child_count: int) -> Base:
foo = Base()
for i in range(child_count):
stuff = random_string()
foo[f"@child_{i}"] = Sub(bar=["asdf", "bar", i, stuff])
return foo
if __name__ == "__main__":
stream_url = "http://hyperion:3000/streams/2372b54c35"
child_count = 10
foo = create_object(child_count)
wrapper = StreamWrapper(stream_url)
transport = wrapper.get_transport()
hash = operations.send(base=foo, transports=[transport], use_default_cache=False)
rec = operations.receive(hash, transport)
print(rec)
-36
View File
@@ -1,36 +0,0 @@
from devtools import debug
from specklepy.api import operations
from specklepy.objects_v2.geometry import Base
from specklepy.objects_v2.units import Units
dct = {
"id": "1234abcd",
"units": None,
"speckle_type": "Base",
"applicationId": None,
"totalChildrenCount": 0,
}
base = Base()
for prop, value in dct.items():
base.__setattr__(prop, value)
debug(base)
debug(base.units)
base.units = "m"
debug(base.units)
base.units = None
debug(base.units)
foo = operations.serialize(base)
base.units = 10
debug(base.units)
debug(foo)
base.units = Units.mm
debug(base.units)
-60
View File
@@ -1,60 +0,0 @@
"""This is an example showcasing the usage of speckle `Base` class."""
# the speckle.objects module exposes all speckle provided classes
from devtools import debug
from specklepy.api import operations
from specklepy.objects import Base
class ExampleSub(Base):
"""
Inheriting from `Base` is done with in the standard way by default.
The syntax is similar to the stdlib dataclass syntax.
No __init__ method definition is required, that is done automatically by the base
type. Also the attributes defined this way are instance attributes despite they
might look like class attributes.
The speckle Base uses the pydantic BaseModel in the background, but ideally that
is not the consumers concern.
**Important note:** currently the way how serialization works, requires
each attribute to have a valid default value, just like `foo` has. This includes
default values for all primitives and complex datastructures.
Failing to provide a default, breaks the receiving end of the transport.
"""
foo: str = "bar"
class SpeckleSub(ExampleSub, speckle_type="custom_speckle_sub"):
"""
Example custom type name registration.
This is an optional feature.
The default value of the speckle_type is generated from the name of the class, but
optionally it may be overridden. This is useful, since the speckle_type has to be
unique for each subclass of speckle Base.
"""
magic: str = "trick"
if __name__ == "__main__":
# example usage
custom_sub = SpeckleSub(
foo=123,
magic="trick",
bar="baric",
extra=123,
)
# support for dynamic attributes
custom_sub.extra_extra = "what is this?"
debug(custom_sub)
serialized = operations.serialize(custom_sub)
deserialized = operations.deserialize(serialized)
# the only difference should be between the two data is that the deserialized
# instance id attribute is not None.
debug(deserialized)
-6
View File
@@ -1,6 +0,0 @@
[tools]
python = "3.13.7"
[settings]
experimental = true
python.uv_venv_auto = true
-100
View File
@@ -1,100 +0,0 @@
[project]
dynamic = ["version"]
# version = "3.0.0a1"
name = "specklepy"
description = "The Python SDK for Speckle"
readme = "README.md"
authors = [{ name = "Speckle Systems", email = "devops@speckle.systems" }]
license = { text = "Apache-2.0" }
requires-python = ">=3.10.0, <4.0"
dependencies = [
"appdirs>=1.4.4",
"attrs>=24.3.0",
"deprecated>=1.2.15",
"gql[requests,websockets]>=3.5.0,<4.0.0",
"httpx>=0.28.1",
"mkdocs>=1.6.1",
"mkdocs-material>=9.6.5",
"mkdocstrings>=0.28.1",
"mkdocstrings-python>=1.15.0",
"pydantic>=2.10.5",
"pydantic-settings>=2.7.1",
"ujson>=5.10.0",
]
[project.optional-dependencies]
speckleifc = ["ifcopenshell>=0.8.3.post2"]
[dependency-groups]
dev = [
"commitizen>=4.1.0",
"devtools>=0.12.2",
"hatch>=1.14.0",
"hatch-vcs>=0.4.0",
"pre-commit>=4.0.1",
"pytest>=8.3.4",
"pytest-asyncio>=0.25.2",
"pytest-cov>=6.0.0",
"pytest-ordering>=0.6",
"ruff>=0.9.2",
"types-deprecated>=1.2.15.20241117",
"types-requests>=2.32.0.20241016",
"types-ujson>=5.10.0.20240515",
]
[project.urls]
repository = "https://github.com/specklesystems/specklepy"
documentation = "https://speckle.guide/dev/py-examples.html"
homepage = "https://speckle.systems/"
[build-system]
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"
[tool.hatch.version]
source = "vcs"
[tool.hatch.build.targets.wheel]
only-include = ["src", "licenses"]
sources = ["src"]
[tool.hatch.build.targets.sdist]
include = ["src", "licenses"]
[tool.hatch.version.raw-options]
local_scheme = "no-local-version"
[tool.commitizen]
name = "cz_conventional_commits"
version = "2.9.2"
tag_format = "$version"
[tool.ruff]
exclude = [".venv", "**/*.yml"]
[tool.ruff.lint]
select = [
# pycodestyle
"E",
# Pyflakes
"F",
# pyupgrade
"UP",
# flake8-bugbear
"B",
# flake8-simplify
"SIM",
# isort
"I",
]
ignore = ["UP006", "UP007", "UP035"]
[[tool.uv.index]]
name = "pypi"
url = "https://pypi.org/simple/"
publish-url = "https://upload.pypi.org/legacy/"
[[tool.uv.index]]
name = "test"
url = "https://test.pypi.org/simple/"
publish-url = "https://test.pypi.org/legacy/"
+31
View File
@@ -0,0 +1,31 @@
aiohttp==3.7.1
appdirs==1.4.4
astroid==2.4.2
async-timeout==3.0.1
attrs==20.3.0
black==20.8b1
certifi==2020.11.8
chardet==3.0.4
click==7.1.2
colorama==0.4.4
gql==3.0.0a4
graphql-core==3.1.2
idna==2.10
isort==5.6.4
lazy-object-proxy==1.4.3
mccabe==0.6.1
multidict==5.0.0
mypy-extensions==0.4.3
pathspec==0.8.1
pydantic==1.7.2
pylint==2.6.0
regex==2020.11.11
requests==2.24.0
six==1.15.0
toml==0.10.2
typed-ast==1.4.1
typing-extensions==3.7.4.3
urllib3==1.25.11
websockets==8.1
wrapt==1.12.1
yarl==1.5.1
+95
View File
@@ -0,0 +1,95 @@
import re
from gql.client import SyncClientSession
from speckle.logging.exceptions import SpeckleException
from typing import Dict
from speckle.api import resources
from speckle.api.resources import commit, stream, object, server, user, subscriptions
from gql import Client, gql
from gql.transport.requests import RequestsHTTPTransport
from gql.transport.aiohttp import AIOHTTPTransport
from gql.transport.websockets import WebsocketsTransport
class SpeckleClient:
DEFAULT_HOST = "staging.speckle.dev"
USE_SSL = True
def __init__(self, host: str = DEFAULT_HOST, use_ssl: bool = USE_SSL) -> None:
ws_protocol = "ws"
http_protocol = "http"
if use_ssl:
ws_protocol = "wss"
http_protocol = "https"
# sanitise host input by removing protocol and trailing slash
host = re.sub(r"((^\w+:|^)\/\/)|(\/$)", "", host)
self.url = f"{http_protocol}://{host}"
self.graphql = self.url + "/graphql"
self.ws_url = f"{ws_protocol}://{host}/graphql"
self.me = None
self.httpclient = Client(
transport=RequestsHTTPTransport(url=self.graphql, verify=True, retries=3)
)
self.wsclient = None
self._init_resources()
def authenticate(self, token: str) -> None:
"""Authenticate the client using a personal access token
The token is saved in the client object and a synchronous GraphQL entrypoint is created
Arguments:
token {str} -- an api token
"""
self.me = {"token": token}
headers = {
"Authorization": f"Bearer {self.me['token']}",
"Content-Type": "application/json",
}
httptransport = RequestsHTTPTransport(
url=self.graphql, headers=headers, verify=True, retries=3
)
wstransport = WebsocketsTransport(
url=self.ws_url,
init_payload={"Authorization": f"Bearer {self.me['token']}"},
)
self.httpclient = Client(transport=httptransport)
self.wsclient = Client(transport=wstransport)
self._init_resources()
def execute_query(self, query: str) -> Dict:
return self.httpclient.execute(query)
def _init_resources(self) -> None:
self.stream = stream.Resource(
me=self.me, basepath=self.url, client=self.httpclient
)
self.commit = commit.Resource(
me=self.me, basepath=self.url, client=self.httpclient
)
self.object = object.Resource(
me=self.me, basepath=self.url, client=self.httpclient
)
self.server = server.Resource(
me=self.me, basepath=self.url, client=self.httpclient
)
self.user = user.Resource(me=self.me, basepath=self.url, client=self.httpclient)
self.subscribe = subscriptions.Resource(
me=self.me,
basepath=self.ws_url,
client=self.wsclient,
)
def __getattr__(self, name):
try:
attr = getattr(resources, name)
return attr.Resource(me=self.me, basepath=self.url, client=self.httpclient)
except:
raise SpeckleException(
f"Method {name} is not supported by the SpeckleClient class"
)
+63
View File
@@ -0,0 +1,63 @@
from __future__ import annotations
from typing import List, Optional
from pydantic import BaseModel
from speckle.transports.sqlite import SQLiteTransport
account_storage = SQLiteTransport(scope="Accounts")
def get_local_accounts() -> List[Account]:
"""Gets all the accounts present in this environment
Returns:
List[Account] -- list of all local accounts or an empty list if no accounts were found
"""
res = account_storage.get_all_objects()
return [Account.parse_raw(r[1]) for r in res] if res else []
def get_default_account() -> Account:
"""Gets this environment's default account if any. If there is no default, the first found will be returned and set as default.
Returns:
Account -- the default account or None if no local accounts were found
"""
accounts = get_local_accounts()
if not accounts:
return None
default = next((acc for acc in accounts if acc.isDefault), None)
if not default:
default = accounts[0]
default.isDefault = True
return default
class ServerInfo(BaseModel):
name: str
company: Optional[str]
url: str
class UserInfo(BaseModel):
name: str
email: str
company: Optional[str]
id: str
class Account(BaseModel):
isDefault: bool
token: str
refreshToken: str
serverInfo: ServerInfo
userInfo: UserInfo
id: str
def __repr__(self) -> str:
return f"Account(email: {self.userInfo.email}, server: {self.serverInfo.url}, isDefault: {self.isDefault})"
def __str__(self) -> str:
return f"Account(email: {self.userInfo.email}, server: {self.serverInfo.url}, isDefault: {self.isDefault})"
+108
View File
@@ -0,0 +1,108 @@
# generated by datamodel-codegen:
# filename: stream_schema.json
# timestamp: 2020-11-17T14:33:13+00:00
from __future__ import annotations
from datetime import datetime
from typing import Any, Dict, List, Optional
from pydantic import BaseModel
class Collaborator(BaseModel):
id: Optional[str]
name: Optional[str]
role: Optional[str]
avatar: Optional[str]
class Commit(BaseModel):
id: Optional[str]
message: Optional[str]
sourceApplication: Optional[str]
totalChildrenCount: Optional[int]
branchName: Optional[str]
parents: Optional[List[str]]
authorName: Optional[str]
authorId: Optional[str]
authorAvatar: Optional[str]
createdAt: Optional[str]
referencedObject: Optional[str]
def __repr__(self) -> str:
return f"Commit( id: {self.id}, message: {self.message}, referencedObject: {self.referencedObject}, authorName: {self.authorName}, createdAt: {self.createdAt} )"
def __str__(self) -> str:
return self.__repr__()
class Commits(BaseModel):
totalCount: Optional[int]
cursor: Optional[Any]
items: List[Commit] = []
class Object(BaseModel):
id: Optional[str]
speckleType: Optional[str]
applicationId: Optional[str]
totalChildrenCount: Optional[int]
createdAt: Optional[str]
class Branch(BaseModel):
id: Optional[str]
name: Optional[str]
description: Optional[str]
commits: Optional[Commits]
class Branches(BaseModel):
totalCount: Optional[int]
cursor: Optional[datetime]
items: List[Branch] = []
class Streams(BaseModel):
totalCount: Optional[int]
cursor: Optional[datetime]
items: List[Stream] = []
class Stream(BaseModel):
id: Optional[str]
name: Optional[str]
description: Optional[str]
isPublic: Optional[bool]
createdAt: Optional[str]
updatedAt: Optional[str]
collaborators: List[Collaborator] = []
branches: Optional[Branches]
commit: Optional[Commit]
object: Optional[Object]
def __repr__(self):
return f"Stream( id: {self.id}, name: {self.name}, description: {self.description}, isPublic: {self.isPublic})"
def __str__(self) -> str:
return self.__repr__()
class User(BaseModel):
id: Optional[str]
email: Optional[str]
name: Optional[str]
bio: Optional[str]
company: Optional[str]
avatar: Optional[str]
verified: Optional[bool]
role: Optional[str]
streams: Optional[Streams]
def __repr__(self):
return f"User( id: {self.id}, name: {self.name}, email: {self.email}, company: {self.company} )"
def __str__(self) -> str:
return self.__repr__()
+93
View File
@@ -0,0 +1,93 @@
from typing import List
from speckle.objects.base import Base
from speckle.transports.memory import MemoryTransport
from speckle.logging.exceptions import SpeckleException
from speckle.transports.abstract_transport import AbstractTransport
from speckle.serialization.base_object_serializer import BaseObjectSerializer
def send(
base: Base,
transports: List[AbstractTransport] = [],
use_default_cache: bool = True,
):
"""Sends an object via the provided transports. Defaults to the local cache.
Arguments:
obj {Base} -- the object you want to send
transports {list} -- where you want to send them
use_default_cache {bool} -- toggle for the default cache. If set to false, it will only send to the provided transports
Returns:
str -- the object id of the sent object
"""
if not transports and not use_default_cache:
raise SpeckleException(
message="You need to provide at least one transport: cannot send with an empty transport list and no default cache"
)
if use_default_cache:
# TODO: finish sqlite transport and chuck it in here
pass
serializer = BaseObjectSerializer(write_transports=transports)
for t in transports:
t.begin_write()
hash, _ = serializer.write_json(base=base)
for t in transports:
t.end_write()
return hash
def receive(
obj_id: str,
remote_transport: AbstractTransport = None,
local_transport: AbstractTransport = None,
) -> Base:
"""Receives an object from a transport.
Arguments:
obj_id {str} -- the id of the object to receive
remote_transport {Transport} -- the transport to receive from
local_transport {Transport} -- the transport to send from
Returns:
Base -- the base object
"""
# TODO: replace with sqlite transport
if not local_transport:
local_transport = MemoryTransport()
serializer = BaseObjectSerializer(read_transport=local_transport)
# try local transport first. if the parent is there, we assume all the children are there and continue wth deserialisation using the local transport
obj_string = local_transport.get_object(obj_id)
if obj_string:
base = serializer.read_json(obj_string=obj_string)
return base
if not remote_transport:
raise SpeckleException(
message="Could not find the specified object using the local transport, and you didn't provide a fallback remote from which to pull it."
)
obj_string = remote_transport.copy_object_and_children(
id=obj_id, target_transport=local_transport
)
return serializer.read_json(obj_string=obj_string)
def serialize(base: Base) -> str:
serializer = BaseObjectSerializer()
return serializer.write_json(base)[1]
def deserialize(obj_string: str) -> Base:
serializer = BaseObjectSerializer()
return serializer.read_json(obj_string=obj_string)
+76
View File
@@ -0,0 +1,76 @@
from logging import error
from speckle.logging.exceptions import GraphQLException, SpeckleException
from typing import Dict, List
from gql.client import Client
from gql.gql import gql
from gql.transport.exceptions import TransportQueryError
class ResourceBase(object):
def __init__(
self,
me: Dict,
basepath: str,
client: Client,
name: str,
methods: list,
) -> None:
self.me = me
self.basepath = basepath
self.client = client
self.name = name
self.methods = methods
self.schema = None
def _step_into_response(self, response: dict, return_type: str or List):
"""Step into the dict to get the relevant data"""
if return_type is None:
return response
elif isinstance(return_type, str):
return response[return_type]
elif isinstance(return_type, List):
for key in return_type:
response = response[key]
return response
def _parse_response(self, response: dict or list, schema=None):
"""Try to create a class instance from the response"""
if isinstance(response, list):
return [self._parse_response(response=r, schema=schema) for r in response]
if schema:
return schema.parse_obj(response)
elif self.schema:
return self.schema.parse_obj(response)
else:
return response
def make_request(
self,
query: gql,
params: Dict = None,
return_type: str or List = None,
schema=None,
parse_response: bool = True,
) -> Dict or GraphQLException:
"""Executes the GraphQL query"""
try:
response = self.client.execute(query, variable_values=params)
except Exception as e:
if isinstance(e, TransportQueryError):
return GraphQLException(
message=f"Failed to execute the GraphQL {self.name} request. Errors: {e.errors}",
errors=e.errors,
data=e.data,
)
else:
return SpeckleException(
message=f"Failed to execute the GraphQL {self.name} request. Inner exception: {e}",
exception=e,
)
response = self._step_into_response(response=response, return_type=return_type)
if parse_response:
return self._parse_response(response=response, schema=schema)
else:
return response
+13
View File
@@ -0,0 +1,13 @@
from pathlib import Path
import sys
import inspect
import pkgutil
from importlib import import_module
for (_, name, _) in pkgutil.iter_modules([Path(__file__).parent]):
imported_module = import_module("." + name, package=__name__)
if hasattr(imported_module, "Resource"):
setattr(sys.modules[__name__], name, imported_module)
+48
View File
@@ -0,0 +1,48 @@
from typing import List, Optional
from gql import gql
from pydantic.main import BaseModel
from speckle.api.resource import ResourceBase
from speckle.api.models import Branch
NAME = "branch"
METHODS = ["create"]
class Resource(ResourceBase):
"""API Access class for branches"""
def __init__(self, me, basepath, client) -> None:
super().__init__(
me=me, basepath=basepath, client=client, name=NAME, methods=METHODS
)
self.schema = Branch
def create(
self, streamId: str, name: str, description: str = "No description provided"
) -> str:
"""Create a new branch on this stream
Arguments:
name {str} -- the name of the new branch
description {str} -- a short description of the branch
Returns:
id {str} -- the newly created branch's id
"""
query = gql(
"""
mutation BranchCreate($branch: BranchCreateInput!){
branchCreate(branch: $branch)
}
"""
)
params = {
"branch": {
"streamId": streamId,
"name": name,
"description": description,
}
}
return self.make_request(query=query, params=params, parse_response=False)
+174
View File
@@ -0,0 +1,174 @@
from typing import Optional, List
from gql import gql
from pydantic.main import BaseModel
from speckle.api.resource import ResourceBase
from speckle.api.models import Commit
NAME = "commit"
METHODS = []
class Resource(ResourceBase):
"""API Access class for commits"""
def __init__(self, me, basepath, client) -> None:
super().__init__(
me=me, basepath=basepath, client=client, name=NAME, methods=METHODS
)
self.schema = Commit
def get(self, stream_id: str, commit_id: str) -> Commit:
"""
Gets a commit given a stream and the commit id
Arguments:
stream_id {str} -- the stream where we can find the commit
commit_id {str} -- the id of the commit you want to get
Returns:
Commit -- the retrieved commit object
"""
query = gql(
"""
query Commit($stream_id: String!, $commit_id: String!) {
stream(id: $stream_id) {
commit(id: $commit_id) {
id
referencedObject
message
authorName
authorId
createdAt
}
}
}
"""
)
params = {"stream_id": stream_id, "commit_id": commit_id}
return self.make_request(
query=query, params=params, return_type=["stream", "commit"]
)
def list(self, stream_id: str, limit: int = 10) -> List[Commit]:
"""
Get a list of commits on a given stream
Arguments:
stream_id {str} -- the stream where the commits are
limit {int} -- the maximum number of commits to fetch (default = 10)
Returns:
List[Commit] -- a list of the most recent commit objects
"""
query = gql(
"""
query Commits($stream_id: String!, $limit: Int!) {
stream(id: $stream_id) {
commits(limit: $limit) {
items {
id
message
authorName
authorId
createdAt
referencedObject
}
}
}
}
"""
)
params = {"stream_id": stream_id, "limit": limit}
return self.make_request(
query=query, params=params, return_type=["stream", "commits", "items"]
)
def create(
self,
stream_id: str,
object_id: str,
branch_name: str = "main",
message: str = "",
source_application: str = "PySpeckle",
) -> str:
"""
Creates a commit on a branch
Arguments:
stream_id {str} -- the stream you want to commit to
object_id {str} -- the hash of your commit object
branch_name {str} -- the name of the branch to commit to (defaults to "main")
message {str} -- optional: a message to give more information about the commit
source_application {str} -- optional: the name of the application creating the commit (defaults to "PySpeckle")
Returns:
str -- the id of the created commit
"""
query = gql(
"""
mutation CommitCreate ($commit: CommitCreateInput!){ commitCreate(commit: $commit)}
"""
)
params = {
"commit": {
"streamId": stream_id,
"branchName": branch_name,
"objectId": object_id,
"message": message,
"sourceApplication": source_application,
}
}
return self.make_request(
query=query, params=params, return_type="commitCreate", parse_response=False
)
def update(self, stream_id: str, commit_id: str, message: str) -> bool:
"""
Update a commit
Arguments:
stream_id {str} -- the id of the stream that contains the commit you'd like to update
commit_id {str} -- the id of the commit you'd like to update
message {str} -- the updated commit message
Returns:
bool -- True if the operation succeeded
"""
query = gql(
"""
mutation CommitUpdate($commit: CommitUpdateInput!){ commitUpdate(commit: $commit)}
"""
)
params = {
"commit": {"streamId": stream_id, "id": commit_id, "message": message}
}
return self.make_request(
query=query, params=params, return_type="commitUpdate", parse_response=False
)
def delete(self, stream_id: str, commit_id: str) -> bool:
"""
Delete a commit
Arguments:
stream_id {str} -- the id of the stream that contains the commit you'd like to delete
commit_id {str} -- the id of the commit you'd like to delete
Returns:
bool -- True if the operation succeeded
"""
query = gql(
"""
mutation CommitDelete($commit: CommitDeleteInput!){ commitDelete(commit: $commit)}
"""
)
params = {"commit": {"streamId": stream_id, "id": commit_id}}
return self.make_request(
query=query, params=params, return_type="commitDelete", parse_response=False
)
+77
View File
@@ -0,0 +1,77 @@
from typing import Dict, List
from gql import gql
from graphql.language import parser
from speckle.api.resource import ResourceBase
from speckle.objects.base import Base
NAME = "object"
METHODS = []
class Resource(ResourceBase):
"""API Access class for objects"""
def __init__(self, me, basepath, client) -> None:
super().__init__(
me=me, basepath=basepath, client=client, name=NAME, methods=METHODS
)
self.schema = Base
def get(self, stream_id: str, object_id: str) -> Base:
"""
Get a stream object
Arguments:
stream_id {str} -- the id of the stream for the object
object_id {str} -- the hash of the object you want to get
Returns:
Base -- the returned Base object
"""
query = gql(
"""
query Object($stream_id: String!, $object_id: String!) {
stream(id: $stream_id) {
id
name
object(id: $object_id) {
id
speckleType
applicationId
createdAt
totalChildrenCount
data
}
}
}
"""
)
params = {"stream_id": stream_id, "object_id": object_id}
return self.make_request(
query=query, params=params, return_type=["stream", "object"]
)
def create(self, stream_id: str, objects: List[Dict]) -> str:
"""
Create a new object on a stream. To send a base object, you can prepare it by running it through the `BaseObjectSerializer.travers_base` function to get a valid (serialisable) object to send.
NOTE: this does not create a commit - you can create one with `SpeckleClient.commit.create`.
Arguments:
stream_id {str} -- the id of the stream you want to send the object to
objects {List[Dict]} -- a list of base dictionary objects (NOTE: must be json serialisable)
Returns:
str -- the id of the object
"""
query = gql(
"""
mutation ObjectCreate($object_input: ObjectCreateInput!) { objectCreate(objectInput: $object_input) }
"""
)
params = {"object_input": {"streamId": stream_id, "objects": objects}}
return self.make_request(
query=query, params=params, return_type="objectCreate", parse_response=False
)
+79
View File
@@ -0,0 +1,79 @@
from typing import Dict
from gql import gql
from gql.client import Client
from speckle.api.resource import ResourceBase
NAME = "server"
METHODS = ["get", "apps"]
class Resource(ResourceBase):
"""API Access class for the server"""
def __init__(self, me, basepath, client) -> None:
super().__init__(
me=me, basepath=basepath, client=client, name=NAME, methods=METHODS
)
def get(self) -> Dict:
"""Get the server info
Returns:
dict -- the server info in dictionary form
"""
query = gql(
"""
query Server {
serverInfo {
name
company
description
adminContact
canonicalUrl
roles {
name
description
resourceTarget
}
scopes {
name
description
}
authStrategies{
id
name
icon
}
}
}
"""
)
return self.make_request(query=query)
def apps(self) -> Dict:
"""Get the apps registered on the server
Returns:
dict -- a dictionary of apps registered on the server
"""
query = gql(
"""
query Apps {
apps {
id
name
description
termsAndConditionsLink
logo
author {
id
name
}
}
}
"""
)
return self.make_request(query=query)
+298
View File
@@ -0,0 +1,298 @@
from re import search
from typing import Dict, List, Optional
from pydantic import BaseModel
from gql import gql
from speckle.api.resource import ResourceBase
from speckle.api.models import Stream
from speckle.logging.exceptions import GraphQLException
NAME = "stream"
METHODS = [
"list",
"create",
"get",
"update",
"delete",
"search",
]
class Resource(ResourceBase):
"""API Access class for streams"""
def __init__(self, me, basepath, client) -> None:
super().__init__(
me=me, basepath=basepath, client=client, name=NAME, methods=METHODS
)
self.schema = Stream
def get(self, id: str, branch_limit: int = 10, commit_limit: int = 10) -> Stream:
"""Get the specified stream from the server
Arguments:
id {str} -- the stream id
branch_limit {int} -- the maximum number of branches to return
commit_limit {int} -- the maximum number of commits to return
Returns:
Stream -- the retrieved stream
"""
query = gql(
"""
query Stream($id: String!, $branch_limit: Int!, $commit_limit: Int!) {
stream(id: $id) {
id
name
description
isPublic
createdAt
updatedAt
collaborators {
id
name
role
avatar
}
branches(limit: $branch_limit) {
totalCount
cursor
items {
id
name
description
commits(limit: $commit_limit) {
totalCount
cursor
items {
id
referencedObject
message
authorName
authorId
createdAt
}
}
}
}
}
}
"""
)
params = {"id": id, "branch_limit": branch_limit, "commit_limit": commit_limit}
return self.make_request(query=query, params=params, return_type="stream")
def list(self, stream_limit: int = 10) -> List[Stream]:
"""Get a list of the user's streams
Arguments:
stream_limit {int} -- The maximum number of streams to return
Returns:
List[Stream] -- A list of Stream objects
"""
query = gql(
"""
query User($stream_limit: Int!) {
user {
id
email
name
bio
company
avatar
verified
profiles
role
streams(limit: $stream_limit) {
totalCount
cursor
items {
id
name
description
isPublic
createdAt
updatedAt
collaborators {
id
name
role
}
}
}
}
}
"""
)
params = {"stream_limit": stream_limit}
return self.make_request(
query=query, params=params, return_type=["user", "streams", "items"]
)
def create(
self,
name: str = "Anonymous Python Stream",
description: str = "No description provided",
is_public: bool = True,
) -> str:
"""Create a new stream
Arguments:
name {str} -- the name of the string
description {str} -- a short description of the stream
is_public {bool} -- whether or not the stream can be viewed by anyone with the id
Returns:
id {str} -- the id of the newly created stream
"""
query = gql(
"""
mutation StreamCreate($stream: StreamCreateInput!) {
streamCreate(stream: $stream)
}
"""
)
params = {
"stream": {"name": name, "description": description, "isPublic": is_public}
}
return self.make_request(
query=query, params=params, return_type="streamCreate", parse_response=False
)
def update(
self, id: str, name: str = None, description: str = None, is_public: bool = None
) -> bool:
"""Update an existing stream
Arguments:
id {str} -- the id of the stream to be updated
name {str} -- the name of the string
description {str} -- a short description of the stream
is_public {bool} -- whether or not the stream can be viewed by anyone with the id
Returns:
bool -- whether the stream update was successful
"""
query = gql(
"""
mutation StreamUpdate($stream: StreamUpdateInput!) {
streamUpdate(stream: $stream)
}
"""
)
params = {
"id": id,
"name": name,
"description": description,
"isPublic": is_public,
}
# remove None values so graphql doesn't cry
params = {"stream": {k: v for k, v in params.items() if v is not None}}
return self.make_request(
query=query, params=params, return_type="streamUpdate", parse_response=False
)
def delete(self, id: str) -> bool:
"""Delete a stream given its id
Arguments:
id {str} -- the id of the stream to delete
Returns:
bool -- whether the deletion was successful
"""
query = gql(
"""
mutation StreamDelete($id: String!) {
streamDelete(id: $id)
}
"""
)
params = {"id": id}
return self.make_request(
query=query, params=params, return_type="streamDelete", parse_response=False
)
def search(
self,
search_query: str,
limit: int = 25,
branch_limit: int = 10,
commit_limit: int = 10,
):
"""Search for streams by name, description, or id
Arguments:
search_query {str} -- a string to search for
limit {int} -- the maximum number of results to return
branch_limit {int} -- the maximum number of branches to return
commit_limit {int} -- the maximum number of commits to return
Returns:
List[Stream] -- a list of Streams that match the search query
"""
query = gql(
"""
query StreamSearch($search_query: String!,$limit: Int!, $branch_limit:Int!, $commit_limit:Int!) {
streams(query: $search_query, limit: $limit) {
items {
id
name
description
isPublic
createdAt
updatedAt
collaborators {
id
name
role
avatar
}
branches(limit: $branch_limit) {
totalCount
cursor
items {
id
name
description
commits(limit: $commit_limit) {
totalCount
cursor
items {
id
referencedObject
message
authorName
authorId
createdAt
}
}
}
}
}
}
}
"""
)
params = {
"search_query": search_query,
"limit": limit,
"branch_limit": branch_limit,
"commit_limit": commit_limit,
}
return self.make_request(
query=query, params=params, return_type=["streams", "items"]
)
+125
View File
@@ -0,0 +1,125 @@
from typing import Callable, Dict, List, Optional, Any
from functools import wraps
from gql import gql
from speckle.api.resource import ResourceBase
from speckle.api.resources.stream import Stream
from speckle.logging.exceptions import GraphQLException, SpeckleException
NAME = "subscribe"
METHODS = [
"stream_added",
"stream_updated",
"stream_removed",
]
def check_wsclient(function):
@wraps(function)
async def check_wsclient_wrapper(self, *args, **kwargs):
if self.client is None:
raise SpeckleException(
"You must authenticate before you can subscribe to events"
)
else:
return await function(self, *args, **kwargs)
return check_wsclient_wrapper
class Resource(ResourceBase):
"""API Access class for subscriptions"""
def __init__(self, me, basepath, client) -> None:
super().__init__(
me=me, basepath=basepath, client=client, name=NAME, methods=METHODS
)
@check_wsclient
async def stream_added(self, callback: Callable = None):
"""Subscribes to new stream added event for your profile. Use this to display an up-to-date list of streams.
Arguments:
callback {Callable[Stream]} -- a function that takes the updated stream as an argument and executes each time a stream is added
Returns:
Stream -- the update stream
"""
query = gql(
"""
subscription { userStreamAdded }
"""
)
return await self.subscribe(
query=query, callback=callback, return_type="userStreamAdded", schema=Stream
)
@check_wsclient
async def stream_updated(self, id: str, callback: Callable = None):
"""Subscribes to stream updated event. Use this in clients/components that pertain only to this stream.
Arguments:
id {str} -- the stream id of the stream to subscribe to
callback {Callable[Stream]} -- a function that takes the updated stream as an argument and executes each time the stream is updated
Returns:
Stream -- the update stream
"""
query = gql(
"""
subscription Update($id: String!) { streamUpdated(streamId: $id) }
"""
)
params = {"id": id}
return await self.subscribe(
query=query,
params=params,
callback=callback,
return_type="streamUpdated",
schema=Stream,
)
@check_wsclient
async def stream_removed(self, callback: Callable = None):
"""Subscribes to stream removed event for your profile. Use this to display an up-to-date list of streams for your profile. NOTE: If someone revokes your permissions on a stream, this subscription will be triggered with an extra value of revokedBy in the payload.
Arguments:
callback {Callable[Dict]} -- a function that takes the returned dict as an argument and executes each time a stream is removed
Returns:
dict -- dict containing 'id' of stream removed and optionally 'revokedBy'
"""
query = gql(
"""
subscription { userStreamRemoved }
"""
)
return await self.subscribe(
query=query,
callback=callback,
return_type="userStreamRemoved",
parse_response=False,
)
@check_wsclient
async def subscribe(
self,
query: gql,
params: Dict = None,
callback: Callable = None,
return_type: str or List = None,
schema=None,
parse_response: bool = True,
):
# if self.client.transport.websocket is None:
# TODO: add multiple subs to the same ws connection
async with self.client as session:
async for res in session.subscribe(query, variable_values=params):
res = self._step_into_response(response=res, return_type=return_type)
if parse_response:
res = self._parse_response(response=res, schema=schema)
if callback is not None:
callback(res)
else:
return res
+86
View File
@@ -0,0 +1,86 @@
from speckle.logging.exceptions import SpeckleException
from typing import List, Optional
from gql import gql
from pydantic.main import BaseModel
from speckle.api.resource import ResourceBase
from speckle.api.models import User
NAME = "user"
METHODS = ["get"]
class Resource(ResourceBase):
"""API Access class for users"""
def __init__(self, me, basepath, client) -> None:
super().__init__(
me=me, basepath=basepath, client=client, name=NAME, methods=METHODS
)
self.schema = User
def get(self, id: str = None) -> User:
"""Gets the profile of a user. If no id argument is provided, will return the current authenticated user's profile (as extracted from the authorization header).
Arguments:
id {str} -- the user id
Returns:
User -- the retrieved user
"""
query = gql(
"""
query User($id: String) {
user(id: $id) {
id
email
name
bio
company
avatar
verified
profiles
role
}
}
"""
)
params = {"id": id}
return self.make_request(query=query, params=params, return_type="user")
def search(self, search_query: str, limit: int = 25) -> List[User]:
"""Searches for user by name or email. The search query must be at least 3 characters long
Arguments:
search_query {str} -- a string to search for
limit {int} -- the maximum number of results to return
Returns:
List[User] -- a list of User objects that match the search query
"""
if len(search_query) < 3:
return SpeckleException(
message="User search query must be at least 3 characters"
)
query = gql(
"""
query UserSearch($search_query: String!, $limit: Int!) {
userSearch(query: $search_query, limit: $limit) {
items {
id
name
bio
company
avatar
verified
}
}
}
"""
)
params = {"search_query": search_query, "limit": limit}
return self.make_request(
query=query, params=params, return_type=["userSearch", "items"]
)
+640
View File
@@ -0,0 +1,640 @@
scalar DateTime
scalar EmailAddress
scalar BigInt
scalar JSONObject
directive @hasScope(scope: String!) on FIELD_DEFINITION
directive @hasRole(role: String!) on FIELD_DEFINITION
type Query {
"""
Stare into the void.
"""
_: String
}
type Mutation{
"""
The void stares back.
"""
_: String
}
type Subscription{
"""
It's lonely in the void.
"""
_: String
},extend type Query {
"""
Gets a specific app from the server.
"""
app( id: String! ): ServerApp
"""
Returns all the publicly available apps on this server.
"""
apps: [ServerAppListItem]
}
type ServerApp {
id: String!
secret: String!
name: String!
description: String
termsAndConditionsLink: String
logo: String
public: Boolean
trustByDefault: Boolean
author: AppAuthor
createdAt: DateTime!
redirectUrl: String!
scopes: [Scope]!
}
type ServerAppListItem {
id: String!
name: String!
description: String
termsAndConditionsLink: String
logo: String
author: AppAuthor
}
type AppAuthor {
name: String
id: String
}
extend type User {
"""
Returns the apps you have authorized.
"""
authorizedApps: [ServerAppListItem]
@hasRole(role: "server:user")
@hasScope(scope: "apps:read")
"""
Returns the apps you have created.
"""
createdApps: [ServerAppListItem]
@hasRole(role: "server:user")
@hasScope(scope: "apps:read")
}
extend type Mutation {
"""
Register a new third party application.
"""
appCreate(app: AppCreateInput!): String!
@hasRole(role: "server:user")
@hasScope(scope: "apps:write")
"""
Update an existing third party application. **Note: This will invalidate all existing tokens, refresh tokens and access codes and will require existing users to re-authorize it.**
"""
appUpdate(app: AppUpdateInput!): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "apps:write")
"""
Deletes a thirty party application.
"""
appDelete(appId: String!): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "apps:write")
"""
Revokes (de-authorizes) an application that you have previously authorized.
"""
appRevokeAccess(appId: String!): Boolean
@hasRole(role: "server:user")
@hasScope(scope: "apps:write")
}
input AppCreateInput {
name: String!
description: String!
termsAndConditionsLink: String
logo: String
public: Boolean
redirectUrl: String!
scopes: [String]!
}
input AppUpdateInput {
id: String!
name: String!
description: String!
termsAndConditionsLink: String
logo: String
public: Boolean
redirectUrl: String!
scopes: [String]!
}
,extend type ServerInfo {
"""
The authentication strategies available on this server.
"""
authStrategies: [AuthStrategy]
}
type AuthStrategy {
id: String!,
name: String!,
icon: String!,
url: String!,
color: String
}
,extend type User{
"""
Returns a list of your personal api tokens.
"""
apiTokens: [ApiToken]
@hasRole(role: "server:user")
@hasScope(scope: "tokens:read")
}
type ApiToken {
id: String!
name: String!
lastChars: String!
scopes: [String]!
createdAt: DateTime! #date
lifespan: BigInt!
lastUsed: String! #date
}
input ApiTokenCreateInput {
scopes: [String!]!,
name: String!,
lifespan: BigInt
}
extend type Mutation {
"""
Creates an personal api token.
"""
apiTokenCreate(token: ApiTokenCreateInput!):String!
@hasRole(role: "server:user")
@hasScope(scope: "tokens:write")
"""
Revokes (deletes) an personal api token.
"""
apiTokenRevoke(token: String!):Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "tokens:write")
}
,extend type Stream {
commits(limit: Int! = 25, cursor: String): CommitCollection
commit(id: String!): Commit
branches(limit: Int! = 25, cursor: String): BranchCollection
branch(name: String!): Branch
}
extend type User {
commits(limit: Int! = 25, cursor: String): CommitCollectionUser
}
type Branch {
id: String!
name: String!
author: User!
description: String
commits(limit: Int! = 25, cursor: String): CommitCollection
}
type Commit {
id: String!
referencedObject: String!
message: String
authorName: String
authorId: String
createdAt: DateTime
}
type CommitCollectionUserNode {
id: String!
referencedObject: String!
message: String
streamId: String
streamName: String
createdAt: DateTime
}
type BranchCollection {
totalCount: Int!
cursor: String
items: [Branch]
}
type CommitCollection {
totalCount: Int!
cursor: String
items: [Commit]
}
type CommitCollectionUser {
totalCount: Int!
cursor: String
items: [CommitCollectionUserNode]
}
extend type Mutation {
branchCreate(branch: BranchCreateInput!): String!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
branchUpdate(branch: BranchUpdateInput!): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
branchDelete(branch: BranchDeleteInput!): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
commitCreate(commit: CommitCreateInput!): String!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
commitUpdate(commit: CommitUpdateInput!): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
commitDelete(commit: CommitDeleteInput!): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
}
extend type Subscription {
# TODO: auth for these subscriptions
"""
Subscribe to branch created event
"""
branchCreated(streamId: String!): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
"""
Subscribe to branch updated event.
"""
branchUpdated(streamId: String!, branchId: String): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
"""
Subscribe to branch deleted event
"""
branchDeleted(streamId: String!): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
"""
Subscribe to commit created event
"""
commitCreated(streamId: String!): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
"""
Subscribe to commit updated event.
"""
commitUpdated(streamId: String!, commitId: String): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
"""
Subscribe to commit deleted event
"""
commitDeleted(streamId: String!): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
}
input BranchCreateInput {
streamId: String!
name: String!
description: String
}
input BranchUpdateInput {
streamId: String!
id: String!
name: String
description: String
}
input BranchDeleteInput {
streamId: String!
id: String!
}
input CommitCreateInput {
streamId: String!
branchName: String!
objectId: String!
message: String
previousCommitIds: [String]
}
input CommitUpdateInput {
streamId: String!
id: String!
message: String!
}
input CommitDeleteInput {
streamId: String!
id: String!
}
,extend type Stream {
object( id: String! ): Object
}
type Object {
id: String!
speckleType: String!
applicationId: String
createdAt: DateTime
totalChildrenCount: Int
"""
The full object, with all its props & other things. **NOTE:** If you're requesting objects for the purpose of recreating & displaying, you probably only want to request this specific field.
"""
data: JSONObject
"""
Get any objects that this object references. In the case of commits, this will give you a commit's constituent objects.
**NOTE**: Providing any of the two last arguments ( `query`, `orderBy` ) will trigger a different code branch that executes a much more expensive SQL query. It is not recommended to do so for basic clients that are interested in purely getting all the objects of a given commit.
"""
children(
limit: Int! = 100,
depth: Int! = 50,
select: [String],
cursor: String,
query: [JSONObject!],
orderBy: JSONObject ): ObjectCollection!
}
type ObjectCollection {
totalCount: Int!
cursor: String
objects: [Object]!
}
extend type Mutation {
objectCreate( objectInput: ObjectCreateInput! ): [String]!
}
input ObjectCreateInput {
"""
The stream against which these objects will be created.
"""
streamId: String!
"""
The objects you want to create.
"""
objects: [JSONObject]!
},extend type Query {
serverInfo: ServerInfo!
}
"""
Information about this server.
"""
type ServerInfo {
name: String!
company: String
description: String
adminContact: String
canonicalUrl: String
termsOfService: String
roles: [Role]!
scopes: [Scope]!
}
"""
Available roles.
"""
type Role {
name: String!
description: String!
resourceTarget: String!
}
"""
Available scopes.
"""
type Scope {
name: String!
description: String!
}
extend type Mutation {
serverInfoUpdate(info: ServerInfoUpdateInput!): Boolean
@hasRole(role: "server:admin")
@hasScope(scope: "server:setup")
}
input ServerInfoUpdateInput {
name: String!
company: String
description: String
adminContact: String
termsOfService: String
}
,extend type Query {
"""
Returns a specific stream.
"""
stream( id: String! ): Stream
"""
All the streams of the current user, pass in the `query` parameter to seach by name, description or ID.
"""
streams( query: String, limit: Int = 25, cursor: String ): StreamCollection
@hasScope(scope: "streams:read")
}
type Stream {
id: String!
name: String!
description: String
isPublic: Boolean!
createdAt: DateTime!
updatedAt: DateTime!
collaborators: [ StreamCollaborator ]!
}
extend type User {
"""
All the streams that a user has access to.
"""
streams( limit: Int! = 25, cursor: String ): StreamCollection
}
type StreamCollaborator {
id: String!
name: String!
role: String!
company: String
avatar: String
}
type StreamCollection {
totalCount: Int!
cursor: String
items: [ Stream ]
}
extend type Mutation {
"""
Creates a new stream.
"""
streamCreate( stream: StreamCreateInput! ): String
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
"""
Updates an existing stream.
"""
streamUpdate( stream: StreamUpdateInput! ): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
"""
Deletes an existing stream.
"""
streamDelete( id: String! ): Boolean!
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
"""
Grants permissions to a user on a given stream.
"""
streamGrantPermission( permissionParams: StreamGrantPermissionInput! ): Boolean
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
"""
Revokes the permissions of a user on a given stream.
"""
streamRevokePermission( permissionParams: StreamRevokePermissionInput! ): Boolean
@hasRole(role: "server:user")
@hasScope(scope: "streams:write")
}
extend type Subscription {
#
# User bound subscriptions that operate on the stream collection of an user
# Example relevant view/usecase: updating the list of streams for a user.
#
"""
Subscribes to new stream added event for your profile. Use this to display an up-to-date list of streams.
**NOTE**: If someone shares a stream with you, this subscription will be triggered with an extra value of `sharedBy` in the payload.
"""
userStreamAdded: JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "profile:read")
"""
Subscribes to stream removed event for your profile. Use this to display an up-to-date list of streams for your profile.
**NOTE**: If someone revokes your permissions on a stream, this subscription will be triggered with an extra value of `revokedBy` in the payload.
"""
userStreamRemoved: JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "profile:read")
#
# Stream bound subscriptions that operate on the stream itself.
# Example relevant view/usecase: a single stream connector, or view, or component in a web app
#
"""
Subscribes to stream updated event. Use this in clients/components that pertain only to this stream.
"""
streamUpdated( streamId: String ): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
"""
Subscribes to stream deleted event. Use this in clients/components that pertain only to this stream.
"""
streamDeleted( streamId: String ): JSONObject
@hasRole(role: "server:user")
@hasScope(scope: "streams:read")
}
input StreamCreateInput {
name: String
description: String
isPublic: Boolean
}
input StreamUpdateInput {
id: String!
name: String
description: String
isPublic: Boolean
}
input StreamGrantPermissionInput {
streamId: String!,
userId: String!,
role: String!
}
input StreamRevokePermissionInput {
streamId: String!,
userId: String!
}
,extend type Query {
"""
Gets the profile of a user. If no id argument is provided, will return the current authenticated user's profile (as extracted from the authorization header).
"""
user(id: String): User
userSearch(
query: String!
limit: Int! = 25
cursor: String
): UserSearchResultCollection
userPwdStrength(pwd: String!): JSONObject
}
"""
Base user type.
"""
type User {
id: String!
suuid: String
email: String
name: String
bio: String
company: String
avatar: String
verified: Boolean
profiles: JSONObject
role: String
}
type UserSearchResultCollection {
cursor: String
items: [UserSearchResult]
}
type UserSearchResult {
id: String!
name: String
bio: String
company: String
avatar: String
verified: Boolean
}
extend type Mutation {
"""
Edits a user's profile.
"""
userUpdate(user: UserUpdateInput!): Boolean!
}
input UserUpdateInput {
name: String
company: String
bio: String
}
+30
View File
@@ -0,0 +1,30 @@
from typing import Any, List
class SpeckleException(Exception):
def __init__(self, message: str, exception: Exception = None) -> None:
self.message = message
self.exception = exception
def __str__(self) -> str:
return f"SpeckleException: {self.message}"
class SerializationException(SpeckleException):
def __init__(self, message: str, object: Any, exception: Exception = None) -> None:
super().__init__(message=message)
self.object = object
self.unhandled_type = type(object)
def __str__(self) -> str:
return f"SpeckleException: Could not serialize object of type {self.unhandled_type}"
class GraphQLException(SpeckleException):
def __init__(self, message: str, errors: List, data=None) -> None:
super().__init__(message=message)
self.errors = errors
self.data = data
def __str__(self) -> str:
return f"GraphQLException: {self.message}"
+14
View File
@@ -0,0 +1,14 @@
from pathlib import Path
import sys
import inspect
import pkgutil
from importlib import import_module
from .base import Base
for (_, name, _) in pkgutil.iter_modules([Path(__file__).parent]):
imported_module = import_module("." + name, package=__name__)
classes = inspect.getmembers(imported_module, inspect.isclass)
for c in classes:
if issubclass(c[1], Base):
setattr(sys.modules[__name__], c[0], c[1])
+140
View File
@@ -0,0 +1,140 @@
from __future__ import annotations
from pydantic import BaseModel
from pydantic.main import Extra
from typing import Dict, List, Optional, Any
from speckle.transports.memory import MemoryTransport
from speckle.logging.exceptions import SpeckleException
PRIMITIVES = (int, float, str, bool)
class Base(BaseModel):
id: Optional[str] = None
totalChildrenCount: Optional[int] = None
applicationId: Optional[str] = None
speckle_type: Optional[str] = "Base"
_chunkable: Dict[str, int] = {} # dict of chunkable props and their max chunk size
def __init__(self, **kwargs) -> None:
super().__init__()
self.speckle_type = self.__class__.__name__
self.__dict__.update(kwargs)
def __repr__(self) -> str:
return f"{self.__class__.__name__}(id: {self.id}, speckle_type: {self.speckle_type}, totalChildrenCount: {self.totalChildrenCount})"
def __str__(self) -> str:
return self.__repr__()
def __setitem__(self, name: str, value: Any) -> None:
self.__dict__[name] = value
def __getitem__(self, name: str) -> Any:
return self.__dict__[name]
def to_dict(self) -> Dict:
"""Convenience method to view the whole base object as a dict"""
base_dict = self.__dict__
for key, value in base_dict.items():
if not value or isinstance(value, PRIMITIVES):
continue
else:
base_dict[key] = self.__dict_helper(value)
return base_dict
def __dict_helper(self, obj: Any) -> Any:
if isinstance(obj, PRIMITIVES):
return obj
if isinstance(obj, Base):
return self.__dict_helper(obj.__dict__)
if isinstance(obj, (list, set)):
return [self.__dict_helper(v) for v in obj]
if isinstance(obj, dict):
for k, v in obj.items():
if not v or isinstance(obj, PRIMITIVES):
pass
else:
obj[k] = self.__dict_helper(v)
return obj
else:
raise SpeckleException(
message=f"Could not convert to dict due to unrecognised type: {type(obj)}"
)
def get_member_names(self) -> List[str]:
"""Get all of the property names on this object, dynamic or not"""
return list(self.__dict__.keys())
def get_typed_member_names(self) -> List[str]:
"""Get all of the names of the defined (typed) properties of this object"""
return list(self.__fields__.keys())
def get_dynamic_member_names(self) -> List[str]:
"""Get all of the names of the dynamic properties of this object"""
return list(set(self.__dict__.keys()) - set(self.__fields__.keys()))
def get_children_count(self) -> int:
"""Get the total count of children Base objects"""
parsed = []
return 1 + self._count_descendants(self, parsed)
def get_id(self, decompose: bool = False) -> str:
if self.id and not decompose:
return self.id
else:
from speckle.serialization.base_object_serializer import (
BaseObjectSerializer,
)
serializer = BaseObjectSerializer()
if decompose:
serializer.write_transports = [MemoryTransport()]
return serializer.traverse_base(self)[0]
def _count_descendants(self, base: Base, parsed: List) -> int:
if base in parsed:
return 0
parsed.append(base)
count = 0
for name, value in base.__dict__.items():
if name.startswith("@"):
continue
else:
count += self._handle_object_count(value, parsed)
return count
def _handle_object_count(self, obj: Any, parsed: List) -> int:
count = 0
if obj == None:
return count
if isinstance(obj, Base):
count += 1
count += self._count_descendants(obj, parsed)
return count
elif isinstance(obj, list):
for item in obj:
if isinstance(item, Base):
count += 1
count += self._count_descendants(item, parsed)
else:
count += self._handle_object_count(item, parsed)
elif isinstance(obj, dict):
for _, value in obj.items():
if isinstance(value, Base):
count += 1
count += self._count_descendants(value, parsed)
else:
count += self._handle_object_count(value, parsed)
return count
class Config:
extra = Extra.allow
class DataChunk(Base):
data: List[Any] = []
+24
View File
@@ -0,0 +1,24 @@
from __future__ import annotations
from typing import List, Optional
from pydantic import BaseModel
from .base import Base
CHUNKABLE_PROPS = {
"vertices": 2000,
"faces": 2000,
"colors": 2000,
"textureCoordinates": 2000,
}
class Mesh(Base):
vertices: List[float] = None
faces: List[int] = None
colors: List[int] = None
textureCoordinates: List[float] = None
def __init__(self, **kwargs) -> None:
super().__init__(**kwargs)
self._chunkable.update(CHUNKABLE_PROPS)
@@ -0,0 +1,309 @@
import json
import hashlib
from speckle import objects
from uuid import uuid4
from typing import Any, Dict, List, Tuple
from speckle.objects.base import Base, DataChunk
from speckle.logging.exceptions import SerializationException, SpeckleException
from speckle.transports.abstract_transport import AbstractTransport
PRIMITIVES = (int, float, str, bool)
def hash_obj(obj: Any) -> str:
return hashlib.sha256(json.dumps(obj).encode()).hexdigest()[:32]
class BaseObjectSerializer:
read_transport: AbstractTransport
write_transports: List[AbstractTransport]
detach_lineage: List[bool] = [] # tracks depth and whether or not to detach
lineage: List[str] = [] # keeps track of hash chain through the object tree
family_tree: Dict[str, Dict[str, int]] = {}
closure_table: Dict[str, Dict[str, int]] = {}
def __init__(
self, write_transports: List[AbstractTransport] = [], read_transport=None
) -> None:
self.write_transports = write_transports
self.read_transport = read_transport
def write_json(self, base: Base):
self.__reset_writer()
self.detach_lineage = [True]
hash, obj = self.traverse_base(base)
return hash, json.dumps(obj)
def traverse_base(self, base: Base) -> Tuple[str, Dict]:
"""Decomposes the given base object and builds a serializable dictionary
Arguments:
base {Base} -- the base object to be decomposed and serialized
Returns:
(str, dict) -- a tuple containing the hash (id) of the base object and the constructed serializable dictionary
"""
if not self.detach_lineage:
self.detach_lineage = [True]
self.lineage.append(uuid4().hex)
object_builder = {"id": ""}
obj, props = base, base.get_member_names()
while props:
prop = props.pop(0)
value = obj[prop]
# skip nulls or props marked to be ignored with "__" or "_"
if not value or prop.startswith(("__", "_")):
continue
# don't prepopulate id as this will mess up hashing
if prop == "id":
continue
chunkable = True if prop in base._chunkable else False
detach = True if prop.startswith("@") or chunkable else False
# 1. handle primitives (ints, floats, strings, and bools)
if isinstance(value, PRIMITIVES):
object_builder[prop] = value
continue
# 2. handle Base objects
elif isinstance(value, Base):
child_obj = self.traverse_value(value, detach=detach)
if detach and self.write_transports:
ref_hash = child_obj["id"]
object_builder[prop] = self.detach_helper(ref_hash=ref_hash)
else:
object_builder[prop] = child_obj
# 3. handle chunkable props
elif chunkable and self.write_transports:
chunks = []
max_size = base._chunkable[prop]
chunk = DataChunk()
for count, item in enumerate(value):
if count and count % max_size == 0:
chunks.append(chunk)
chunk = DataChunk()
chunk.data.append(item)
chunks.append(chunk)
chunk_refs = []
for c in chunks:
self.detach_lineage.append(detach)
ref_hash, _ = self.traverse_base(c)
ref_obj = self.detach_helper(ref_hash=ref_hash)
chunk_refs.append(ref_obj)
object_builder[prop] = chunk_refs
# 4. handle all other cases
else:
child_obj = self.traverse_value(value)
object_builder[prop] = child_obj
hash = hash_obj(object_builder)
object_builder["id"] = hash
detached = self.detach_lineage.pop()
# add closures to the object
if self.lineage[-1] in self.family_tree:
object_builder["__closure"] = self.closure_table[hash] = {
ref: depth - len(self.detach_lineage)
for ref, depth in self.family_tree[self.lineage[-1]].items()
}
# write detached or root objects to transports
if detached and self.write_transports:
for t in self.write_transports:
t.save_object(id=hash, serialized_object=json.dumps(object_builder))
del self.lineage[-1]
return hash, object_builder
def traverse_value(self, obj: Any, detach: bool = False) -> Any:
"""Decomposes a given object and constructs a serializable object or dictionary
Arguments:
obj {Any} -- the value to decompose
Returns:
Any -- a serializable version of the given object
"""
if isinstance(obj, PRIMITIVES):
return obj
elif isinstance(obj, (list, tuple, set)):
return [self.traverse_value(o) for o in obj]
elif isinstance(obj, dict):
for k, v in obj.items():
if isinstance(v, PRIMITIVES):
continue
else:
obj[k] = self.traverse_value(v)
return obj
elif isinstance(obj, Base):
self.detach_lineage.append(detach)
_, base_obj = self.traverse_base(obj)
return base_obj
else:
try:
return obj.dict()
except:
SerializationException(
message=f"Failed to handle {type(obj)} in `BaseObjectSerializer.traverse_value`",
object=obj,
)
return str(obj)
def detach_helper(self, ref_hash: str) -> Dict[str, str]:
"""Helper to keep track of detached objects and their depth in the family tree and create reference objects to place in the parent object
Arguments:
ref_hash {str} -- the hash of the fully traversed object
Returns:
dict -- a reference object to be inserted into the given object's parent
"""
for parent in self.lineage:
if parent not in self.family_tree:
self.family_tree[parent] = {}
if ref_hash not in self.family_tree[parent] or self.family_tree[parent][
ref_hash
] > len(self.detach_lineage):
self.family_tree[parent][ref_hash] = len(self.detach_lineage)
return {
"referencedId": ref_hash,
"speckle_type": "reference",
}
def __reset_writer(self) -> None:
"""Reinitializes the lineage, and other variables that get used during the json writing process"""
self.detach_lineage = []
self.lineage = []
self.family_tree = {}
self.closure_table = {}
def read_json(self, obj_string: str) -> Base:
"""Recomposes a Base object from the string representation of the object
Arguments:
obj_string {str} -- the string representation of the object
Returns:
Base -- the base object with all it's children attached
"""
if not obj_string:
return None
obj = json.loads(obj_string)
base = self.recompose_base(obj=obj)
return base
def recompose_base(self, obj: dict) -> Base:
"""Steps through a base object dictionary and recomposes the base object
Arguments:
obj {dict} -- the dictionary representation of the object
Returns:
Base -- the base object with all its children attached
"""
# make sure an obj was passed and create dict if string was somehow passed
if not obj:
return
if isinstance(obj, str):
obj = json.loads(obj)
if obj["speckle_type"] == "reference":
obj = self.get_child(obj=obj)
# initialise the base object using `speckle_type`
base = getattr(objects, obj["speckle_type"], Base)()
# get total children count
if "__closure" in obj:
if not self.read_transport:
raise SpeckleException(
message="Cannot resolve reference - no read transport is defined"
)
closure = obj.pop("__closure")
base.totalChildrenCount = len(closure)
for prop, value in obj.items():
# 1. handle primitives (ints, floats, strings, and bools)
if isinstance(value, PRIMITIVES):
base[prop] = value
continue
# 2. handle referenced child objects
elif "referencedId" in value:
ref_hash = value["referencedId"]
ref_obj_str = self.read_transport.get_object(id=ref_hash)
if not ref_obj_str:
raise SpeckleException(
f"Could not find the referenced child object of id `{ref_hash}` in the given read transport: {self.read_transport.name}"
)
ref_obj = json.loads(ref_obj_str)
base[prop] = self.recompose_base(obj=ref_obj)
# 3. handle all other cases (base objects, lists, and dicts)
else:
base[prop] = self.handle_value(value)
return base
def handle_value(self, obj: Any):
"""Helper for recomposing a base object by handling the dictionary representation's values
Arguments:
obj {Any} -- a value from the base object dictionary
Returns:
Any -- the handled value (primitive, list, dictionary, or Base)
"""
if isinstance(obj, PRIMITIVES):
return obj
# lists (regular and chunked)
if isinstance(obj, list):
obj_list = [self.handle_value(o) for o in obj]
# handle chunked lists
if isinstance(obj_list[0], DataChunk):
data = []
for o in obj_list:
data.extend(o["data"])
return data
else:
return obj_list
# bases
if isinstance(obj, dict) and "speckle_type" in obj:
return self.recompose_base(obj=obj)
# dictionaries
if isinstance(obj, dict):
for k, v in obj.items():
if isinstance(v, PRIMITIVES):
continue
else:
obj[k] = self.handle_value(v)
return obj
def get_child(self, obj: Dict):
ref_hash = obj["referencedId"]
ref_obj_str = self.read_transport.get_object(id=ref_hash)
if not ref_obj_str:
raise SpeckleException(
f"Could not find the referenced child object of id `{ref_hash}` in the given read transport: {self.read_transport.name}"
)
return json.loads(ref_obj_str)
@@ -1,12 +1,24 @@
from abc import ABC, abstractmethod
from typing import Dict, List, Optional
from typing import Any, Optional
from pydantic import BaseModel
from pydantic.main import Extra
# __________________
# | |
# | this is v wip |
# | pls be careful |
# |__________________|
# (\__/) ||
# (•ㅅ•) ||
# /  
class AbstractTransport(ABC):
class AbstractTransport(ABC, BaseModel):
_name: str = "Abstract"
@property
@abstractmethod
def name(self):
pass
return type(self)._name
@abstractmethod
def begin_write(self) -> None:
@@ -15,9 +27,7 @@ class AbstractTransport(ABC):
@abstractmethod
def end_write(self) -> None:
"""
Optional: signals to the transport that no more items will need to be written.
"""
"""Optional: signals to the transport that no more items will need to be written."""
pass
@abstractmethod
@@ -38,8 +48,7 @@ class AbstractTransport(ABC):
Arguments:
id {str} -- the hash of the object
source_transport {AbstractTransport)
-- the transport through which the object can be found
source_transport {AbstractTransport) -- the transport through which the object can be found
"""
pass
@@ -51,21 +60,7 @@ class AbstractTransport(ABC):
id {str} -- the hash of the object
Returns:
str -- the full string representation
of the object (or null if no object is found)
"""
pass
@abstractmethod
def has_objects(self, id_list: List[str]) -> Dict[str, bool]:
"""Checks the presence of multiple objects.
Arguments:
id_list -- List of object id to be checked
Returns:
Dict[str, bool] -- keys: input ids, values:
whether the transport has that object
str -- the full string representation of the object (or null if no object is found)
"""
pass
@@ -77,9 +72,12 @@ class AbstractTransport(ABC):
Arguments:
id {str} -- the id of the object you want to copy
target_transport {AbstractTransport}
-- the transport you want to copy the object to
target_transport {AbstractTransport} -- the transport you want to copy the object to
Returns:
str -- the string representation of the root object
"""
pass
class Config:
extra = Extra.allow
arbitrary_types_allowed = True
@@ -1,18 +1,18 @@
from typing import Dict, List
from specklepy.transports.abstract_transport import AbstractTransport
import json
from typing import Any
from speckle.logging.exceptions import SpeckleException
from speckle.transports.abstract_transport import AbstractTransport
class MemoryTransport(AbstractTransport):
def __init__(self, name="Memory") -> None:
super().__init__()
self._name = name
self.objects = {}
self.saved_object_count = 0
_name: str = "Memory"
objects: dict = {}
saved_object_count: int = 0
@property
def name(self) -> str:
return self._name
def __init__(self, name=None, **data: Any) -> None:
super().__init__(**data)
if name:
self._name = name
def __repr__(self) -> str:
return f"MemoryTransport(objects: {len(self.objects)})"
@@ -27,11 +27,11 @@ class MemoryTransport(AbstractTransport):
) -> None:
raise NotImplementedError
def get_object(self, id: str) -> str | None:
return self.objects.get(id, None)
def has_objects(self, id_list: List[str]) -> Dict[str, bool]:
return {id: (id in self.objects) for id in id_list}
def get_object(self, id: str) -> str or None:
if id in self.objects:
return self.objects[id]
else:
return None
def begin_write(self) -> None:
self.saved_object_count = 0
+103
View File
@@ -0,0 +1,103 @@
import requests
from asyncio import Queue, Task
from typing import Any, Dict, List, Type
from speckle.api.client import SpeckleClient
from speckle.logging.exceptions import SpeckleException
from speckle.transports.abstract_transport import AbstractTransport
class ServerTransport(AbstractTransport):
_name = "RemoteTransport"
url: str = None
stream_id: str = None
saved_obj_count: int = 0
session: requests.Session = None
__queue: Queue = None
__workers: List[Task] = []
def __init__(self, client: SpeckleClient, stream_id: str, **data: Any) -> None:
super().__init__(**data)
# TODO: replace client with account or some other auth avenue
self.url = client.url
self.stream_id = stream_id
self.session = requests.Session()
self.session.headers.update(
{"Authorization": f"Bearer {client.me['token']}", "Accept": "text/plain"}
)
def begin_write(self) -> None:
self.saved_obj_count = 0
def end_write(self) -> None:
pass
# TODO: add save task to queue and process as the root is being deserialised
def save_object(self, id: str, serialized_object: str) -> None:
endpoint = f"{self.url}/objects/{self.stream_id}"
r = self.session.post(
url=endpoint,
files={"batch-1": ("batch-1", f"[{serialized_object}]")},
)
if r.status_code != 201:
raise SpeckleException(
message=f"Could not save the object to the server - status code {r.status_code}"
)
def save_object_from_transport(
self, id: str, source_transport: AbstractTransport
) -> None:
obj_string = source_transport.get_object(id=id)
self.save_object(id=id, serialized_object=obj_string)
def get_object(self, id: str) -> str:
# endpoint = f"{self.url}/objects/{self.stream_id}/{id}/single"
# r = self.session.get(endpoint, stream=True)
# _, obj = next(r.iter_lines().decode("utf-8")).split("\t")
# return obj
raise SpeckleException(
"Getting a single object using `ServerTransport.get_object()` is not implemented. To get an object from the server, please use the `SpeckleClient.object.get()` route",
NotImplementedError,
)
def copy_object_and_children(
self, id: str, target_transport: AbstractTransport
) -> str:
endpoint = f"{self.url}/objects/{self.stream_id}/{id}"
r = self.session.get(endpoint, stream=True)
if r.encoding is None:
r.encoding = "utf-8"
lines = r.iter_lines(decode_unicode=True)
# save first (root) obj for return
root_hash, root_obj = next(lines).split("\t")
target_transport.save_object(root_hash, root_obj)
# iter through returned objects saving them as we go
for line in lines:
if line:
hash, obj = line.split("\t")
target_transport.save_object(hash, obj)
return root_obj
# async def stream_res(self, endpoint: str) -> str:
# data = b""
# async with aiohttp.ClientSession() as session:
# session.headers.update(
# {
# "Authorization": f"{self.session.headers['Authorization']}",
# "Accept": "text/plain",
# }
# )
# async with session.get(endpoint) as res:
# while True:
# chunk = await res.content.read(self.chunk_size)
# if not chunk:
# break
# data += chunk
# return data.decode("utf-8")
+199
View File
@@ -0,0 +1,199 @@
import os
import sys
import time
import sched
import sqlite3
from typing import Any
from appdirs import user_data_dir
from contextlib import closing
from multiprocessing import Process, Queue
from speckle.transports.abstract_transport import AbstractTransport
from speckle.logging.exceptions import SpeckleException
class SQLiteTransport(AbstractTransport):
_name = "SQLite"
_root_path: str = None
_is_writing: bool = False
_scheduler = sched.scheduler(time.time, time.sleep)
_polling_interval = 0.5 # seconds
__connection: sqlite3.Connection = None
__queue: Queue = Queue()
app_name: str = ""
scope: str = ""
saved_obj_count: int = 0
def __init__(
self,
base_path: str = None,
app_name: str = None,
scope: str = None,
**data: Any,
) -> None:
super().__init__(**data)
self.app_name = app_name or "Speckle"
self.scope = scope or "Objects"
base_path = base_path or self.__get_base_path()
os.makedirs(base_path, exist_ok=True)
self._root_path = os.path.join(os.path.join(base_path, f"{self.scope}.db"))
self.__initialise()
def __repr__(self) -> str:
return f"SQLiteTransport(app: '{self.app_name}', scope: '{self.scope}')"
def __write_timer_elapsed(self):
print("WRITE TIMER ELAPSED")
proc = Process(target=_run_queue, args=(self.__queue, self._root_path))
proc.start()
proc.join()
def __get_base_path(self):
# from appdirs https://github.com/ActiveState/appdirs/blob/master/appdirs.py
# default mac path is not the one we use (we use unix path), so using special case for this
system = sys.platform
if system.startswith("java"):
import platform
os_name = platform.java_ver()[3][0]
if os_name.startswith("Mac"):
system = "darwin"
if system == "darwin":
path = os.path.expanduser("~/.config/")
return os.path.join(path, self.app_name)
else:
return user_data_dir(appname=self.app_name, appauthor=False, roaming=True)
def __consume_queue(self):
if self._is_writing or self.__queue.empty():
return
print("CONSUME QUEUE")
self._is_writing = True
while not self.__queue.empty():
data = self.__queue.get()
self.save_object_sync(data[0], data[1])
self._is_writing = False
self._scheduler.enter(
delay=self._polling_interval, priority=1, action=self.__consume_queue
)
self._scheduler.run(blocking=True)
def save_object(self, id: str, serialized_object: str) -> None:
"""Adds an object to the queue and schedules it to be saved.
Arguments:
id {str} -- the object id
serialized_object {str} -- the full string representation of the object
"""
print("SAVE OBJECT")
self.__queue.put((id, serialized_object))
self._scheduler.enter(
delay=self._polling_interval, priority=1, action=self.__consume_queue
)
self._scheduler.run(blocking=True)
def save_object_from_transport(
self, id: str, source_transport: AbstractTransport
) -> None:
"""Adds an object from the given transport to the queue and schedules it to be saved.
Arguments:
id {str} -- the object id
source_transport {AbstractTransport) -- the transport through which the object can be found
"""
serialized_object = source_transport.get_object(id)
self.__queue.put((id, serialized_object))
raise NotImplementedError
def save_object_sync(self, id: str, serialized_object: str) -> None:
"""Directly saves an object into the database.
Arguments:
id {str} -- the object id
serialized_object {str} -- the full string representation of the object
"""
self.__check_connection()
try:
with closing(self.__connection.cursor()) as c:
c.execute(
"INSERT OR IGNORE INTO objects(hash, content) VALUES(?,?)",
(id, serialized_object),
)
self.__connection.commit()
except Exception as ex:
raise SpeckleException(
f"Could not save the object to the local db. Inner exception: {ex}", ex
)
def get_object(self, id: str) -> str or None:
self.__check_connection()
with closing(self.__connection.cursor()) as c:
row = c.execute(
"SELECT * FROM objects WHERE hash = ? LIMIT 1", (id,)
).fetchone()
return row[1] if row else None
def begin_write(self):
self.saved_obj_count = 0
def end_write(self):
pass
def copy_object_and_children(
self, id: str, target_transport: AbstractTransport
) -> str:
raise NotImplementedError
def get_all_objects(self):
"""Returns all the objects in the store. NOTE: do not use for large collections!"""
self.__check_connection()
with closing(self.__connection.cursor()) as c:
rows = c.execute("SELECT * FROM objects").fetchall()
return rows
def close(self):
"""Close the connection to the database"""
if self.__connection:
self.__connection.close()
self.__connection = None
def __initialise(self) -> None:
self.__connection = sqlite3.connect(self._root_path)
with closing(self.__connection.cursor()) as c:
c.execute(
""" CREATE TABLE IF NOT EXISTS objects(
hash TEXT PRIMARY KEY,
content TEXT
) WITHOUT ROWID;"""
)
c.execute("PRAGMA journal_mode='wal';")
c.execute("PRAGMA count_changes=OFF;")
c.execute("PRAGMA temp_store=MEMORY;")
self.__connection.commit()
def __check_connection(self):
if not self.__connection:
self.__connection = sqlite3.connect(self._root_path)
def __del__(self):
self.__connection.close()
def _run_queue(queue: Queue, root_path: str):
if queue.empty():
return
print("RUN QUEUE")
conn = sqlite3.connect(root_path)
while not queue.empty():
data = queue.get()
with closing(conn.cursor()) as c:
c.execute(
"INSERT OR IGNORE INTO objects(hash, content) VALUES(?,?)",
(data[0], data[1]),
)
conn.commit()
conn.close()
-1
View File
@@ -1 +0,0 @@
::: specklepy.api.client.SpeckleClient
@@ -1 +0,0 @@
::: specklepy.objects.data_objects.DataObject
@@ -1 +0,0 @@
::: specklepy.objects.data_objects.QgisObject
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IBlenderObject
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.ICurve
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IDataObject
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IDisplayValue
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IBlenderObject
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IHasArea
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IHasUnits
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IHasVolume
@@ -1 +0,0 @@
::: specklepy.objects.interfaces.IProperties
@@ -1 +0,0 @@
::: specklepy.objects.other.RenderMaterial
@@ -1 +0,0 @@
::: specklepy.objects.primitive.Interval
@@ -1 +0,0 @@
::: specklepy.objects.proxies.ColorProxy
@@ -1 +0,0 @@
::: specklepy.objects.proxies.GroupProxy
@@ -1 +0,0 @@
::: specklepy.objects.proxies.InstanceDefinitionProxy
@@ -1 +0,0 @@
::: specklepy.objects.proxies.InstanceProxy
@@ -1 +0,0 @@
::: specklepy.objects.proxies.RenderMaterialProxy
-1
View File
@@ -1 +0,0 @@
::: specklepy.objects.base.Base
@@ -1 +0,0 @@
::: specklepy.objects.geometry.arc.Arc
@@ -1 +0,0 @@
::: specklepy.objects.geometry.box.Box
@@ -1 +0,0 @@
::: specklepy.objects.geometry.circle.Circle
@@ -1 +0,0 @@
::: specklepy.objects.geometry.control_point.ControlPoint
@@ -1 +0,0 @@
::: specklepy.objects.geometry.ellipse.Ellipse
@@ -1 +0,0 @@
::: specklepy.objects.geometry.line.Line
@@ -1 +0,0 @@
::: specklepy.objects.geometry.mesh.Mesh
@@ -1 +0,0 @@
::: specklepy.objects.geometry.plane.Plane
@@ -1 +0,0 @@
::: specklepy.objects.geometry.point.Point
@@ -1 +0,0 @@
::: specklepy.objects.geometry.point_cloud.PointCloud
@@ -1 +0,0 @@
::: specklepy.objects.geometry.polycurve.Polycurve
@@ -1 +0,0 @@
::: specklepy.objects.geometry.polyline.Polyline
@@ -1 +0,0 @@
::: specklepy.objects.geometry.spiral.Spiral
@@ -1 +0,0 @@
::: specklepy.objects.geometry.surface.Surface
@@ -1 +0,0 @@
::: specklepy.objects.geometry.vector.Vector
Binary file not shown.

Before

Width:  |  Height:  |  Size: 386 B

-29
View File
@@ -1,29 +0,0 @@
# Introduction
Welcome to the Specklepy Developer Docs - a single source of documentation on everything Specklepy! If you're looking for info on how to use Speckle, check our [user guide](https://speckle.guide/).
### Code Repository
The Python SDK can be found in our [repository](//github.com/specklesystems/specklepy), its readme contains instructions on how to build it.
### Installation
You can install it using pip
```
pip install specklepy
```
### Key Components
SpecklePy has three main parts:
1. a `SpeckleClient` which allows you to interact with the server API
2. `operations` and `transports` for sending and receiving large objects
3. a `Base` object and accompaniying serializer for creating and customizing your own Speckle objects
### Local Data Paths
It may be helpful to know where the local accounts and object cache dbs are stored. Depending on on your OS, you can find the dbs at:
- Windows: `APPDATA` or `<USER>\AppData\Roaming\Speckle`
- Linux: `$XDG_DATA_HOME` or by default `~/.local/share/Speckle`
- Mac: `~/.config/Speckle`
@@ -1 +0,0 @@
::: speckle_automate.automation_context.AutomationContext
-64
View File
@@ -1,64 +0,0 @@
site_name: Specklepy Docs
theme:
name: material
favicon: assets/speckle_logo.png
logo: assets/speckle_logo.png
features:
- navigation.tabs
palette:
# Palette toggle for light mode
- scheme: default
primary: white
toggle:
icon: material/weather-night
name: Switch to dark mode
# Palette toggle for dark mode
- scheme: slate
primary: black
logo: assets/logo_white.png
toggle:
icon: material/weather-sunny
name: Switch to light mode
markdown_extensions:
- pymdownx.highlight:
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences
extra_css:
- css/mkdocstrings.css
plugins:
- search
- mkdocstrings:
handlers:
python:
paths: [.]
options:
parameter_headings: false
members_order: source
separate_signature: true
filters: ["!^_"] #Ignore _ prefixed properties
docstring_options:
ignore_init_summary: true
merge_init_into_class: true
show_signature_annotations: true
signature_crossrefs: true
show_if_no_docstring: true
show_labels: true
show_source: true
show_symbol_type_heading: true
show_symbol_type_toc: true
show_bases: false
heading_level: 3
inventories:
- url: https://docs.python.org/3/objects.inv
domains: [py, std]
-24
View File
@@ -1,24 +0,0 @@
"""This module contains an SDK for working with Speckle Automate."""
from speckle_automate.automation_context import AutomationContext
from speckle_automate.runner import execute_automate_function, run_function
from speckle_automate.schema import (
AutomateBase,
AutomationResult,
AutomationRunData,
AutomationStatus,
ObjectResultLevel,
ResultCase,
)
__all__ = [
"AutomationContext",
"AutomateBase",
"AutomationStatus",
"AutomationResult",
"AutomationRunData",
"ResultCase",
"ObjectResultLevel",
"run_function",
"execute_automate_function",
]
-519
View File
@@ -1,519 +0,0 @@
"""This module provides an abstraction layer above the Speckle Automate runtime."""
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional, Union
import httpx
from gql import gql
from speckle_automate.schema import (
AutomateBase,
AutomationResult,
AutomationRunData,
AutomationStatus,
ObjectResultLevel,
ResultCase,
)
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.version_inputs import CreateVersionInput
from specklepy.core.api.models.current import Model, Version
from specklepy.logging.exceptions import SpeckleException
from specklepy.objects.base import Base
from specklepy.transports.memory import MemoryTransport
from specklepy.transports.server import ServerTransport
@dataclass
class AutomationContext:
"""A context helper class.
This class exposes methods to work with the Speckle Automate context inside
Speckle Automate functions.
An instance of AutomationContext is injected into every run of a function.
"""
automation_run_data: AutomationRunData
speckle_client: SpeckleClient
_server_transport: ServerTransport
_speckle_token: str
#: keep a memory transponrt at hand, to speed up things if needed
_memory_transport: MemoryTransport = field(default_factory=MemoryTransport)
#: added for performance measuring
_init_time: float = field(default_factory=time.perf_counter)
_automation_result: AutomationResult = field(default_factory=AutomationResult)
@classmethod
def initialize(
cls, automation_run_data: Union[str, AutomationRunData], speckle_token: str
) -> "AutomationContext":
"""Bootstrap the AutomateSDK from raw data.
Todo:
----
* bootstrap a structlog logger instance
* expose a logger, that ppl can use instead of print
"""
# parse the json value if its not an initialized project data instance
automation_run_data = (
automation_run_data
if isinstance(automation_run_data, AutomationRunData)
else AutomationRunData.model_validate_json(automation_run_data)
)
speckle_client = SpeckleClient(
automation_run_data.speckle_server_url,
automation_run_data.speckle_server_url.startswith("https"),
)
speckle_client.authenticate_with_token(speckle_token)
if not speckle_client.account:
msg = (
f"Could not authenticate to {automation_run_data.speckle_server_url}",
"with the provided token",
)
raise ValueError(msg)
server_transport = ServerTransport(
automation_run_data.project_id, speckle_client
)
return cls(automation_run_data, speckle_client, server_transport, speckle_token)
@property
def run_status(self) -> AutomationStatus:
"""Get the status of the automation run."""
return self._automation_result.run_status
@property
def status_message(self) -> Optional[str]:
"""Get the current status message."""
return self._automation_result.status_message
def elapsed(self) -> float:
"""Return the elapsed time in seconds since the initialization time."""
return time.perf_counter() - self._init_time
def receive_version(self) -> Base:
"""Receive the Speckle project version that triggered this automation run."""
# TODO: this is a quick hack to keep implementation consistency.
# Move to proper receive many versions
version_id = self.automation_run_data.triggers[0].payload.version_id
try:
version = self.speckle_client.version.get(
version_id, self.automation_run_data.project_id
)
except SpeckleException as err:
raise ValueError(
f"""Could not receive specified version.
Is your environment configured correctly?
project_id: {self.automation_run_data.project_id}
model_id: {self.automation_run_data.triggers[0].payload.model_id}
version_id: {self.automation_run_data.triggers[0].payload.version_id}
"""
) from err
if not version.referenced_object:
raise Exception(
"This version is past the version history limit,",
" cannot execute an automation on it",
)
base = operations.receive(
version.referenced_object, self._server_transport, self._memory_transport
)
# self._closure_tree = base["__closure"]
print(
f"It took {self.elapsed():.2f} seconds to receive",
f" the speckle version {version_id}",
)
return base
def create_new_model_in_project(
self, model_name: str, model_description: Optional[str] = None
) -> Model:
input = CreateModelInput(
name=model_name,
description=model_description,
project_id=self.automation_run_data.project_id,
)
return self.speckle_client.model.create(input)
def get_model(self, model_id: str) -> Model:
"""
Args:
model_id (str): The id of the model to get
"""
return self.speckle_client.model.get(
model_id, self.automation_run_data.project_id
)
def create_new_version_in_project(
self, root_object: Base, model_id: str, version_message: str = ""
) -> Version:
"""Save a base model to a new version on the project.
Args:
root_object (Base): The Speckle base object for the new version.
model_id (str): Id of model to create the new version on.
version_message (str): The message for the new version.
"""
matching_trigger = [
t
for t in self.automation_run_data.triggers
if t.payload.model_id == model_id
]
if matching_trigger:
raise ValueError(
f"The target model: {model_id} cannot match the model"
f" that triggered this automation:"
f" {matching_trigger[0].payload.model_id}"
)
root_object_id = operations.send(
root_object,
[self._server_transport, self._memory_transport],
use_default_cache=False,
)
create_version_input = CreateVersionInput(
object_id=root_object_id,
model_id=model_id,
project_id=self.automation_run_data.project_id,
message=version_message,
source_application="SpeckleAutomate",
)
version = self.speckle_client.version.create(create_version_input)
self._automation_result.result_versions.append(version.id)
return version
@property
def context_view(self) -> Optional[str]:
return self._automation_result.result_view
def set_context_view(
self,
# f"{model_id}@{version_id} or {model_id} "
resource_ids: Optional[List[str]] = None,
include_source_model_version: bool = True,
) -> None:
link_resources = (
[
f"{t.payload.model_id}@{t.payload.version_id}"
for t in self.automation_run_data.triggers
]
if include_source_model_version
else []
)
if resource_ids:
link_resources.extend(resource_ids)
if not link_resources:
raise Exception(
"We do not have enough resource ids to compose a context view"
)
self._automation_result.result_view = (
f"/projects/{self.automation_run_data.project_id}"
f"/models/{','.join(link_resources)}"
)
def report_run_status(self) -> None:
"""Report the current run status to the project of this automation."""
query = gql(
"""
mutation AutomateFunctionRunStatusReport(
$projectId: String!
$functionRunId: String!
$status: AutomateRunStatus!
$statusMessage: String
$results: JSONObject
$contextView: String
){
automateFunctionRunStatusReport(input: {
projectId: $projectId
functionRunId: $functionRunId
status: $status
statusMessage: $statusMessage
contextView: $contextView
results: $results
})
}
"""
)
if self.run_status in [AutomationStatus.SUCCEEDED, AutomationStatus.FAILED]:
results_dict = self._automation_result.model_dump(by_alias=True)
results = {
"version": 3,
"values": {
"objectResults": results_dict["objectResults"],
"versionResult": results_dict["versionResult"],
"blobIds": self._automation_result.blobs,
},
}
else:
results = None
params = {
"projectId": self.automation_run_data.project_id,
"functionRunId": self.automation_run_data.function_run_id,
"status": self.run_status.value,
"statusMessage": self._automation_result.status_message,
"results": results,
"contextView": self._automation_result.result_view,
}
print(f"Reporting run status with content: {params}")
self.speckle_client.httpclient.execute(query, params)
def store_file_result(self, file_path: Union[Path, str]) -> str:
"""Save a file attached to the project of this automation."""
path_obj = (
Path(file_path).resolve() if isinstance(file_path, str) else file_path
)
class UploadResult(AutomateBase):
blob_id: str
file_name: str
upload_status: int
class BlobUploadResponse(AutomateBase):
upload_results: list[UploadResult]
if not path_obj.exists():
raise ValueError("The given file path doesn't exist")
files = {path_obj.name: path_obj.open("rb")}
url = (
f"{self.automation_run_data.speckle_server_url}api/stream/"
f"{self.automation_run_data.project_id}/blob"
)
data = (
httpx.post(
url,
files=files,
headers={"authorization": f"Bearer {self._speckle_token}"},
)
.raise_for_status()
.json()
)
upload_response = BlobUploadResponse.model_validate(data)
if len(upload_response.upload_results) != 1:
raise ValueError("Expecting one upload result.")
self._automation_result.blobs.extend(
[upload_result.blob_id for upload_result in upload_response.upload_results]
)
return upload_response.upload_results[0].blob_id
def mark_run_failed(
self, status_message: str, version_result: dict[str, Any] | None = None
) -> None:
"""
Mark the current run a failure.
Args:
status_message: Optional message to be displayed.
version_result: Optional data object,
that will be attached to the run results.
The dictionary should be JSON serializable
"""
self._mark_run(AutomationStatus.FAILED, status_message, version_result)
def mark_run_exception(self, status_message: str) -> None:
"""Mark the current run a failure."""
self._mark_run(AutomationStatus.EXCEPTION, status_message, None)
def mark_run_success(
self, status_message: str | None, version_result: dict[str, Any] | None = None
) -> None:
"""
Mark the current run a success with an optional message.
Args:
status_message: Optional message to be displayed.
version_result: Optional data object,
that will be attached to the run results.
The dictionary should be JSON serializable
"""
self._mark_run(AutomationStatus.SUCCEEDED, status_message, version_result)
def _mark_run(
self,
status: AutomationStatus,
status_message: str | None,
version_result: dict[str, Any] | None,
) -> None:
duration = self.elapsed()
self._automation_result.status_message = status_message
self._automation_result.run_status = status
self._automation_result.elapsed = duration
self._automation_result.version_result = version_result
msg = f"Automation run {status.value} after {duration:.2f} seconds."
print("\n".join([msg, status_message]) if status_message else msg)
def attach_error_to_objects(
self,
category: str,
affected_objects: Union[Base, List[Base]],
message: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
visual_overrides: Optional[Dict[str, Any]] = None,
) -> None:
"""Add a new error case to the run results.
Args:
category (str): A short tag for the event type.
affected_objects (Union[Base, List[Base]]): A single object or a list of
objects that are causing the error case.
message (Optional[str]): Optional message.
metadata: User provided metadata key value pairs
visual_overrides: Case specific 3D visual overrides.
"""
self.attach_result_to_objects(
ObjectResultLevel.ERROR,
category,
affected_objects,
message,
metadata,
visual_overrides,
)
def attach_warning_to_objects(
self,
category: str,
affected_objects: Union[Base, List[Base]],
message: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
visual_overrides: Optional[Dict[str, Any]] = None,
) -> None:
"""Add a new warning case to the run results.
Args:
category (str): A short tag for the event type.
affected_objects (Union[Base, List[Base]]): A single object or a list of
objects that are causing the warning case.
message (Optional[str]): Optional message.
metadata: User provided metadata key value pairs
visual_overrides: Case specific 3D visual overrides.
"""
self.attach_result_to_objects(
ObjectResultLevel.WARNING,
category,
affected_objects,
message,
metadata,
visual_overrides,
)
def attach_success_to_objects(
self,
category: str,
affected_objects: Union[Base, List[Base]],
message: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
visual_overrides: Optional[Dict[str, Any]] = None,
) -> None:
"""Add a new success case to the run results.
Args:
category (str): A short tag for the event type.
affected_objects (Union[Base, List[Base]]): A single object or a list of
objects that are causing the success case.
message (Optional[str]): Optional message.
metadata: User provided metadata key value pairs
visual_overrides: Case specific 3D visual overrides.
"""
self.attach_result_to_objects(
ObjectResultLevel.SUCCESS,
category,
affected_objects,
message,
metadata,
visual_overrides,
)
def attach_info_to_objects(
self,
category: str,
affected_objects: Union[Base, List[Base]],
message: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
visual_overrides: Optional[Dict[str, Any]] = None,
) -> None:
"""Add a new info case to the run results.
Args:
category (str): A short tag for the event type.
affected_objects (Union[Base, List[Base]]): A single object or a list of
objects that are causing the info case.
message (Optional[str]): Optional message.
metadata: User provided metadata key value pairs
visual_overrides: Case specific 3D visual overrides.
"""
self.attach_result_to_objects(
ObjectResultLevel.INFO,
category,
affected_objects,
message,
metadata,
visual_overrides,
)
def attach_result_to_objects(
self,
level: ObjectResultLevel,
category: str,
affected_objects: Union[Base, List[Base]],
message: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
visual_overrides: Optional[Dict[str, Any]] = None,
) -> None:
"""Add a new result case to the run results.
Args:
level: Result level.
category (str): A short tag for the event type.
affected_objects (Union[Base, List[Base]]): A single object or a list of
objects that are causing the info case.
message (Optional[str]): Optional message.
metadata: User provided metadata key value pairs
visual_overrides: Case specific 3D visual overrides.
"""
if isinstance(affected_objects, list):
if len(affected_objects) < 1:
raise ValueError(
f"Need atleast one object to report a(n) {level.value.upper()}"
)
object_list = affected_objects
else:
object_list = [affected_objects]
ids: Dict[str, Optional[str]] = {}
for o in object_list:
# validate that the Base.id is not None. If its a None, throw an Exception
if not o.id:
raise Exception(
f"You can only attach {level} results to objects with an id."
)
ids[o.id] = o.applicationId
print(
f"Created new {level.value.upper()}"
f" category: {category} caused by: {message}"
)
self._automation_result.object_results.append(
ResultCase(
category=category,
level=level,
object_app_ids=ids,
message=message,
metadata=metadata,
visual_overrides=visual_overrides,
)
)
-144
View File
@@ -1,144 +0,0 @@
"""Some useful helpers for working with automation data."""
import pytest
from gql import gql
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict
from speckle_automate.schema import AutomationRunData, TestAutomationRunData
from specklepy.api.client import SpeckleClient
class TestAutomationEnvironment(BaseSettings):
"""Get known environment variables from local `.env` file"""
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
env_prefix="speckle_",
extra="ignore",
)
token: str = Field()
server_url: str = Field()
project_id: str = Field()
automation_id: str = Field()
@pytest.fixture()
def test_automation_environment() -> TestAutomationEnvironment:
return TestAutomationEnvironment()
@pytest.fixture()
def test_automation_token(
test_automation_environment: TestAutomationEnvironment,
) -> str:
"""Provide a speckle token for the test suite."""
return test_automation_environment.token
@pytest.fixture()
def speckle_client(
test_automation_environment: TestAutomationEnvironment,
) -> SpeckleClient:
"""Initialize a SpeckleClient for testing."""
speckle_client = SpeckleClient(
test_automation_environment.server_url,
test_automation_environment.server_url.startswith("https"),
)
speckle_client.authenticate_with_token(test_automation_environment.token)
return speckle_client
def create_test_automation_run(
speckle_client: SpeckleClient, project_id: str, test_automation_id: str
) -> TestAutomationRunData:
"""Create test run to report local test results to"""
query = gql(
"""
mutation CreateTestRun(
$projectId: ID!,
$automationId: ID!
) {
projectMutations {
automationMutations(projectId: $projectId) {
createTestAutomationRun(automationId: $automationId) {
automationRunId
functionRunId
triggers {
payload {
modelId
versionId
}
triggerType
}
}
}
}
}
"""
)
params = {"automationId": test_automation_id, "projectId": project_id}
result = speckle_client.httpclient.execute(query, params)
print(result)
return TestAutomationRunData.model_validate(
result["projectMutations"]["automationMutations"]["createTestAutomationRun"]
)
@pytest.fixture()
def test_automation_run(
speckle_client: SpeckleClient,
test_automation_environment: TestAutomationEnvironment,
) -> TestAutomationRunData:
return create_test_automation_run(
speckle_client,
test_automation_environment.project_id,
test_automation_environment.automation_id,
)
def create_test_automation_run_data(
speckle_client: SpeckleClient,
test_automation_environment: TestAutomationEnvironment,
) -> AutomationRunData:
"""Create automation run data for a new run for a given test automation"""
test_automation_run_data = create_test_automation_run(
speckle_client,
test_automation_environment.project_id,
test_automation_environment.automation_id,
)
return AutomationRunData(
project_id=test_automation_environment.project_id,
speckle_server_url=test_automation_environment.server_url,
automation_id=test_automation_environment.automation_id,
automation_run_id=test_automation_run_data.automation_run_id,
function_run_id=test_automation_run_data.function_run_id,
triggers=test_automation_run_data.triggers,
)
@pytest.fixture()
def test_automation_run_data(
speckle_client: SpeckleClient,
test_automation_environment: TestAutomationEnvironment,
) -> AutomationRunData:
return create_test_automation_run_data(speckle_client, test_automation_environment)
__all__ = [
"test_automation_environment",
"test_automation_token",
"speckle_client",
"test_automation_run",
"test_automation_run_data",
]
-194
View File
@@ -1,194 +0,0 @@
"""Function execution module.
Provides mechanisms to execute any function,
that conforms to the AutomateFunction "interface"
"""
import json
import sys
import traceback
from pathlib import Path
from typing import Callable, Optional, Tuple, TypeVar, Union, overload
from pydantic import create_model
from pydantic.json_schema import GenerateJsonSchema
from speckle_automate.automation_context import AutomationContext
from speckle_automate.schema import AutomateBase, AutomationRunData, AutomationStatus
T = TypeVar("T", bound=AutomateBase)
AutomateFunction = Callable[[AutomationContext, T], None]
AutomateFunctionWithoutInputs = Callable[[AutomationContext], None]
def _read_input_data(inputs_location: str) -> str:
input_path = Path(inputs_location)
if not input_path.exists():
raise ValueError(f"Cannot find the function inputs file at {input_path}")
return input_path.read_text()
def _parse_input_data(
input_location: str, input_schema: Optional[type[T]]
) -> Tuple[AutomationRunData, Optional[T], str]:
input_json_string = _read_input_data(input_location)
class FunctionRunData(AutomateBase):
speckle_token: str
automation_run_data: AutomationRunData
function_inputs: None = None
parser_model = FunctionRunData
if input_schema:
parser_model = create_model(
"FunctionRunDataWithInputs",
function_inputs=(input_schema, ...),
__base__=FunctionRunData,
)
input_data = parser_model.model_validate_json(input_json_string)
return (
input_data.automation_run_data,
input_data.function_inputs,
input_data.speckle_token,
)
@overload
def execute_automate_function(
automate_function: AutomateFunction[T],
input_schema: type[T],
) -> None: ...
@overload
def execute_automate_function(
automate_function: AutomateFunctionWithoutInputs,
) -> None: ...
class AutomateGenerateJsonSchema(GenerateJsonSchema):
def generate(self, schema, mode="validation"):
json_schema = super().generate(schema, mode=mode)
json_schema["$schema"] = self.schema_dialect
return json_schema
def execute_automate_function(
automate_function: Union[AutomateFunction[T], AutomateFunctionWithoutInputs],
input_schema: Optional[type[T]] = None,
):
"""Runs the provided automate function with the input schema."""
# first arg is the python file name, we do not need that
args = sys.argv[1:]
if len(args) != 2:
raise ValueError("Incorrect number of arguments specified need 2")
# we rely on a command name convention to decide what to do.
# this is here, so that the function authors do not see any of this
command, argument = args
if command == "generate_schema":
path = Path(argument)
schema = json.dumps(
input_schema.model_json_schema(
by_alias=True, schema_generator=AutomateGenerateJsonSchema
)
if input_schema
else {}
)
path.write_text(schema)
elif command == "run":
automation_run_data, function_inputs, speckle_token = _parse_input_data(
argument, input_schema
)
automation_context = AutomationContext.initialize(
automation_run_data, speckle_token
)
if function_inputs:
automation_context = run_function(
automation_context,
automate_function, # type: ignore
function_inputs, # type: ignore
)
else:
automation_context = AutomationContext.initialize(
automation_run_data, speckle_token
)
automation_context = run_function(
automation_context,
automate_function, # type: ignore
)
# if we've gotten this far,
# the execution should technically be completed as expected
# thus exiting with 0 is the schemantically correct thing to do
exit_code = (
1 if automation_context.run_status == AutomationStatus.EXCEPTION else 0
)
exit(exit_code)
else:
raise NotImplementedError(f"Command: '{command}' is not supported.")
@overload
def run_function(
automation_context: AutomationContext,
automate_function: AutomateFunction[T],
inputs: T,
) -> AutomationContext: ...
@overload
def run_function(
automation_context: AutomationContext,
automate_function: AutomateFunctionWithoutInputs,
) -> AutomationContext: ...
def run_function(
automation_context: AutomationContext,
automate_function: Union[AutomateFunction[T], AutomateFunctionWithoutInputs],
inputs: Optional[T] = None,
) -> AutomationContext:
"""Run the provided function with the automate sdk context."""
automation_context.report_run_status()
try:
# avoiding complex type gymnastics here on the internals.
# the external type overloads make this correct
if inputs:
automate_function(automation_context, inputs) # type: ignore
else:
automate_function(automation_context) # type: ignore
# the function author forgot to mark the function success
if automation_context.run_status not in [
AutomationStatus.FAILED,
AutomationStatus.SUCCEEDED,
AutomationStatus.EXCEPTION,
]:
automation_context.mark_run_success(
"WARNING: Automate assumed a success status,"
" but it was not marked as so by the function."
)
except Exception:
trace = traceback.format_exc()
print(trace)
automation_context.mark_run_exception(
"Function error. Check the automation run logs for details."
)
finally:
if not automation_context.context_view:
automation_context.set_context_view()
automation_context.report_run_status()
return automation_context
-99
View File
@@ -1,99 +0,0 @@
""""""
from enum import Enum
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field
from pydantic.alias_generators import to_camel
class AutomateBase(BaseModel):
"""Use this class as a base model for automate related DTO."""
model_config = ConfigDict(alias_generator=to_camel, populate_by_name=True)
class VersionCreationTriggerPayload(AutomateBase):
"""Represents the version creation trigger payload."""
model_id: str
version_id: str
class VersionCreationTrigger(AutomateBase):
"""Represents a single version creation trigger for the automation run."""
trigger_type: Literal["versionCreation"]
payload: VersionCreationTriggerPayload
class AutomationRunData(BaseModel):
"""Values of the project / model that triggered the run of this function."""
project_id: str
speckle_server_url: str
automation_id: str
automation_run_id: str
function_run_id: str
triggers: list[VersionCreationTrigger]
model_config = ConfigDict(
alias_generator=to_camel, populate_by_name=True, protected_namespaces=()
)
class TestAutomationRunData(BaseModel):
"""Values of the run created in the test automation for local test results."""
automation_run_id: str
function_run_id: str
triggers: list[VersionCreationTrigger]
model_config = ConfigDict(
alias_generator=to_camel, populate_by_name=True, protected_namespaces=()
)
class AutomationStatus(str, Enum):
"""Set the status of the automation."""
INITIALIZING = "INITIALIZING"
RUNNING = "RUNNING"
FAILED = "FAILED"
SUCCEEDED = "SUCCEEDED"
EXCEPTION = "EXCEPTION"
class ObjectResultLevel(str, Enum):
"""Possible status message levels for object reports."""
SUCCESS = "SUCCESS"
INFO = "INFO"
WARNING = "WARNING"
ERROR = "ERROR"
class ResultCase(AutomateBase):
"""A result case."""
category: str
level: ObjectResultLevel
object_app_ids: dict[str, str | None]
message: str | None
metadata: dict[str, Any] | None
visual_overrides: dict[str, Any] | None
class AutomationResult(AutomateBase):
"""Schema accepted by the Speckle server as a result for an automation run."""
elapsed: float = 0
result_view: str | None = None
result_versions: list[str] = Field(default_factory=list)
blobs: list[str] = Field(default_factory=list)
run_status: AutomationStatus = AutomationStatus.RUNNING
status_message: str | None = None
object_results: list[ResultCase] = Field(default_factory=list)
version_result: dict[str, Any] | None = None
-61
View File
@@ -1,61 +0,0 @@
import json
import time
import traceback
from argparse import ArgumentParser
from os import getenv
from speckleifc.main import open_and_convert_file
from specklepy.core.api.client import SpeckleClient
from specklepy.logging import metrics
def cmd_line_import() -> None:
parser = ArgumentParser(
prog="speckleifc",
description="imports a file",
)
parser.add_argument("file_path")
parser.add_argument("output_path")
parser.add_argument("project_id")
parser.add_argument("version_message")
parser.add_argument("model_id")
# parser.add_argument("model_name")
# parser.add_argument("region_name")
args = parser.parse_args()
TOKEN = getenv("USER_TOKEN")
assert TOKEN is not None
SERVER_URL = getenv("SPECKLE_SERVER_URL") or "http://127.0.0.1:3000"
metrics.set_host_app(
"ifc",
)
try:
client = SpeckleClient(SERVER_URL, use_ssl=not SERVER_URL.startswith("http://"))
client.authenticate_with_token(TOKEN)
project = client.project.get(args.project_id)
version = open_and_convert_file(
args.file_path,
project,
args.version_message,
args.model_id,
client,
)
with open(args.output_path, "w") as f:
json.dump({"success": True, "commitId": version.id}, f)
except Exception as e:
error_msg = f"IFC Importer failed with exception:\n{traceback.format_exc()}"
print(error_msg)
# Write error result
with open(args.output_path, "w") as f:
json.dump({"success": False, "error": str(e)}, f)
if __name__ == "__main__":
start = time.time()
cmd_line_import()
print(f"Total time (including cleanup): {(time.time() - start) * 1000}ms")
@@ -1,35 +0,0 @@
from typing import cast
from ifcopenshell.entity_instance import entity_instance
from speckleifc.property_extraction import extract_properties
from specklepy.objects.base import Base
from specklepy.objects.data_objects import DataObject
def data_object_to_speckle(
display_value: list[Base],
step_element: entity_instance,
children: list[Base],
current_storey: str | None = None,
) -> DataObject:
guid = cast(str, step_element.GlobalId)
name = cast(str, step_element.Name or guid)
properties = extract_properties(step_element)
# Add building storey information if available and not a building storey itself
if current_storey and not step_element.is_a("IfcBuildingStorey"):
properties["Building Storey"] = current_storey
data_object = DataObject(
applicationId=guid,
properties=properties,
name=name or guid,
displayValue=display_value,
)
data_object["@elements"] = children
data_object["ifcType"] = step_element.is_a()
return data_object

Some files were not shown because too many files have changed in this diff Show More