Compare commits

...

10 Commits

Author SHA1 Message Date
Jedd Morgan fd8c2a32f9 chore(speckleifc): changed ifc status messages (#478)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* remove this function

* Changed progress messages
2025-12-09 17:27:26 +00:00
Jedd Morgan ba8c356d82 chore(speckleifc): Ifc metrics slug tweaks (#477)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* ifc metrics

* add http server tests for metrics

* clean up tests

* change back to localhost:3000

* comment

* renamed wrapper for clarity

* fix unrelated model_ingestion
2025-12-09 16:18:21 +01:00
Gergő Jedlicska 8249cd2184 Jedd/cxpla 340 specklepy (#475)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* First pass

* add tests

* Add cancellation

* fix

* status changes

* fixes

* test fixes

* tests(subscriptions): fix model ingestion tests

* feat(modelingestion): rename resource and add some more tests

* feat(ifcimport): use new modelingestion api

* feat: wrap up new ingestion

* fix: model ingestion payload and test server url

* fix: test port was 3000

* fix: remove version message from model ingestion success input

* fix: test subs cancelled

* ci: signal public of private envv in ci

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-12-08 13:25:54 +01:00
Gergő Jedlicska 7c108a9d43 feat(speckle_automate): version receive metrics (#470)
make sure the automate metrics are attributed to automate host app use
the version metrics to report the version received into automate.

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-11-27 13:27:25 +00:00
Dogukan Karatas 2f2e8ba734 fix: netlify toml path fix (#473)
* updated path

* Add chaining

* debugging deploy

* docs added

* revert back

* disable mise

* feat: use mise for docs build

* fix: netlify do not tripple quote

* ci: netlify use mise

* ci: just run build

---------

Co-authored-by: Mike Tasset <mike.tasset@gmail.com>
Co-authored-by: Gergo Jedlicska <gergo@jedlicska.com>
2025-11-21 15:26:59 +01:00
Dogukan Karatas 9685a2741b updated toml (#472) 2025-11-20 17:13:55 +01:00
Dogukan Karatas 5702d116d0 docs: API Reference (#471)
* first pass

* round2

* experiments

* Moved docs one layer higher

* docs poc

* re-lock

* created docs

* some ci work

* updated toml

* ruff version update

* update toml

* docs group

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-11-20 16:27:01 +01:00
Arseney d440bb5c0f DefaultRule in GraphTraversal was missing should_return method (#469)
* Changed positions in code for _get_active_rule

* should return function added to DefaultRule

* DefaultRule should_return method returns false
2025-11-18 16:12:14 +00:00
Gergő Jedlicska 309c78da37 feat(automate_sdk): support version result reporting (#468)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* feat(automate_sdk): support version result reporting

* fix(automate_tests): skip test, its not working in CI

* docs(automate_sdk): describe new args
2025-11-05 21:50:40 +01:00
Jedd Morgan ff812d5ad9 add 520 status to retry policy (#467) 2025-11-05 10:49:36 +00:00
97 changed files with 4272 additions and 1357 deletions
+2
View File
@@ -59,6 +59,8 @@ jobs:
test-public: # Run integration tests against the public server image
name: Test (public)
runs-on: ubuntu-latest
env:
IS_PUBLIC: "true"
strategy:
matrix:
python-version:
+1
View File
@@ -0,0 +1 @@
words = ["specklepy"]
Binary file not shown.

After

Width:  |  Height:  |  Size: 386 B

+50
View File
@@ -0,0 +1,50 @@
# specklepy API Reference
> The Python SDK for Speckle - Build powerful AEC data workflows
**specklepy** is the Python SDK for Speckle, enabling you to interact with Speckle Server, send and receive geometry, and build custom integrations for the AEC industry.
## What is specklepy?
specklepy is a comprehensive Python library that provides:
* **Object-based data exchange** - Send and receive geometry and BIM data without files
* **GraphQL API client** - Full access to Speckle Server's API
* **Extensible object model** - Create custom objects that inherit from `Base`
* **Multiple transport options** - Store data locally (SQLite), in-memory, or on Speckle Server
* **Geometry support** - Rich geometric primitives (Point, Line, Mesh, etc.)
## Speckle Automate
Speckle Automate is a fully fledged CI/CD platform designed to run custom code on Speckle models whenever a new version is available.
As a software developer, you can develop Functions that others in your team consume in Automations. From creating reports to running code compliance checks to wind simulations, there is no limit to what you can do with Automate.
## Installation
Install specklepy using pip:
```bash
pip install specklepy
```
## Quick Example
```python
from specklepy.api.client import SpeckleClient
from specklepy.api.credentials import get_default_account
from specklepy.objects.geometry import Point
# Authenticate
client = SpeckleClient(host="https://app.speckle.systems")
account = get_default_account()
client.authenticate_with_account(account)
# Create geometry
point = Point(x=10, y=20, z=5)
```
## Getting Help
- **Community Forum**: [speckle.community](https://speckle.community/c/help/developers)
- **GitHub Issues**: [github.com/specklesystems/specklepy](https://github.com/specklesystems/specklepy/issues)
@@ -0,0 +1 @@
::: speckle_automate.AutomationContext
+3
View File
@@ -0,0 +1,3 @@
::: speckle_automate.runner.execute_automate_function
::: speckle_automate.runner.run_function
+11
View File
@@ -0,0 +1,11 @@
::: speckle_automate.AutomateBase
::: speckle_automate.AutomationRunData
::: speckle_automate.AutomationResult
::: speckle_automate.ResultCase
::: speckle_automate.AutomationStatus
::: speckle_automate.ObjectResultLevel
+1
View File
@@ -0,0 +1 @@
::: specklepy.api.client.SpeckleClient
+5
View File
@@ -0,0 +1,5 @@
::: specklepy.api.credentials.Account
::: specklepy.api.credentials.UserInfo
::: specklepy.api.credentials.StreamWrapper
+7
View File
@@ -0,0 +1,7 @@
::: specklepy.api.operations.send
::: specklepy.api.operations.receive
::: specklepy.api.operations.serialize
::: specklepy.api.operations.deserialize
@@ -0,0 +1 @@
::: specklepy.api.resources.ActiveUserResource
@@ -0,0 +1 @@
::: specklepy.api.resources.FileImportResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ModelResource
@@ -0,0 +1 @@
::: specklepy.api.resources.OtherUserResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ProjectInviteResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ProjectResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ServerResource
@@ -0,0 +1 @@
::: specklepy.api.resources.SubscriptionResource
@@ -0,0 +1 @@
::: specklepy.api.resources.VersionResource
@@ -0,0 +1 @@
::: specklepy.api.resources.WorkspaceResource
+1
View File
@@ -0,0 +1 @@
::: specklepy.core.api.enums.ProjectVisibility
+11
View File
@@ -0,0 +1,11 @@
::: specklepy.core.api.inputs.ProjectCreateInput
::: specklepy.core.api.inputs.ProjectUpdateInput
::: specklepy.core.api.inputs.CreateModelInput
::: specklepy.core.api.inputs.UpdateModelInput
::: specklepy.core.api.inputs.CreateVersionInput
::: specklepy.core.api.inputs.UpdateVersionInput
+13
View File
@@ -0,0 +1,13 @@
::: specklepy.core.api.models.User
::: specklepy.core.api.models.LimitedUser
::: specklepy.core.api.models.ServerInfo
::: specklepy.core.api.models.Project
::: specklepy.core.api.models.Model
::: specklepy.core.api.models.Version
::: specklepy.core.api.models.current.Workspace
+7
View File
@@ -0,0 +1,7 @@
::: specklepy.logging.exceptions.SpeckleException
::: specklepy.logging.exceptions.GraphQLException
::: specklepy.logging.exceptions.SerializationException
::: specklepy.logging.exceptions.SpeckleWarning
+3
View File
@@ -0,0 +1,3 @@
::: specklepy.objects.Base
::: specklepy.objects.base.DataChunk
+5
View File
@@ -0,0 +1,5 @@
::: specklepy.objects.DataObject
::: specklepy.objects.QgisObject
::: specklepy.objects.BlenderObject
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Arc
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Box
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Circle
@@ -0,0 +1 @@
::: specklepy.objects.geometry.ControlPoint
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Curve
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Ellipse
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Line
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Mesh
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Plane
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Point
@@ -0,0 +1 @@
::: specklepy.objects.geometry.PointCloud
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Polycurve
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Polyline
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Spiral
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Surface
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Vector
@@ -0,0 +1,7 @@
::: specklepy.objects.graph_traversal.GraphTraversal
::: specklepy.objects.graph_traversal.TraversalContext
::: specklepy.objects.graph_traversal.TraversalRule
::: specklepy.objects.graph_traversal.DefaultRule
@@ -0,0 +1 @@
::: specklepy.objects.models.collections.collection.Collection
@@ -0,0 +1 @@
::: specklepy.objects.other.RenderMaterial
@@ -0,0 +1 @@
::: specklepy.objects.primitive.Interval
@@ -0,0 +1 @@
::: specklepy.objects.proxies.ColorProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.GroupProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.InstanceDefinitionProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.InstanceProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.LevelProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.RenderMaterialProxy
@@ -0,0 +1 @@
::: specklepy.serialization.base_object_serializer.BaseObjectSerializer
+7
View File
@@ -0,0 +1,7 @@
::: specklepy.transports.abstract_transport.AbstractTransport
::: specklepy.transports.memory.MemoryTransport
::: specklepy.transports.sqlite.SQLiteTransport
::: specklepy.transports.server.ServerTransport
+304
View File
@@ -0,0 +1,304 @@
.md-content h1 {
font-size: 1.125rem;
font-weight: 700;
color: #111827;
letter-spacing: -0.025em;
line-height: 1.2;
}
@media (min-width: 640px) {
.md-content h1 {
font-size: 1.25rem;
}
}
[data-md-color-scheme="slate"] .md-content h1 {
color: #e5e7eb;
}
.md-content h2 {
font-size: 1rem;
font-weight: 700;
color: #111827;
letter-spacing: -0.025em;
margin-top: 2rem;
}
@media (min-width: 640px) {
.md-content h2 {
font-size: 1.125rem;
}
}
[data-md-color-scheme="slate"] .md-content h2 {
color: #e5e7eb;
}
.md-content h3 {
font-size: 0.9375rem;
font-weight: 600;
color: #1f2937;
margin-top: 1.5rem;
}
@media (min-width: 640px) {
.md-content h3 {
font-size: 1rem;
}
}
[data-md-color-scheme="slate"] .md-content h3 {
color: #d1d5db;
}
.md-content p,
.md-content li {
font-size: 0.875rem;
line-height: 1.6;
color: #374151;
margin-top: 0.5rem;
}
[data-md-color-scheme="slate"] .md-content p,
[data-md-color-scheme="slate"] .md-content li {
color: #d1d5db;
}
.doc.doc-object-name {
font-size: 1.125rem;
font-weight: 700;
color: #111827;
}
[data-md-color-scheme="slate"] .doc.doc-object-name {
color: #e5e7eb;
}
.doc.doc-heading {
font-size: 0.9375rem;
font-weight: 600;
color: #1f2937;
}
[data-md-color-scheme="slate"] .doc.doc-heading {
color: #d1d5db;
}
.md-content code {
font-size: 0.75rem;
color: #dc2626;
background-color: #f3f4f6;
padding: 0.125rem 0.375rem;
border-radius: 0.25rem;
}
[data-md-color-scheme="slate"] .md-content code {
color: #fca5a5;
background-color: #1f2937;
}
.md-content pre code {
font-size: 0.75rem;
color: inherit;
background-color: transparent;
padding: 0;
}
.md-content a {
color: #2563eb;
text-decoration: none;
}
.md-content a:hover {
color: #1d4ed8;
text-decoration: underline;
}
[data-md-color-scheme="slate"] .md-content a {
color: #60a5fa;
}
[data-md-color-scheme="slate"] .md-content a:hover {
color: #93c5fd;
}
[data-md-color-scheme="slate"] {
--md-default-bg-color: #000000;
--md-default-fg-color: #ffffff;
--md-code-bg-color: #0a0a0a;
--md-code-fg-color: #ffffff;
}
[data-md-color-scheme="slate"] .md-header {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-tabs {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-footer {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-sidebar {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-nav {
background-color: #000000;
}
.highlight pre,
.highlight code {
background-color: #f6f8fa !important;
color: #24292e !important;
}
.highlight .k,
.highlight .kc,
.highlight .kd,
.highlight .kn,
.highlight .kp,
.highlight .kr,
.highlight .kt {
color: #d73a49 !important;
}
.highlight .s,
.highlight .s1,
.highlight .s2,
.highlight .sb,
.highlight .sc,
.highlight .sd,
.highlight .se,
.highlight .sh,
.highlight .si,
.highlight .sx,
.highlight .sr,
.highlight .ss {
color: #032f62 !important;
}
.highlight .nf,
.highlight .fm,
.highlight .nc {
color: #6f42c1 !important;
}
.highlight .m,
.highlight .mf,
.highlight .mh,
.highlight .mi,
.highlight .mo {
color: #005cc5 !important;
}
.highlight .c,
.highlight .c1,
.highlight .cm,
.highlight .cp,
.highlight .cs {
color: #6a737d !important;
font-style: italic;
}
.highlight .o,
.highlight .ow {
color: #d73a49 !important;
}
.highlight .n,
.highlight .na,
.highlight .nb,
.highlight .nd,
.highlight .ni,
.highlight .nl,
.highlight .nn,
.highlight .nx,
.highlight .py,
.highlight .nt,
.highlight .nv,
.highlight .bp,
.highlight .vc,
.highlight .vg,
.highlight .vi {
color: #e36209 !important;
}
[data-md-color-scheme="slate"] .highlight pre,
[data-md-color-scheme="slate"] .highlight code {
background-color: #0d1117 !important;
color: #e6edf3 !important;
}
[data-md-color-scheme="slate"] .highlight .k,
[data-md-color-scheme="slate"] .highlight .kc,
[data-md-color-scheme="slate"] .highlight .kd,
[data-md-color-scheme="slate"] .highlight .kn,
[data-md-color-scheme="slate"] .highlight .kp,
[data-md-color-scheme="slate"] .highlight .kr,
[data-md-color-scheme="slate"] .highlight .kt {
color: #ff7b72 !important;
}
[data-md-color-scheme="slate"] .highlight .s,
[data-md-color-scheme="slate"] .highlight .s1,
[data-md-color-scheme="slate"] .highlight .s2,
[data-md-color-scheme="slate"] .highlight .sb,
[data-md-color-scheme="slate"] .highlight .sc,
[data-md-color-scheme="slate"] .highlight .sd,
[data-md-color-scheme="slate"] .highlight .se,
[data-md-color-scheme="slate"] .highlight .sh,
[data-md-color-scheme="slate"] .highlight .si,
[data-md-color-scheme="slate"] .highlight .sx,
[data-md-color-scheme="slate"] .highlight .sr,
[data-md-color-scheme="slate"] .highlight .ss {
color: #a5d6ff !important;
}
[data-md-color-scheme="slate"] .highlight .nf,
[data-md-color-scheme="slate"] .highlight .fm,
[data-md-color-scheme="slate"] .highlight .nc {
color: #d2a8ff !important;
}
[data-md-color-scheme="slate"] .highlight .m,
[data-md-color-scheme="slate"] .highlight .mf,
[data-md-color-scheme="slate"] .highlight .mh,
[data-md-color-scheme="slate"] .highlight .mi,
[data-md-color-scheme="slate"] .highlight .mo {
color: #79c0ff !important;
}
[data-md-color-scheme="slate"] .highlight .c,
[data-md-color-scheme="slate"] .highlight .c1,
[data-md-color-scheme="slate"] .highlight .cm,
[data-md-color-scheme="slate"] .highlight .cp,
[data-md-color-scheme="slate"] .highlight .cs {
color: #8b949e !important;
font-style: italic;
}
[data-md-color-scheme="slate"] .highlight .o,
[data-md-color-scheme="slate"] .highlight .ow {
color: #ff7b72 !important;
}
[data-md-color-scheme="slate"] .highlight .n,
[data-md-color-scheme="slate"] .highlight .na,
[data-md-color-scheme="slate"] .highlight .nb,
[data-md-color-scheme="slate"] .highlight .nd,
[data-md-color-scheme="slate"] .highlight .ni,
[data-md-color-scheme="slate"] .highlight .nl,
[data-md-color-scheme="slate"] .highlight .nn,
[data-md-color-scheme="slate"] .highlight .nx,
[data-md-color-scheme="slate"] .highlight .py,
[data-md-color-scheme="slate"] .highlight .nt,
[data-md-color-scheme="slate"] .highlight .nv,
[data-md-color-scheme="slate"] .highlight .bp,
[data-md-color-scheme="slate"] .highlight .vc,
[data-md-color-scheme="slate"] .highlight .vg,
[data-md-color-scheme="slate"] .highlight .vi {
color: #ffa657 !important;
}
+21
View File
@@ -1,6 +1,27 @@
[tools]
python = "3.13.7"
uv = "0.9.11"
[settings]
experimental = true
python.uv_venv_auto = true
[tasks.install]
run= "uv sync --all-extras --all-groups"
[tasks.install_docs]
run= "uv sync --group docs"
[tasks.build_docs]
description = "Build static docs "
run = "uv run mkdocs build"
depends = ['install_docs']
[tasks.test]
run = "uv run pytest"
[env]
IS_PUBLIC = "false"
+130
View File
@@ -0,0 +1,130 @@
site_name: specklepy Docs
theme:
name: material
font:
text: Inter
favicon: assets/speckle_logo.png
logo: assets/speckle_logo.png
features:
- navigation.tabs
palette:
# Palette toggle for light mode
- scheme: default
primary: white
toggle:
icon: material/weather-night
name: Switch to dark mode
# Palette toggle for dark mode
- scheme: slate
primary: black
logo: assets/logo_white.png
toggle:
icon: material/weather-sunny
name: Switch to light mode
markdown_extensions:
- pymdownx.highlight:
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences
extra_css:
- stylesheets/extra.css
plugins:
- search
- mkdocstrings:
handlers:
python:
paths: [src]
options:
parameter_headings: false
members_order: source
separate_signature: true
filters: ["!^_"] #Ignore _ prefixed properties
docstring_options:
ignore_init_summary: true
merge_init_into_class: true
show_signature_annotations: true
signature_crossrefs: true
show_if_no_docstring: true
show_labels: true
show_source: true
show_symbol_type_heading: true
show_symbol_type_toc: true
show_bases: false
heading_level: 3
inventories:
- url: https://docs.python.org/3/objects.inv
domains: [py, std]
nav:
- Home: index.md
- specklepy SDK:
- API:
- Client: specklepy/api/client.md
- Credentials: specklepy/api/credentials.md
- Operations: specklepy/api/operations.md
- Resources:
- ActiveUserResource: specklepy/api/resources/ActiveUserResource.md
- FileImportResource: specklepy/api/resources/FileImportResource.md
- ModelResource: specklepy/api/resources/ModelResource.md
- OtherUserResource: specklepy/api/resources/OtherUserResource.md
- ProjectInviteResource: specklepy/api/resources/ProjectInviteResource.md
- ProjectResource: specklepy/api/resources/ProjectResource.md
- ServerResource: specklepy/api/resources/ServerResource.md
- SubscriptionResource: specklepy/api/resources/SubscriptionResource.md
- VersionResource: specklepy/api/resources/VersionResource.md
- WorkspaceResource: specklepy/api/resources/WorkspaceResource.md
- Objects:
- Base: specklepy/objects/base.md
- Data Objects: specklepy/objects/data_objects.md
- Geometry:
- Arc: specklepy/objects/geometry/Arc.md
- Box: specklepy/objects/geometry/Box.md
- Circle: specklepy/objects/geometry/Circle.md
- ControlPoint: specklepy/objects/geometry/ControlPoint.md
- Curve: specklepy/objects/geometry/Curve.md
- Ellipse: specklepy/objects/geometry/Ellipse.md
- Line: specklepy/objects/geometry/Line.md
- Mesh: specklepy/objects/geometry/Mesh.md
- Plane: specklepy/objects/geometry/Plane.md
- Point: specklepy/objects/geometry/Point.md
- PointCloud: specklepy/objects/geometry/PointCloud.md
- Polycurve: specklepy/objects/geometry/Polycurve.md
- Polyline: specklepy/objects/geometry/Polyline.md
- Spiral: specklepy/objects/geometry/Spiral.md
- Surface: specklepy/objects/geometry/Surface.md
- Vector: specklepy/objects/geometry/Vector.md
- Primitives:
- Interval: specklepy/objects/primitives/interval.md
- Other:
- RenderMaterial: specklepy/objects/other/render_material.md
- Collection: specklepy/objects/other/collection.md
- Proxies:
- ColorProxy: specklepy/objects/proxies/ColorProxy.md
- GroupProxy: specklepy/objects/proxies/GroupProxy.md
- InstanceProxy: specklepy/objects/proxies/InstanceProxy.md
- InstanceDefinitionProxy: specklepy/objects/proxies/InstanceDefinitionProxy.md
- LevelProxy: specklepy/objects/proxies/LevelProxy.md
- RenderMaterialProxy: specklepy/objects/proxies/RenderMaterialProxy.md
- Graph Traversal: specklepy/objects/graph_traversal/traversal.md
- Transports: specklepy/transports/transports.md
- Serialization: specklepy/serialization/serializer.md
- Core API:
- Models: specklepy/core/api/models/models.md
- Inputs: specklepy/core/api/inputs/inputs.md
- Enums: specklepy/core/api/enums.md
- Logging:
- Exceptions: specklepy/logging/exceptions.md
- Speckle Automate:
- AutomationContext: speckle_automate/automation_context.md
- Runner: speckle_automate/runner.md
- Schema: speckle_automate/schema.md
+3
View File
@@ -0,0 +1,3 @@
[build]
command = "mise run build_docs"
publish = "site"
+9 -1
View File
@@ -22,6 +22,7 @@ dependencies = [
speckleifc = ["ifcopenshell>=0.8.3.post2"]
[dependency-groups]
dev = [
"commitizen>=4.1.0",
"devtools>=0.12.2",
@@ -32,11 +33,18 @@ dev = [
"pytest-asyncio>=0.25.2",
"pytest-cov>=6.0.0",
"pytest-ordering>=0.6",
"ruff>=0.9.2",
"pytest_httpserver >=1.1.3",
"ruff==0.9.2",
"types-deprecated>=1.2.15.20241117",
"types-requests>=2.32.0.20241016",
"types-ujson>=5.10.0.20240515",
]
docs = [
"mkdocs>=1.6.1",
"mkdocs-material>=9.6.5",
"mkdocstrings>=0.28.1",
"mkdocstrings-python>=1.15.0",
]
[project.urls]
repository = "https://github.com/specklesystems/specklepy"
+54 -17
View File
@@ -19,8 +19,12 @@ from speckle_automate.schema import (
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.version_inputs import CreateVersionInput
from specklepy.core.api.inputs.version_inputs import (
CreateVersionInput,
MarkReceivedVersionInput,
)
from specklepy.core.api.models.current import Model, Version
from specklepy.logging import metrics
from specklepy.logging.exceptions import SpeckleException
from specklepy.objects.base import Base
from specklepy.transports.memory import MemoryTransport
@@ -66,6 +70,7 @@ class AutomationContext:
if isinstance(automation_run_data, AutomationRunData)
else AutomationRunData.model_validate_json(automation_run_data)
)
metrics.set_host_app("automate")
speckle_client = SpeckleClient(
automation_run_data.speckle_server_url,
automation_run_data.speckle_server_url.startswith("https"),
@@ -100,6 +105,7 @@ class AutomationContext:
"""Receive the Speckle project version that triggered this automation run."""
# TODO: this is a quick hack to keep implementation consistency.
# Move to proper receive many versions
project_id = self.automation_run_data.project_id
version_id = self.automation_run_data.triggers[0].payload.version_id
try:
version = self.speckle_client.version.get(
@@ -109,7 +115,7 @@ class AutomationContext:
raise ValueError(
f"""Could not receive specified version.
Is your environment configured correctly?
project_id: {self.automation_run_data.project_id}
project_id: {project_id}
model_id: {self.automation_run_data.triggers[0].payload.model_id}
version_id: {self.automation_run_data.triggers[0].payload.version_id}
"""
@@ -124,6 +130,13 @@ class AutomationContext:
base = operations.receive(
version.referenced_object, self._server_transport, self._memory_transport
)
self.speckle_client.version.received(
MarkReceivedVersionInput(
version_id=version_id,
project_id=project_id,
source_application="automate_function",
)
)
# self._closure_tree = base["__closure"]
print(
f"It took {self.elapsed():.2f} seconds to receive",
@@ -245,24 +258,24 @@ class AutomationContext:
"""
)
if self.run_status in [AutomationStatus.SUCCEEDED, AutomationStatus.FAILED]:
object_results = {
"version": 2,
results_dict = self._automation_result.model_dump(by_alias=True)
results = {
"version": 3,
"values": {
"objectResults": self._automation_result.model_dump(by_alias=True)[
"objectResults"
],
"objectResults": results_dict["objectResults"],
"versionResult": results_dict["versionResult"],
"blobIds": self._automation_result.blobs,
},
}
else:
object_results = None
results = None
params = {
"projectId": self.automation_run_data.project_id,
"functionRunId": self.automation_run_data.function_run_id,
"status": self.run_status.value,
"statusMessage": self._automation_result.status_message,
"results": object_results,
"results": results,
"contextView": self._automation_result.result_view,
}
print(f"Reporting run status with content: {params}")
@@ -312,25 +325,49 @@ class AutomationContext:
return upload_response.upload_results[0].blob_id
def mark_run_failed(self, status_message: str) -> None:
"""Mark the current run a failure."""
self._mark_run(AutomationStatus.FAILED, status_message)
def mark_run_failed(
self, status_message: str, version_result: dict[str, Any] | None = None
) -> None:
"""
Mark the current run a failure.
Args:
status_message: Optional message to be displayed.
version_result: Optional data object,
that will be attached to the run results.
The dictionary should be JSON serializable
"""
self._mark_run(AutomationStatus.FAILED, status_message, version_result)
def mark_run_exception(self, status_message: str) -> None:
"""Mark the current run a failure."""
self._mark_run(AutomationStatus.EXCEPTION, status_message)
self._mark_run(AutomationStatus.EXCEPTION, status_message, None)
def mark_run_success(self, status_message: Optional[str]) -> None:
"""Mark the current run a success with an optional message."""
self._mark_run(AutomationStatus.SUCCEEDED, status_message)
def mark_run_success(
self, status_message: str | None, version_result: dict[str, Any] | None = None
) -> None:
"""
Mark the current run a success with an optional message.
Args:
status_message: Optional message to be displayed.
version_result: Optional data object,
that will be attached to the run results.
The dictionary should be JSON serializable
"""
self._mark_run(AutomationStatus.SUCCEEDED, status_message, version_result)
def _mark_run(
self, status: AutomationStatus, status_message: Optional[str]
self,
status: AutomationStatus,
status_message: str | None,
version_result: dict[str, Any] | None,
) -> None:
duration = self.elapsed()
self._automation_result.status_message = status_message
self._automation_result.run_status = status
self._automation_result.elapsed = duration
self._automation_result.version_result = version_result
msg = f"Automation run {status.value} after {duration:.2f} seconds."
print("\n".join([msg, status_message]) if status_message else msg)
+5 -7
View File
@@ -88,10 +88,8 @@ def create_test_automation_run(
print(result)
return (
result.get("projectMutations")
.get("automationMutations")
.get("createTestAutomationRun")
return TestAutomationRunData.model_validate(
result["projectMutations"]["automationMutations"]["createTestAutomationRun"]
)
@@ -123,9 +121,9 @@ def create_test_automation_run_data(
project_id=test_automation_environment.project_id,
speckle_server_url=test_automation_environment.server_url,
automation_id=test_automation_environment.automation_id,
automation_run_id=test_automation_run_data["automationRunId"],
function_run_id=test_automation_run_data["functionRunId"],
triggers=test_automation_run_data["triggers"],
automation_run_id=test_automation_run_data.automation_run_id,
function_run_id=test_automation_run_data.function_run_id,
triggers=test_automation_run_data.triggers,
)
+12 -11
View File
@@ -1,7 +1,7 @@
""""""
from enum import Enum
from typing import Any, Dict, List, Literal, Optional
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field
from pydantic.alias_generators import to_camel
@@ -36,7 +36,7 @@ class AutomationRunData(BaseModel):
automation_run_id: str
function_run_id: str
triggers: List[VersionCreationTrigger]
triggers: list[VersionCreationTrigger]
model_config = ConfigDict(
alias_generator=to_camel, populate_by_name=True, protected_namespaces=()
@@ -49,7 +49,7 @@ class TestAutomationRunData(BaseModel):
automation_run_id: str
function_run_id: str
triggers: List[VersionCreationTrigger]
triggers: list[VersionCreationTrigger]
model_config = ConfigDict(
alias_generator=to_camel, populate_by_name=True, protected_namespaces=()
@@ -80,19 +80,20 @@ class ResultCase(AutomateBase):
category: str
level: ObjectResultLevel
object_app_ids: Dict[str, Optional[str]]
message: Optional[str]
metadata: Optional[Dict[str, Any]]
visual_overrides: Optional[Dict[str, Any]]
object_app_ids: dict[str, str | None]
message: str | None
metadata: dict[str, Any] | None
visual_overrides: dict[str, Any] | None
class AutomationResult(AutomateBase):
"""Schema accepted by the Speckle server as a result for an automation run."""
elapsed: float = 0
result_view: Optional[str] = None
result_versions: List[str] = Field(default_factory=list)
blobs: List[str] = Field(default_factory=list)
result_view: str | None = None
result_versions: list[str] = Field(default_factory=list)
blobs: list[str] = Field(default_factory=list)
run_status: AutomationStatus = AutomationStatus.RUNNING
status_message: Optional[str] = None
status_message: str | None = None
object_results: list[ResultCase] = Field(default_factory=list)
version_result: dict[str, Any] | None = None
+6 -3
View File
@@ -18,7 +18,7 @@ def cmd_line_import() -> None:
parser.add_argument("output_path")
parser.add_argument("project_id")
parser.add_argument("version_message")
parser.add_argument("model_id")
parser.add_argument("model_ingestion_id")
# parser.add_argument("model_name")
# parser.add_argument("region_name")
@@ -32,6 +32,8 @@ def cmd_line_import() -> None:
"ifc",
)
client: SpeckleClient | None = None
try:
client = SpeckleClient(SERVER_URL, use_ssl=not SERVER_URL.startswith("http://"))
client.authenticate_with_token(TOKEN)
@@ -41,13 +43,14 @@ def cmd_line_import() -> None:
args.file_path,
project,
args.version_message,
args.model_id,
args.model_ingestion_id,
client,
)
with open(args.output_path, "w") as f:
json.dump({"success": True, "commitId": version.id}, f)
except Exception as e:
error_msg = f"IFC Importer failed with exception:\n{traceback.format_exc()}"
stack_trace = traceback.format_exc()
error_msg = f"IFC Importer failed with exception:\n{stack_trace}"
print(error_msg)
# Write error result
+100 -34
View File
@@ -1,9 +1,19 @@
import contextlib
import importlib.metadata
import time
import traceback
from pathlib import Path
from speckleifc.ifc_geometry_processing import open_ifc
from speckleifc.importer import ImportJob
from specklepy.core.api.client import SpeckleClient
from specklepy.core.api.inputs.version_inputs import CreateVersionInput
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionFailedInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
SourceDataInput,
)
from specklepy.core.api.models.current import Project, Version
from specklepy.core.api.operations import send
from specklepy.logging import metrics
@@ -13,49 +23,105 @@ from specklepy.transports.server import ServerTransport
def open_and_convert_file(
file_path: str,
project: Project,
version_message: str | None,
model_id: str,
model_ingestion_id: str,
client: SpeckleClient,
) -> Version:
start = time.time()
very_start = start
try:
start = time.time()
very_start = start
path = Path(file_path)
account = client.account
server_url = account.serverInfo.url
assert server_url
remote_transport = ServerTransport(project.id, account=account)
specklepy_version = importlib.metadata.version("specklepy")
client.model_ingestion.start_processing(
ModelIngestionStartProcessingInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
progress_message="Importing IFC file",
source_data=SourceDataInput(
file_name=path.name,
file_size_bytes=path.stat().st_size,
source_application_slug=metrics.HOST_APP,
source_application_version=specklepy_version,
),
)
)
ifc_file = open_ifc(file_path) # pyright: ignore[reportUnknownVariableType]
account = client.account
server_url = account.serverInfo.url
assert server_url
remote_transport = ServerTransport(project.id, account=account)
import_job = ImportJob(ifc_file) # pyright: ignore[reportUnknownArgumentType]
data = import_job.convert()
ifc_file = open_ifc(file_path) # pyright: ignore[reportUnknownVariableType]
print(f"File conversion complete after {(time.time() - start) * 1000}ms")
client.model_ingestion.update_progress(
ModelIngestionUpdateInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
progress_message="Converting file",
progress=None,
)
)
import_job = ImportJob(ifc_file) # pyright: ignore[reportUnknownArgumentType]
data = import_job.convert()
start = time.time()
print(f"File conversion complete after {(time.time() - start) * 1000}ms")
root_id = send(data, transports=[remote_transport], use_default_cache=False)
print(f"Sending to speckle complete after: {(time.time() - start) * 1000}ms")
start = time.time()
start = time.time()
client.model_ingestion.update_progress(
ModelIngestionUpdateInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
progress_message="Uploading objects",
progress=None,
)
)
root_id = send(data, transports=[remote_transport], use_default_cache=False)
print(f"Sending to speckle complete after: {(time.time() - start) * 1000}ms")
create_version = CreateVersionInput(
object_id=root_id,
model_id=model_id,
project_id=project.id,
message=version_message,
source_application="ifc",
)
version = client.version.create(create_version)
end = time.time()
print(f"Version committed after: {(end - start) * 1000}ms")
start = time.time()
print(f"Total time (to commit): {(end - very_start) * 1000}ms")
del ifc_file
version_id = client.model_ingestion.complete(
ModelIngestionSuccessInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
root_object_id=root_id,
# version_message=version_message,
)
)
custom_properties = {"ui": "dui3", "actionSource": "import"}
if project.workspace_id:
custom_properties["workspace_id"] = project.workspace_id
metrics.track(metrics.SEND, account, custom_properties, send_sync=True)
# needed to query version until ingestion api expands to serve it
version = client.version.get(version_id, project.id)
return version
end = time.time()
print(f"Version committed after: {(end - start) * 1000}ms")
print(f"Total time (to commit): {(end - very_start) * 1000}ms")
del ifc_file
custom_properties = {"ui": "dui3", "actionSource": "import"}
if project.workspace_id:
custom_properties["workspace_id"] = project.workspace_id
metrics.track(
metrics.SEND,
account,
custom_properties,
send_sync=True,
track_email=True,
)
return version
except Exception as e:
stack_trace = traceback.format_exc()
with contextlib.suppress(Exception):
# make sure to not report process kills when we're cancelling
client.model_ingestion.fail_with_error(
ModelIngestionFailedInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
error_reason=str(e),
error_stacktrace=stack_trace,
)
)
raise e
+7
View File
@@ -4,6 +4,7 @@ from specklepy.api.credentials import Account
from specklepy.api.resources import (
ActiveUserResource,
FileImportResource,
ModelIngestionResource,
ModelResource,
OtherUserResource,
ProjectInviteResource,
@@ -119,6 +120,12 @@ class SpeckleClient(CoreSpeckleClient):
client=self.httpclient,
server_version=server_version,
)
self.model_ingestion = ModelIngestionResource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.file_import = FileImportResource(
account=self.account,
basepath=self.url,
+5
View File
@@ -1,5 +1,8 @@
from specklepy.api.resources.current.active_user_resource import ActiveUserResource
from specklepy.api.resources.current.file_import_resource import FileImportResource
from specklepy.api.resources.current.model_ingestion_resource import (
ModelIngestionResource,
)
from specklepy.api.resources.current.model_resource import ModelResource
from specklepy.api.resources.current.other_user_resource import OtherUserResource
from specklepy.api.resources.current.project_invite_resource import (
@@ -22,4 +25,6 @@ __all__ = [
"SubscriptionResource",
"VersionResource",
"WorkspaceResource",
"FileImportResource",
"ModelIngestionResource",
]
@@ -15,7 +15,7 @@ from specklepy.logging import metrics
class FileImportResource(CoreResource):
"""API Access class for projects"""
"""API Access class for file imports"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
@@ -0,0 +1,53 @@
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
)
from specklepy.core.api.models.current import (
ModelIngestion,
)
from specklepy.core.api.resources import (
ModelIngestionResource as CoreResource,
)
from specklepy.logging import metrics
class ModelIngestionResource(CoreResource):
"""API Access class for model ingestion"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(account, basepath, client, server_version)
def create(self, input: ModelIngestionCreateInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Create"})
return super().create(input)
def update_progress(self, input: ModelIngestionUpdateInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Update"})
return super().update_progress(input)
def start_processing(
self, input: ModelIngestionStartProcessingInput
) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Start Processing"})
return super().start_processing(input)
def requeue(self, input: ModelIngestionRequeueInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Update"})
return super().requeue(input)
def complete(self, input: ModelIngestionSuccessInput) -> str:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion End"})
return super().complete(input)
def fail_with_error(self, input: ModelIngestionFailedInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Error"})
return super().fail_with_error(input)
def fail_with_cancel(self, input: ModelIngestionCancelledInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Cancel"})
return super().fail_with_cancel(input)
+7
View File
@@ -12,6 +12,7 @@ from specklepy.core.api.credentials import Account
from specklepy.core.api.resources import (
ActiveUserResource,
FileImportResource,
ModelIngestionResource,
ModelResource,
OtherUserResource,
ProjectInviteResource,
@@ -250,6 +251,12 @@ class SpeckleClient:
client=self.httpclient,
server_version=server_version,
)
self.model_ingestion = ModelIngestionResource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.subscription = SubscriptionResource(
account=self.account,
basepath=self.ws_url,
+15
View File
@@ -30,3 +30,18 @@ class ProjectVersionsUpdatedMessageType(str, Enum):
CREATED = "CREATED"
DELETED = "DELETED"
UPDATED = "UPDATED"
class ProjectModelIngestionUpdatedMessageType(str, Enum):
CANCELLATION_REQUESTED = "cancellationRequested"
CREATED = "created"
DELETED = "deleted"
UPDATED = "updated"
class ModelIngestionStatus(str, Enum):
CANCELLED = "cancelled"
FAILED = "failed"
PROCESSING = "processing"
QUEUED = "queued"
SUCCESS = "success"
@@ -0,0 +1,76 @@
from specklepy.core.api.enums import ProjectModelIngestionUpdatedMessageType
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
class SourceDataInput(GraphQLBaseModel):
source_application_slug: str
source_application_version: str
file_name: str | None
file_size_bytes: int | None
class ModelIngestionCreateInput(GraphQLBaseModel):
model_id: str
project_id: str
progress_message: str
source_data: SourceDataInput
class ModelIngestionStartProcessingInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress_message: str
source_data: SourceDataInput
class ModelIngestionRequeueInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress_message: str
class ModelIngestionUpdateInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress: float | None
progress_message: str
class ModelIngestionSuccessInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
root_object_id: str
class ModelIngestionFailedInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
error_reason: str
error_stacktrace: str | None
class ModelIngestionCancelledInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
cancellation_message: str
class ModelIngestionRequestCancellationInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
cancellation_message: str
class ModelIngestionReference(GraphQLBaseModel):
"""
`@oneOf` i.e. server expects **either** `ingestion_id` or `model_id`, but not both.
"""
ingestion_id: str | None
model_id: str | None
class ProjectModelIngestionSubscriptionInput(GraphQLBaseModel):
project_id: str
ingestion_reference: ModelIngestionReference
message_type: ProjectModelIngestionUpdatedMessageType | None = None
+15 -1
View File
@@ -1,7 +1,7 @@
from datetime import datetime
from typing import Generic, List, TypeVar
from specklepy.core.api.enums import ProjectVisibility
from specklepy.core.api.enums import ModelIngestionStatus, ProjectVisibility
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
from specklepy.logging.exceptions import WorkspacePermissionException
@@ -244,3 +244,17 @@ class FileImport(GraphQLBaseModel):
class FileUploadUrl(GraphQLBaseModel):
url: str
file_id: str
class ModelIngestionStatusData(GraphQLBaseModel):
status: ModelIngestionStatus
progress_message: str | None = None
class ModelIngestion(GraphQLBaseModel):
id: str
created_at: datetime
updated_at: datetime
cancellation_requested: bool
model_id: str
status_data: ModelIngestionStatusData
@@ -1,12 +1,13 @@
from typing import Optional
from specklepy.core.api.enums import (
ProjectModelIngestionUpdatedMessageType,
ProjectModelsUpdatedMessageType,
ProjectUpdatedMessageType,
ProjectVersionsUpdatedMessageType,
UserProjectsUpdatedMessageType,
)
from specklepy.core.api.models.current import Model, Project, Version
from specklepy.core.api.models.current import Model, ModelIngestion, Project, Version
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
@@ -33,3 +34,8 @@ class ProjectVersionsUpdatedMessage(GraphQLBaseModel):
type: ProjectVersionsUpdatedMessageType
model_id: str
version: Optional[Version]
class ProjectModelIngestionUpdatedMessage(GraphQLBaseModel):
model_ingestion: ModelIngestion
type: ProjectModelIngestionUpdatedMessageType
@@ -1,5 +1,8 @@
from specklepy.core.api.resources.current.active_user_resource import ActiveUserResource
from specklepy.core.api.resources.current.file_import_resource import FileImportResource
from specklepy.core.api.resources.current.model_ingestion_resource import (
ModelIngestionResource,
)
from specklepy.core.api.resources.current.model_resource import ModelResource
from specklepy.core.api.resources.current.other_user_resource import OtherUserResource
from specklepy.core.api.resources.current.project_invite_resource import (
@@ -24,4 +27,6 @@ __all__ = [
"SubscriptionResource",
"VersionResource",
"WorkspaceResource",
"FileImportResource",
"ModelIngestionResource",
]
@@ -16,13 +16,15 @@ from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
from specklepy.logging.exceptions import SpeckleException
NAME = "file_import"
class UploadFileResponse(GraphQLBaseModel):
etag: str
class FileImportResource(ResourceBase):
"""API Access class for project invites"""
"""API Access class for file imports"""
def __init__(
self,
@@ -36,7 +38,7 @@ class FileImportResource(ResourceBase):
basepath=basepath,
client=client,
server_version=server_version,
name="file-import",
name=NAME,
)
def finish_file_import_job(self, input: FinishFileImportInput) -> bool:
@@ -0,0 +1,398 @@
from typing import Any, Optional, Tuple
from gql import Client, gql
from specklepy.api.credentials import Account
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequestCancellationInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
)
from specklepy.core.api.models.current import (
ModelIngestion,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
NAME = "ingestion"
class ModelIngestionResource(ResourceBase):
"""API Access class for model ingestion"""
def __init__(
self,
account: Account,
basepath: str,
client: Client,
server_version: Optional[Tuple[Any, ...]],
) -> None:
super().__init__(
account=account,
basepath=basepath,
client=client,
name=NAME,
server_version=server_version,
)
def get_ingestion(self, project_id: str, model_id: str) -> ModelIngestion:
QUERY = gql(
"""
query Query($projectId: String!, $modelId: String!) {
data:project(id: $projectId) {
data:model(id: $modelId) {
data:ingestion {
id
createdAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"projectId": project_id,
"modelId": model_id,
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
def create(self, input: ModelIngestionCreateInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionCreate($input: ModelIngestionCreateInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: create(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def start_processing(
self, input: ModelIngestionStartProcessingInput
) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionStartProcessing($input: ModelIngestionStartProcessingInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: startProcessing(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def requeue(self, input: ModelIngestionRequeueInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionRequeue($input: ModelIngestionRequeueInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: requeue(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def update_progress(self, input: ModelIngestionUpdateInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionUpdateProgress(
$input: ModelIngestionUpdateInput!
) {
data: projectMutations {
data: modelIngestionMutations {
data: updateProgress(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def complete(self, input: ModelIngestionSuccessInput) -> str:
"""
Request that the server completes the ingestion by creating a version
If successful, the job will be in a terminal "successful" state.
For failed Ingestions, use `fail_with_error` instead
For user cancellation, use `fail_with_cancelled` instead
Arguments:
input {ModelIngestionSuccessInput} -- input variable
Returns:
str -- the id of the version that was just created to complete the ingestion
"""
QUERY = gql(
"""
mutation IngestionComplete($input: ModelIngestionSuccessInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: completeWithVersion(input: $input) {
data:statusData {
... on ModelIngestionSuccessStatus {
data:versionId
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[DataResponse[DataResponse[str]]]]],
QUERY,
variables,
).data.data.data.data.data
def fail_with_error(self, input: ModelIngestionFailedInput) -> ModelIngestion:
"""
Fail the job with an error.
For user requested cancellation, use `fail_with_cancelled` instead
"""
QUERY = gql(
"""
mutation IngestionFailWithError($input: ModelIngestionFailedInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: failWithError(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
def fail_with_cancel(self, input: ModelIngestionCancelledInput) -> ModelIngestion:
"""
Fail the ingestion with a `cancelled` status.
This should only be done if the user has explicitly requested cancellation
Other forms of cancellation use `fail_with_error`
The ingestion should then enter a terminal "canceled" state
"""
QUERY = gql(
"""
mutation IngestionFailWithCancel($input: ModelIngestionCancelledInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: failWithCancel(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
def request_cancellation(
self, input: ModelIngestionRequestCancellationInput
) -> ModelIngestion:
"""
Request that the ingestion is canceled.
Note: simply calling this mutation does not immediately cancel,
it doesn't even guarantee it will be canceled at all.
It's up to the client to observe this cancellation request
via `subscription.project_model_ingestion_cancellation_requested`
and report it as cancelled (via `ingestion.fail_with_cancel`
See "cooperative cancellation pattern"
"""
QUERY = gql(
"""
mutation IngestionRequestCancellation($input: ModelIngestionRequestCancellationInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: requestCancellation (input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
@@ -6,17 +6,25 @@ from graphql import DocumentNode
from pydantic import BaseModel
from typing_extensions import TypeVar
from specklepy.core.api.enums import ProjectModelIngestionUpdatedMessageType
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionReference,
ProjectModelIngestionSubscriptionInput,
)
from specklepy.core.api.models import (
ProjectModelsUpdatedMessage,
ProjectUpdatedMessage,
ProjectVersionsUpdatedMessage,
UserProjectsUpdatedMessage,
)
from specklepy.core.api.models.subscription_messages import (
ProjectModelIngestionUpdatedMessage,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
from specklepy.logging.exceptions import SpeckleException
NAME = "subscribe"
NAME = "subscription"
TEventArgs = TypeVar("TEventArgs", bound=BaseModel)
@@ -202,6 +210,66 @@ class SubscriptionResource(ResourceBase):
callback=lambda d: callback(d.data),
)
async def project_model_ingestion_updated(
self,
callback: Callable[[ProjectModelIngestionUpdatedMessage], None],
input: ProjectModelIngestionSubscriptionInput,
) -> None:
QUERY = gql(
"""
subscription IngestionUpdated($input: ProjectModelIngestionSubscriptionInput!) {
data: projectModelIngestionUpdated(input: $input) {
modelIngestion {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
type
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(
warnings="error", by_alias=True, exclude_none=True
),
}
await self.subscribe_2(
DataResponse[ProjectModelIngestionUpdatedMessage],
QUERY,
variables,
callback=lambda d: callback(d.data),
)
async def project_model_ingestion_cancellation_requested(
self,
callback: Callable[[ProjectModelIngestionUpdatedMessage], None],
project_id: str,
ingestion_id: str,
) -> None:
await self.project_model_ingestion_updated(
callback,
ProjectModelIngestionSubscriptionInput(
project_id=project_id,
ingestion_reference=ModelIngestionReference(
ingestion_id=ingestion_id, model_id=None
),
message_type=ProjectModelIngestionUpdatedMessageType.CANCELLATION_REQUESTED,
),
)
@check_wsclient
async def subscribe_2(
self,
@@ -14,7 +14,7 @@ from specklepy.core.api.models import ResourceCollection, Version
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
NAME = "model"
NAME = "version"
class VersionResource(ResourceBase):
@@ -16,7 +16,7 @@ NAME = "workspace"
class WorkspaceResource(ResourceBase):
"""API Access class for models"""
"""API Access class for workspaces"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
+41 -7
View File
@@ -1,19 +1,20 @@
import contextlib
import getpass
import hashlib
import importlib.metadata
import logging
import platform
import queue
import sys
import threading
from typing import Any
from typing import Any, Literal
import requests
from specklepy.core.api.credentials import Account
"""
Anonymous telemetry to help us understand how to make a better Speckle.
Lightweight usage telemetry to help us understand how to make a better Speckle.
This really helps us to deliver a better open source project and product!
"""
TRACK = True
@@ -22,13 +23,14 @@ HOST_APP_VERSION = f"python {'.'.join(map(str, sys.version_info[:2]))}"
PLATFORMS = {"win32": "Windows", "cygwin": "Windows", "darwin": "Mac OS X"}
LOG = logging.getLogger(__name__)
METRICS_TRACKER = None
METRICS_TRACKER: "MetricsTracker | None" = None
# actions
SDK = "SDK Action"
CONNECTOR = "Connector Action"
RECEIVE = "Receive"
SEND = "Send"
ACTIONS = Literal["SDK Action", "Connector Action", "Receive", "Send"]
def disable():
@@ -48,15 +50,32 @@ def set_host_app(host_app: str, host_app_version: str | None = None):
def track(
action: str,
action: ACTIONS,
account: Account | None = None,
custom_props: dict | None = None,
send_sync: bool = False,
track_email: bool = False,
):
"""
:param action:
:type action: ACTIONS
:param account:
:type account: Account | None
:param custom_props:
:type custom_props: dict | None
:param send_sync: When `True`, the track event is executed synchronously,
and any exceptions will be raised.
When `False`, the track it is deferred to a queue, and any exceptions will be
swallowed and reported as warnings.
:type send_sync: bool
:param track_email: When `True`, the users plain text email address will be included
:type track_email: bool
"""
if not TRACK:
return
tracker = initialise_tracker(account)
event_params: dict[str, Any] = {
"event": action,
"properties": {
@@ -72,6 +91,18 @@ def track(
if custom_props:
event_params["properties"].update(custom_props)
if track_email:
event_params["properties"]["email"] = tracker.last_email
try:
specklepy_version = importlib.metadata.version("specklepy")
event_params["properties"]["core_version"] = specklepy_version
except importlib.metadata.PackageNotFoundError:
if send_sync:
raise
else:
LOG.warning("Failed to read specklepy's version number", exc_info=True)
if send_sync:
tracker.send_event(event_params)
else:
@@ -84,7 +115,7 @@ def initialise_tracker(account: Account | None = None) -> "MetricsTracker":
METRICS_TRACKER = MetricsTracker()
if account:
METRICS_TRACKER.set_last_user(account.userInfo.email)
METRICS_TRACKER.set_last_user_email(account.userInfo.email)
METRICS_TRACKER.set_last_server(account.serverInfo.url)
return METRICS_TRACKER
@@ -103,6 +134,7 @@ class MetricsTracker(metaclass=Singleton):
analytics_url: str = "https://analytics.speckle.systems/track?ip=1"
analytics_token: str = "acd87c5a50b56df91a795e999812a3a4"
last_user: str = ""
last_email: str = ""
last_server: str | None = None
platform: str
@@ -121,17 +153,19 @@ class MetricsTracker(metaclass=Singleton):
if node and user:
self.last_user = f"@{self.hash(f'{node}-{user}')}"
def set_last_user(self, email: str | None) -> None:
def set_last_user_email(self, email: str | None) -> None:
if not email:
return
self.last_user = f"@{self.hash(email)}"
self.last_email = email
def set_last_server(self, server: str | None) -> None:
if not server:
return
self.last_server = self.hash(server)
def hash(self, value: str) -> str:
@staticmethod
def hash(value: str) -> str:
inputList = value.lower().split("://")
input = inputList[len(inputList) - 1].split("/")[0].split("?")[0]
return hashlib.md5(input.encode("utf-8")).hexdigest().upper()
+24
View File
@@ -323,6 +323,30 @@ def _validate_type(t: Optional[type], value: Any) -> Tuple[bool, Any]:
@dataclass(kw_only=True)
class Base(_RegisteringBase, speckle_type="Base"):
"""Base class for all Speckle objects.
The base object class is the foundation of all data being
transferred with Speckle. Any custom data structure that you want to transfer via
Speckle should inherit from it.
Objects in Speckle are immutable for storage purposes. When any property changes,
the object gets a new identity (hash). This hash is stored in the `id` property
after serialization.
Attributes:
id: Unique identifier (hash) for the object. This is typically
set automatically during serialization and depends on the object's properties.
applicationId: Optional identifier for the application that created
this object, can store the host application's native object ID.
```py title="Example"
from specklepy.objects.base import Base
obj = Base(id="some-id", applicationId="my-app")
obj["custom_prop"] = 42 # Add a dynamic property
obj["@detached_prop"] = another_object # Add a detached property
```
"""
id: Union[str, None] = None
# totalChildrenCount: Union[int, None] = None
applicationId: Union[str, None] = None
+4
View File
@@ -18,6 +18,10 @@ class DataObject(
speckle_type="Objects.Data.DataObject",
detachable={"displayValue"},
):
"""
A generic data object that can hold arbitrary properties and display values.
"""
name: str
properties: Dict[str, object]
displayValue: List[Base]
+39
View File
@@ -9,6 +9,30 @@ from specklepy.objects.interfaces import ICurve, IHasUnits
@dataclass(kw_only=True)
class Arc(Base, IHasUnits, ICurve, speckle_type="Objects.Geometry.Arc"):
"""
An arc defined by a plane, start point, mid point and end point.
This class represents a circular arc in 3D space, defined by three points
and a plane. The arc is a portion of a circle that lies on the specified plane.
Attributes:
plane: The plane on which the arc lies
startPoint: The starting point of the arc
midPoint: A point on the arc between the start and end points
endPoint: The ending point of the arc.
```py title="Example"
from specklepy.objects.geometry.plane import Plane
from specklepy.objects.geometry.point import Point
plane = Plane(origin=Point(0, 0, 0), normal=Point(0, 0, 1))
start = Point(1, 0, 0)
mid = Point(0.7071, 0.7071, 0)
end = Point(0, 1, 0)
arc = Arc(plane=plane, startPoint=start, midPoint=mid, endPoint=end)
```
"""
plane: Plane
startPoint: Point
midPoint: Point
@@ -16,10 +40,20 @@ class Arc(Base, IHasUnits, ICurve, speckle_type="Objects.Geometry.Arc"):
@property
def radius(self) -> float:
"""Calculates the radius of the arc.
Returns:
The radius of the arc, as the distance from the start point to the origin.
"""
return self.startPoint.distance_to(self.plane.origin)
@property
def length(self) -> float:
"""Calculates the length of the arc.
Returns:
The length of the arc.
"""
start_to_mid = self.startPoint.distance_to(self.midPoint)
mid_to_end = self.midPoint.distance_to(self.endPoint)
r = self.radius
@@ -30,6 +64,11 @@ class Arc(Base, IHasUnits, ICurve, speckle_type="Objects.Geometry.Arc"):
@property
def measure(self) -> float:
"""Calculates the angular measure of the arc in radians.
Returns:
The angular measure of the arc in radians.
"""
start_to_mid = self.startPoint.distance_to(self.midPoint)
mid_to_end = self.midPoint.distance_to(self.endPoint)
r = self.radius
+33 -1
View File
@@ -9,7 +9,29 @@ from specklepy.objects.primitive import Interval
@dataclass(kw_only=True)
class Box(Base, IHasUnits, IHasArea, IHasVolume, speckle_type="Objects.Geometry.Box"):
"""
a 3-dimensional box oriented on a plane
A 3-dimensional box oriented on a plane.
This class represents a rectangular prism in 3D space, defined by a base plane and
three intervals specifying its dimensions along the x, y, and z axes.
Attributes:
basePlane: The plane on which the box is oriented
xSize: The interval defining the box's size along the x-axis
ySize: The interval defining the box's size along the y-axis
zSize: The interval defining the box's size along the z-axis
```py title="Example"
from specklepy.objects.geometry.plane import Plane
from specklepy.objects.geometry.point import Point
from specklepy.objects.primitive import Interval
base_plane = Plane(origin=Point(0, 0, 0), normal=Point(0, 0, 1))
x_size = Interval(start=0, end=10)
y_size = Interval(start=0, end=5)
z_size = Interval(start=0, end=3)
box = Box(basePlane=base_plane, xSize=x_size, ySize=y_size, zSize=z_size)
```
"""
basePlane: Plane
@@ -29,6 +51,11 @@ class Box(Base, IHasUnits, IHasArea, IHasVolume, speckle_type="Objects.Geometry.
@property
def area(self) -> float:
"""Calculates the surface area of the box.
Returns:
The total surface area of the box.
"""
return 2 * (
self.xSize.length * self.ySize.length
+ self.xSize.length * self.zSize.length
@@ -37,4 +64,9 @@ class Box(Base, IHasUnits, IHasArea, IHasVolume, speckle_type="Objects.Geometry.
@property
def volume(self) -> float:
"""Calculates the volume of the box.
Returns:
The volume of the box.
"""
return self.xSize.length * self.ySize.length * self.zSize.length
@@ -0,0 +1,13 @@
from specklepy.objects.graph_traversal.traversal import (
DefaultRule,
GraphTraversal,
TraversalContext,
TraversalRule,
)
__all__ = [
"GraphTraversal",
"TraversalContext",
"TraversalRule",
"DefaultRule",
]
@@ -23,6 +23,9 @@ class ITraversalRule(Protocol):
@final
@define(slots=True, frozen=True)
class DefaultRule:
def should_return(self) -> bool:
return False
def get_members_to_traverse(self, _) -> Set[str]:
return set()
@@ -47,6 +50,15 @@ class TraversalContext:
class GraphTraversal:
_rules: List[ITraversalRule]
def _get_active_rule(self, o: Base) -> Optional[ITraversalRule]:
for rule in self._rules:
if rule.does_rule_hold(o):
return rule
return None
def _get_active_rule_or_default_rule(self, o: Base) -> ITraversalRule:
return self._get_active_rule(o) or _default_rule
def traverse(self, root: Base) -> Iterator[TraversalContext]:
stack: List[TraversalContext] = []
@@ -110,15 +122,6 @@ class GraphTraversal:
for o in GraphTraversal.traverse_member(obj):
yield o
def _get_active_rule_or_default_rule(self, o: Base) -> ITraversalRule:
return self._get_active_rule(o) or _default_rule
def _get_active_rule(self, o: Base) -> Optional[ITraversalRule]:
for rule in self._rules:
if rule.does_rule_hold(o):
return rule
return None
@final
@define(slots=True, frozen=True)
@@ -21,7 +21,7 @@ def setup_session(auth_token: str | None) -> requests.Session:
read=3,
connect=3,
backoff_factor=0.5,
status_forcelist=(500, 502, 503, 504, 408, 429),
status_forcelist=(500, 502, 503, 504, 520, 408, 429),
allowed_methods=["HEAD", "GET", "OPTIONS", "POST", "PUT", "DELETE"],
raise_on_status=False,
)
@@ -0,0 +1,268 @@
from datetime import datetime
import pytest
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.core.api.enums import ModelIngestionStatus
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
SourceDataInput,
)
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.project_inputs import ProjectCreateInput
from specklepy.core.api.models.current import (
Model,
ModelIngestion,
ModelIngestionStatusData,
Project,
ProjectVisibility,
Version,
)
from specklepy.logging.exceptions import GraphQLException
from specklepy.objects.base import Base
from specklepy.transports.server.server import ServerTransport
from tests.integration.conftest import is_public
@pytest.mark.run()
@pytest.mark.skipif(is_public(), reason="The public API does not support these tests")
class TestIngestionResource:
@pytest.fixture
def project(self, client: SpeckleClient):
return client.project.create(
ProjectCreateInput(
name="test", description=None, visibility=ProjectVisibility.PUBLIC
)
)
@pytest.fixture
def model(self, client: SpeckleClient, project: Project):
return client.model.create(
CreateModelInput(name="test", description=None, project_id=project.id)
)
@pytest.fixture()
def ingestion(
self, client: SpeckleClient, model: Model, project: Project
) -> ModelIngestion:
input = ModelIngestionCreateInput(
model_id=model.id,
project_id=project.id,
progress_message="Starting processing",
source_data=SourceDataInput(
source_application_slug="pytest",
source_application_version="0.0.0",
file_name=None,
file_size_bytes=None,
),
)
ingestion = client.model_ingestion.create(input)
assert isinstance(ingestion, ModelIngestion)
assert isinstance(ingestion.id, str)
assert isinstance(ingestion.created_at, datetime)
assert isinstance(ingestion.updated_at, datetime)
assert isinstance(ingestion.cancellation_requested, bool)
assert isinstance(ingestion.model_id, str)
assert isinstance(ingestion.status_data, ModelIngestionStatusData)
assert isinstance(ingestion.status_data.progress_message, str | None)
assert ingestion.status_data.status == ModelIngestionStatus.PROCESSING
assert not ingestion.cancellation_requested
assert ingestion.model_id == model.id
return ingestion
def test_update_progress(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
def update(progress: float | None, message: str):
input = ModelIngestionUpdateInput(
ingestion_id=ingestion.id,
project_id=project.id,
progress=progress,
progress_message=message,
)
res = client.model_ingestion.update_progress(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.PROCESSING
update(None, "None")
update(0.1, "0.1")
update(0.5, "Whoa-oh! We're half way there!")
update(1, "Finished")
update(0.2, "Back to processing again")
def test_start_processing_and_requeue(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
# just setting the baseline state
assert ingestion.status_data.status == ModelIngestionStatus.PROCESSING
def requeue(message: str):
input = ModelIngestionRequeueInput(
project_id=project.id,
ingestion_id=ingestion.id,
progress_message=message,
)
res = client.model_ingestion.requeue(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.QUEUED
def start_processing(message: str):
input = ModelIngestionStartProcessingInput(
ingestion_id=ingestion.id,
project_id=project.id,
progress_message=message,
source_data=SourceDataInput(
source_application_slug="test",
source_application_version="test",
file_name="test",
file_size_bytes=1,
),
)
res = client.model_ingestion.start_processing(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.PROCESSING
requeue("put it back in there")
start_processing("go for it")
requeue("and again")
start_processing("run forest run")
def update(progress: float | None, message: str):
input = ModelIngestionUpdateInput(
ingestion_id=ingestion.id,
project_id=project.id,
progress=progress,
progress_message=message,
)
res = client.model_ingestion.update_progress(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.PROCESSING
update(None, "None")
update(0.1, "0.1")
update(0.5, "Whoa-oh! We're half way there!")
update(1, "Finished")
update(0.2, "Back to processing again")
def test_error(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
input = ModelIngestionFailedInput(
ingestion_id=ingestion.id,
project_id=project.id,
error_reason="Failed to integration test an error",
error_stacktrace="over here in test_error",
)
res = client.model_ingestion.fail_with_error(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message is None
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.FAILED
# trying to fail for a second time should throw
# with pytest.raises(GraphQLException):
# _ = client.ingestion.fail_with_error(input)
def test_complete(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
remote = ServerTransport(project.id, client)
object_id = operations.send(
Base(applicationId="ASDFGHJKL"), [remote], use_default_cache=False
)
input = ModelIngestionSuccessInput(
ingestion_id=ingestion.id,
root_object_id=object_id,
project_id=project.id,
# version_message=None,
)
res = client.model_ingestion.complete(input)
assert isinstance(res, str)
version = client.version.get(res, project.id)
assert isinstance(version, Version)
# trying to complete for a second time should throw
# with pytest.raises(GraphQLException):
# _ = client.ingestion.complete(input)
def test_cancel(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
input = ModelIngestionCancelledInput(
ingestion_id=ingestion.id,
project_id=project.id,
cancellation_message="This was cancelled for testing purposes",
)
res = client.model_ingestion.fail_with_cancel(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message is None
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.CANCELLED
# Cancel again, should be idempotent
res = client.model_ingestion.fail_with_cancel(input)
assert res.status_data.progress_message is None
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.CANCELLED
def test_error_non_existent_ingestion(
self, client: SpeckleClient, project: Project
):
input = ModelIngestionFailedInput(
ingestion_id="Non-existent-ingestion",
project_id=project.id,
error_reason="Failed to integration test an error",
error_stacktrace="over here in test_error",
)
with pytest.raises(GraphQLException):
_ = client.model_ingestion.fail_with_error(input)
def test_complete_failed_non_existent_ingestion(
self, client: SpeckleClient, project: Project
):
input = ModelIngestionFailedInput(
ingestion_id="Non-existent-ingestion",
project_id=project.id,
error_reason="Failed to integration test an error",
error_stacktrace="over here in test_error",
)
with pytest.raises(GraphQLException):
_ = client.model_ingestion.fail_with_error(input)
def test_complete_non_existent_root_object(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
input = ModelIngestionSuccessInput(
ingestion_id=ingestion.id,
root_object_id="asdfasdfasdfasfd",
project_id=project.id,
# version_message=None,
)
with pytest.raises(GraphQLException):
_ = client.model_ingestion.complete(input)
@@ -1,15 +1,26 @@
import asyncio
from typing import Dict, Optional
from sys import platform
from typing import Dict
import pytest
from specklepy.api.client import SpeckleClient
from specklepy.core.api.enums import (
ModelIngestionStatus,
ProjectModelIngestionUpdatedMessageType,
ProjectModelsUpdatedMessageType,
ProjectUpdatedMessageType,
ProjectVersionsUpdatedMessageType,
UserProjectsUpdatedMessageType,
)
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCreateInput,
ModelIngestionReference,
ModelIngestionRequestCancellationInput,
ModelIngestionUpdateInput,
ProjectModelIngestionSubscriptionInput,
SourceDataInput,
)
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.project_inputs import (
ProjectCreateInput,
@@ -24,9 +35,16 @@ from specklepy.core.api.models import (
UserProjectsUpdatedMessage,
Version,
)
from tests.integration.conftest import create_client, create_version
from specklepy.core.api.models.current import ModelIngestion
from specklepy.core.api.models.subscription_messages import (
ProjectModelIngestionUpdatedMessage,
)
from tests.integration.conftest import create_client, create_version, is_public
WAIT_PERIOD = 0.4 # time in seconds
# WSL is slow AF, so for local runs, we're being extra generous
# For CI runs on linux,m a much smaller wait time is acceptable
SETUP_TIME_SECONDS = 1 if platform == "linux" else 4
MAX_WAIT_TIME_SECONDS = 0.75 if platform == "linux" else 5
@pytest.mark.run()
@@ -55,48 +73,68 @@ class TestSubscriptionResource:
)
return model1
@pytest.fixture
def test_model_ingestion(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model: Model,
) -> ModelIngestion:
project = subscription_client.model_ingestion.create(
ModelIngestionCreateInput(
project_id=test_project.id,
model_id=test_model.id,
progress_message="",
source_data=SourceDataInput(
source_application_slug="pytest",
source_application_version="0.0.0",
file_name=None,
file_size_bytes=None,
),
)
)
return project
@pytest.mark.asyncio
async def test_user_projects_updated(
self,
subscription_client: SpeckleClient,
) -> None:
message: Optional[UserProjectsUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[UserProjectsUpdatedMessage] = loop.create_future()
def callback(d: UserProjectsUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.user_projects_updated(callback)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
input = ProjectCreateInput(name=None, description=None, visibility=None)
created = subscription_client.project.create(input)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, UserProjectsUpdatedMessage)
assert message.id == created.id
assert message.type == UserProjectsUpdatedMessageType.ADDED
assert isinstance(message.project, Project)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
async def test_project_models_updated(
self, subscription_client: SpeckleClient, test_project: Project
) -> None:
message: Optional[ProjectModelsUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelsUpdatedMessage] = loop.create_future()
def callback(d: ProjectModelsUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_models_updated(
@@ -104,51 +142,53 @@ class TestSubscriptionResource:
)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
input = CreateModelInput(
name="my model", description="myDescription", project_id=test_project.id
)
created = subscription_client.model.create(input)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectModelsUpdatedMessage)
assert message.id == created.id
assert message.type == ProjectModelsUpdatedMessageType.CREATED
assert isinstance(message.model, Model)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
async def test_project_updated(
self, subscription_client: SpeckleClient, test_project: Project
) -> None:
message: Optional[ProjectUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectUpdatedMessage] = loop.create_future()
def callback(d: ProjectUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_updated(callback, test_project.id)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(
SETUP_TIME_SECONDS
) # Give time for subscription to be triggered
input = ProjectUpdateInput(id=test_project.id, name="This is my new name")
created = subscription_client.project.update(input)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectUpdatedMessage)
assert message.id == created.id
assert message.type == ProjectUpdatedMessageType.UPDATED
assert isinstance(message.project, Project)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
async def test_project_versions_updated(
@@ -157,13 +197,12 @@ class TestSubscriptionResource:
test_project: Project,
test_model: Model,
) -> None:
message: Optional[ProjectVersionsUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectVersionsUpdatedMessage] = loop.create_future()
def callback(d: ProjectVersionsUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_versions_updated(
@@ -171,15 +210,181 @@ class TestSubscriptionResource:
)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
created = create_version(subscription_client, test_project.id, test_model.id)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectVersionsUpdatedMessage)
assert message.id == created.id
assert message.type == ProjectVersionsUpdatedMessageType.CREATED
assert isinstance(message.version, Version)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
@pytest.mark.skipif(
is_public(), reason="The public API does not support these tests"
)
async def test_project_model_ingestion_cancellation(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model_ingestion: ModelIngestion,
) -> None:
assert not test_model_ingestion.cancellation_requested
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelIngestionUpdatedMessage] = (
loop.create_future()
)
def callback(d: ProjectModelIngestionUpdatedMessage):
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_model_ingestion_cancellation_requested(
callback, test_project.id, ingestion_id=test_model_ingestion.id
)
)
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
cancellation_request = ModelIngestionRequestCancellationInput(
ingestion_id=test_model_ingestion.id,
project_id=test_project.id,
cancellation_message="Please cancel",
)
created = subscription_client.model_ingestion.request_cancellation(
cancellation_request
)
assert created.id == test_model_ingestion.id
assert created.cancellation_requested
assert created.status_data.status == ModelIngestionStatus.PROCESSING
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectModelIngestionUpdatedMessage)
assert message.model_ingestion.id == created.id
assert message.model_ingestion.cancellation_requested
assert (
message.type
== ProjectModelIngestionUpdatedMessageType.CANCELLATION_REQUESTED
)
assert created.status_data.status == ModelIngestionStatus.PROCESSING
if not task.cancel():
await task
@pytest.mark.asyncio
@pytest.mark.skipif(
is_public(), reason="The public API does not support these tests"
)
async def test_project_model_ingestion_cancellation_isnt_triggered_by_updates(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model_ingestion: ModelIngestion,
) -> None:
assert not test_model_ingestion.cancellation_requested
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelIngestionUpdatedMessage] = (
loop.create_future()
)
def callback(d: ProjectModelIngestionUpdatedMessage):
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_model_ingestion_cancellation_requested(
callback, test_project.id, ingestion_id=test_model_ingestion.id
)
)
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
cancellation_request = ModelIngestionUpdateInput(
ingestion_id=test_model_ingestion.id,
project_id=test_project.id,
progress=None,
progress_message="this is just an ordinary update",
)
created = subscription_client.model_ingestion.update_progress(
cancellation_request
)
assert created.id == test_model_ingestion.id
assert not created.cancellation_requested
assert created.status_data.status == ModelIngestionStatus.PROCESSING
await asyncio.sleep(MAX_WAIT_TIME_SECONDS)
assert (
not future.done()
) # make sure the sub did not call back and resolve the future
if not task.cancel():
await task
@pytest.mark.asyncio
@pytest.mark.skipif(
is_public(), reason="The public API does not support these tests"
)
async def test_project_model_ingestion_updates(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model_ingestion: ModelIngestion,
) -> None:
assert not test_model_ingestion.cancellation_requested
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelIngestionUpdatedMessage] = (
loop.create_future()
)
def callback(d: ProjectModelIngestionUpdatedMessage):
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_model_ingestion_updated(
callback,
input=ProjectModelIngestionSubscriptionInput(
project_id=test_project.id,
ingestion_reference=ModelIngestionReference(
ingestion_id=test_model_ingestion.id, model_id=None
),
),
# ingestion_id=test_model_ingestion.id,
)
)
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
progress_message = "this is just an ordinary update"
cancellation_request = ModelIngestionUpdateInput(
ingestion_id=test_model_ingestion.id,
project_id=test_project.id,
progress=None,
progress_message=progress_message,
)
created = subscription_client.model_ingestion.update_progress(
cancellation_request
)
assert created.id == test_model_ingestion.id
assert not created.cancellation_requested
assert created.status_data.status == ModelIngestionStatus.PROCESSING
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectModelIngestionUpdatedMessage)
assert message.model_ingestion.id == created.id
assert not message.model_ingestion.cancellation_requested
assert message.type == ProjectModelIngestionUpdatedMessageType.UPDATED
assert message.model_ingestion.status_data.progress_message == progress_message
assert created.status_data.status == ModelIngestionStatus.PROCESSING
if not task.cancel():
await task
+5
View File
@@ -1,3 +1,4 @@
import os
import random
import uuid
from typing import Dict
@@ -29,6 +30,10 @@ def host() -> str:
return "localhost:3000"
def is_public() -> bool:
return os.getenv("IS_PUBLIC", "false").lower() == "true"
def seed_user(host: str) -> Dict[str, str]:
seed = uuid.uuid4().hex
user_dict = {
@@ -14,10 +14,14 @@ from speckle_automate import (
run_function,
)
from speckle_automate.fixtures import (
TestAutomationEnvironment,
create_test_automation_run_data,
)
from speckle_automate.schema import AutomateBase
from specklepy.api.client import SpeckleClient
from specklepy.core.api.enums import ProjectVisibility
from specklepy.core.api.inputs import ProjectCreateInput
from specklepy.core.api.models import Project
from specklepy.core.api.models.current import Model, Version
from specklepy.core.helpers import crypto_random_string
from specklepy.objects.base import Base
@@ -43,18 +47,33 @@ def test_client(speckle_server_url: str, speckle_token: str) -> SpeckleClient:
return test_client
@pytest.fixture
def project(test_client: SpeckleClient) -> Project:
return test_client.project.create(
ProjectCreateInput(
name="test", description=None, visibility=ProjectVisibility.PRIVATE
)
)
@pytest.fixture
def automation_run_data(
test_client: SpeckleClient, speckle_server_url: str
test_client: SpeckleClient,
speckle_server_url: str,
speckle_token: str,
project: Project,
) -> AutomationRunData:
"""TODO: Set up a test automation for integration testing"""
project_id = crypto_random_string(10)
test_automation_id = crypto_random_string(10)
return create_test_automation_run_data(
test_client, speckle_server_url, project_id, test_automation_id
environment = TestAutomationEnvironment(
token=speckle_token,
server_url=speckle_server_url,
project_id=project.id,
automation_id=test_automation_id,
)
return create_test_automation_run_data(test_client, environment)
@pytest.fixture
def automation_context(
@@ -133,7 +152,7 @@ def automate_function(
raise ValueError("Cannot operate on objects without their id's.")
automation_context.attach_error_to_objects(
"Forbidden speckle_type",
version_root_object.id,
version_root_object,
"This project should not contain the type: "
f"{function_inputs.forbidden_speckle_type}",
)
@@ -164,7 +183,7 @@ def test_function_run(automation_context: AutomationContext) -> None:
assert automation_context.run_status == AutomationStatus.FAILED
status = get_automation_status(
automation_context.automation_run_data.project_id,
automation_context.automation_run_data.model_id,
automation_context.automation_run_data.triggers[0].payload.model_id,
automation_context.speckle_client,
)
assert status["status"] == automation_context.run_status
@@ -205,7 +224,7 @@ def test_create_version_in_project_raises_error_for_same_model(
) -> None:
with pytest.raises(ValueError):
automation_context.create_new_version_in_project(
Base(), automation_context.automation_run_data.branch_name
Base(), automation_context.automation_run_data.triggers[0].payload.model_id
)
@@ -220,8 +239,8 @@ def test_create_version_in_project(
model, version = automation_context.create_new_version_in_project(
root_object, "foobar"
)
isinstance(model, Model)
isinstance(version, Version)
assert isinstance(model, Model)
assert isinstance(version, Version)
@pytest.mark.skip(
@@ -230,9 +249,11 @@ def test_create_version_in_project(
def test_set_context_view(automation_context: AutomationContext) -> None:
automation_context.set_context_view()
trigger = automation_context.automation_run_data.triggers[0].payload
assert automation_context.context_view is not None
assert automation_context.context_view.endswith(
f"models/{automation_context.automation_run_data.model_id}@{automation_context.automation_run_data.version_id}"
f"models/{trigger.model_id}@{trigger.version_id}"
)
automation_context.report_run_status()
@@ -244,7 +265,7 @@ def test_set_context_view(automation_context: AutomationContext) -> None:
assert automation_context.context_view is not None
assert automation_context.context_view.endswith(
f"models/{automation_context.automation_run_data.model_id}@{automation_context.automation_run_data.version_id},{dummy_context}"
f"models/{trigger.model_id}@{trigger.version_id},{dummy_context}"
)
automation_context.report_run_status()
+92
View File
@@ -0,0 +1,92 @@
from typing import Any, Callable
import pytest
from pytest_httpserver import HTTPServer
from requests import HTTPError
from werkzeug import Request, Response
from specklepy.core.api.client import SpeckleClient
from specklepy.logging import metrics
PATH = "/"
def assert_common_properties(payload: Any) -> None:
assert payload["event"] == "SDK Action"
assert payload["properties"]["token"] == "acd87c5a50b56df91a795e999812a3a4"
assert payload["properties"]["type"] == "action"
assert payload["properties"]["server_id"]
assert payload["properties"]["distinct_id"]
assert payload["properties"]["hostApp"] == "python"
assert payload["properties"]["hostAppVersion"]
assert payload["properties"]["core_version"]
def handler(extra_check: Callable[[Any], bool]) -> Callable[[Request], Response]:
def inner(request: Request) -> Response:
json = request.get_json()
payload = json[0]
assert_common_properties(payload)
assert extra_check(payload)
return Response("", 200)
return inner
def test_metrics_track(httpserver: HTTPServer, client: SpeckleClient):
with ScopedMetricsSetup(httpserver.url_for(PATH)) as _:
# Test No email
httpserver.expect_oneshot_request(PATH, "post").respond_with_handler(
handler(lambda payload: "email" not in payload["properties"])
)
metrics.track("SDK Action", client.account, None, True, False)
# Test With email
httpserver.expect_oneshot_request(PATH, "post").respond_with_handler(
handler(
lambda payload: payload["properties"]["email"]
== client.account.userInfo.email
)
)
metrics.track("SDK Action", client.account, None, True, True)
# Test With custom value
httpserver.expect_oneshot_request(PATH, "post").respond_with_handler(
handler(
lambda payload: payload["properties"]["myCustomProp"] == "myCustomValue"
)
)
metrics.track(
"SDK Action", client.account, {"myCustomProp": "myCustomValue"}, True, True
)
def test_metrics_errors(httpserver: HTTPServer):
with ScopedMetricsSetup(httpserver.url_for(PATH)) as _:
httpserver.expect_oneshot_request(PATH, "post").respond_with_data("", 400)
# Expect send_sync == true to mean mean it will raise
with pytest.raises(HTTPError):
metrics.track("SDK Action", send_sync=True)
# Expect send_sync == false to mean mean it won't
metrics.track("SDK Action")
class ScopedMetricsSetup:
"""
Scoped setup and tear down for enabling metrics tracking
"""
tracker: metrics.MetricsTracker
def __init__(self, metrics_url: str):
self.tracker = metrics.initialise_tracker()
self.tracker.analytics_url = metrics_url
def __enter__(self):
metrics.enable()
def __exit__(self, exc_type, exc_value, traceback):
metrics.disable()
metrics.METRICS_TRACKER = None
Generated
+1984 -1207
View File
File diff suppressed because it is too large Load Diff