Compare commits

..

22 Commits

Author SHA1 Message Date
Jedd Morgan 20a412bc65 feat(speckle_ifc): More granular IFC conversion progress (#492)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* IFC importer progress

* Fix broken tests after Ingestion state changes

* Tests

* final tweaks

* add timeout to ModelIngestionCreateInput

* print timing in seconds

* default to None

* small tweak

* fix test

* Fix other tests

* Add progress transport
2026-04-14 14:15:53 +01:00
Jonathon Broughton aa16234e7f feat(automate): allow automation results with no affected objects (#488)
* allow empty affected objects

* adds unit tests for `attach_result_to_objects` method

Introduces tests for handling empty object lists and objects with IDs.

Enhances error handling for cases where objects lack IDs, ensuring robustness in the functionality.

Confirms that the method correctly appends results under various scenarios.

* line length
2026-02-24 20:21:59 +00:00
Jonathon Broughton c1f82fa0d2 fix(tests): Update broken test cases for StreamWrapper URLs (#489)
* Update test cases for StreamWrapper URLs

* Update branch name in StreamWrapper test

* Update project URLs in test_wrapper.py

* Uncomment URLs in test_to_string function

Uncommented specific URLs in the test case to enable testing.
2026-02-23 11:29:35 +01:00
Jedd Morgan c53a51c8ad Jrm/can create model ingestion (#486)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* Add `canCreateModelIngestion` model permission check

* format

* oops
2026-01-29 14:23:44 +00:00
Jedd Morgan c1f27b78f9 feat(api)!: Add model permission checks (#485)
* Add model permission checks

* test_public

* This is the real fix

* mistake

* public api resource
2026-01-29 12:04:21 +01:00
Jedd Morgan 49d4b7d44d doc: MarkReceivedVersionInput clarification (#484)
* MarkReceivedVersionInput clarification

* Reformat
2026-01-27 19:52:30 +03:00
Jedd Morgan 7181f50dda update nullability of invitedBy (#483) 2026-01-15 20:06:13 +03:00
Mucahit Bilal GOKER 2f84214786 feat(ifc): add parentId to nested objects (#481)
* add parentId to nested objects

* rename to parentApplicationId

* implement jedd's feedback

* ruff check

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2026-01-08 09:04:12 +03:00
Jedd Morgan 0fe1af8e75 Update PostgreSQL connection string in docker-compose (#482) 2026-01-07 15:54:26 +00:00
Gergő Jedlicska 6297943fe1 gergo/version message for ingestion (#480)
* feat: use mise for docs build

* feat(modelingestion): add version message reporting
2026-01-05 11:46:31 +00:00
Gergő Jedlicska 428bbe2c3d gergo/queryIngestionFix (#479)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* feat: use mise for docs build

* fix: getting the ingestion query needs to use model ingestion id
2025-12-11 10:44:36 +01:00
Jedd Morgan 0ca22891bc fallback to cgal (#476) 2025-12-10 10:09:00 +00:00
Jedd Morgan fd8c2a32f9 chore(speckleifc): changed ifc status messages (#478)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* remove this function

* Changed progress messages
2025-12-09 17:27:26 +00:00
Jedd Morgan ba8c356d82 chore(speckleifc): Ifc metrics slug tweaks (#477)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* ifc metrics

* add http server tests for metrics

* clean up tests

* change back to localhost:3000

* comment

* renamed wrapper for clarity

* fix unrelated model_ingestion
2025-12-09 16:18:21 +01:00
Gergő Jedlicska 8249cd2184 Jedd/cxpla 340 specklepy (#475)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* First pass

* add tests

* Add cancellation

* fix

* status changes

* fixes

* test fixes

* tests(subscriptions): fix model ingestion tests

* feat(modelingestion): rename resource and add some more tests

* feat(ifcimport): use new modelingestion api

* feat: wrap up new ingestion

* fix: model ingestion payload and test server url

* fix: test port was 3000

* fix: remove version message from model ingestion success input

* fix: test subs cancelled

* ci: signal public of private envv in ci

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-12-08 13:25:54 +01:00
Gergő Jedlicska 7c108a9d43 feat(speckle_automate): version receive metrics (#470)
make sure the automate metrics are attributed to automate host app use
the version metrics to report the version received into automate.

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-11-27 13:27:25 +00:00
Dogukan Karatas 2f2e8ba734 fix: netlify toml path fix (#473)
* updated path

* Add chaining

* debugging deploy

* docs added

* revert back

* disable mise

* feat: use mise for docs build

* fix: netlify do not tripple quote

* ci: netlify use mise

* ci: just run build

---------

Co-authored-by: Mike Tasset <mike.tasset@gmail.com>
Co-authored-by: Gergo Jedlicska <gergo@jedlicska.com>
2025-11-21 15:26:59 +01:00
Dogukan Karatas 9685a2741b updated toml (#472) 2025-11-20 17:13:55 +01:00
Dogukan Karatas 5702d116d0 docs: API Reference (#471)
* first pass

* round2

* experiments

* Moved docs one layer higher

* docs poc

* re-lock

* created docs

* some ci work

* updated toml

* ruff version update

* update toml

* docs group

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-11-20 16:27:01 +01:00
Arseney d440bb5c0f DefaultRule in GraphTraversal was missing should_return method (#469)
* Changed positions in code for _get_active_rule

* should return function added to DefaultRule

* DefaultRule should_return method returns false
2025-11-18 16:12:14 +00:00
Gergő Jedlicska 309c78da37 feat(automate_sdk): support version result reporting (#468)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* feat(automate_sdk): support version result reporting

* fix(automate_tests): skip test, its not working in CI

* docs(automate_sdk): describe new args
2025-11-05 21:50:40 +01:00
Jedd Morgan ff812d5ad9 add 520 status to retry policy (#467) 2025-11-05 10:49:36 +00:00
113 changed files with 4997 additions and 1396 deletions
+2
View File
@@ -59,6 +59,8 @@ jobs:
test-public: # Run integration tests against the public server image
name: Test (public)
runs-on: ubuntu-latest
env:
IS_PUBLIC: "true"
strategy:
matrix:
python-version:
+1
View File
@@ -0,0 +1 @@
words = ["specklepy"]
+1 -4
View File
@@ -97,10 +97,7 @@ services:
STRATEGY_LOCAL: "true"
POSTGRES_URL: "postgres"
POSTGRES_USER: "speckle"
POSTGRES_PASSWORD: "speckle"
POSTGRES_DB: "speckle"
POSTGRES_URL: 'postgres://speckle:speckle@postgres:5432/speckle'
ENABLE_MP: "false"
LOG_PRETTY: "true"
Binary file not shown.

After

Width:  |  Height:  |  Size: 386 B

+50
View File
@@ -0,0 +1,50 @@
# specklepy API Reference
> The Python SDK for Speckle - Build powerful AEC data workflows
**specklepy** is the Python SDK for Speckle, enabling you to interact with Speckle Server, send and receive geometry, and build custom integrations for the AEC industry.
## What is specklepy?
specklepy is a comprehensive Python library that provides:
* **Object-based data exchange** - Send and receive geometry and BIM data without files
* **GraphQL API client** - Full access to Speckle Server's API
* **Extensible object model** - Create custom objects that inherit from `Base`
* **Multiple transport options** - Store data locally (SQLite), in-memory, or on Speckle Server
* **Geometry support** - Rich geometric primitives (Point, Line, Mesh, etc.)
## Speckle Automate
Speckle Automate is a fully fledged CI/CD platform designed to run custom code on Speckle models whenever a new version is available.
As a software developer, you can develop Functions that others in your team consume in Automations. From creating reports to running code compliance checks to wind simulations, there is no limit to what you can do with Automate.
## Installation
Install specklepy using pip:
```bash
pip install specklepy
```
## Quick Example
```python
from specklepy.api.client import SpeckleClient
from specklepy.api.credentials import get_default_account
from specklepy.objects.geometry import Point
# Authenticate
client = SpeckleClient(host="https://app.speckle.systems")
account = get_default_account()
client.authenticate_with_account(account)
# Create geometry
point = Point(x=10, y=20, z=5)
```
## Getting Help
- **Community Forum**: [speckle.community](https://speckle.community/c/help/developers)
- **GitHub Issues**: [github.com/specklesystems/specklepy](https://github.com/specklesystems/specklepy/issues)
@@ -0,0 +1 @@
::: speckle_automate.AutomationContext
+3
View File
@@ -0,0 +1,3 @@
::: speckle_automate.runner.execute_automate_function
::: speckle_automate.runner.run_function
+11
View File
@@ -0,0 +1,11 @@
::: speckle_automate.AutomateBase
::: speckle_automate.AutomationRunData
::: speckle_automate.AutomationResult
::: speckle_automate.ResultCase
::: speckle_automate.AutomationStatus
::: speckle_automate.ObjectResultLevel
+1
View File
@@ -0,0 +1 @@
::: specklepy.api.client.SpeckleClient
+5
View File
@@ -0,0 +1,5 @@
::: specklepy.api.credentials.Account
::: specklepy.api.credentials.UserInfo
::: specklepy.api.credentials.StreamWrapper
+7
View File
@@ -0,0 +1,7 @@
::: specklepy.api.operations.send
::: specklepy.api.operations.receive
::: specklepy.api.operations.serialize
::: specklepy.api.operations.deserialize
@@ -0,0 +1 @@
::: specklepy.api.resources.ActiveUserResource
@@ -0,0 +1 @@
::: specklepy.api.resources.FileImportResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ModelResource
@@ -0,0 +1 @@
::: specklepy.api.resources.OtherUserResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ProjectInviteResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ProjectResource
@@ -0,0 +1 @@
::: specklepy.api.resources.ServerResource
@@ -0,0 +1 @@
::: specklepy.api.resources.SubscriptionResource
@@ -0,0 +1 @@
::: specklepy.api.resources.VersionResource
@@ -0,0 +1 @@
::: specklepy.api.resources.WorkspaceResource
+1
View File
@@ -0,0 +1 @@
::: specklepy.core.api.enums.ProjectVisibility
+11
View File
@@ -0,0 +1,11 @@
::: specklepy.core.api.inputs.ProjectCreateInput
::: specklepy.core.api.inputs.ProjectUpdateInput
::: specklepy.core.api.inputs.CreateModelInput
::: specklepy.core.api.inputs.UpdateModelInput
::: specklepy.core.api.inputs.CreateVersionInput
::: specklepy.core.api.inputs.UpdateVersionInput
+13
View File
@@ -0,0 +1,13 @@
::: specklepy.core.api.models.User
::: specklepy.core.api.models.LimitedUser
::: specklepy.core.api.models.ServerInfo
::: specklepy.core.api.models.Project
::: specklepy.core.api.models.Model
::: specklepy.core.api.models.Version
::: specklepy.core.api.models.current.Workspace
+7
View File
@@ -0,0 +1,7 @@
::: specklepy.logging.exceptions.SpeckleException
::: specklepy.logging.exceptions.GraphQLException
::: specklepy.logging.exceptions.SerializationException
::: specklepy.logging.exceptions.SpeckleWarning
+3
View File
@@ -0,0 +1,3 @@
::: specklepy.objects.Base
::: specklepy.objects.base.DataChunk
+5
View File
@@ -0,0 +1,5 @@
::: specklepy.objects.DataObject
::: specklepy.objects.QgisObject
::: specklepy.objects.BlenderObject
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Arc
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Box
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Circle
@@ -0,0 +1 @@
::: specklepy.objects.geometry.ControlPoint
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Curve
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Ellipse
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Line
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Mesh
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Plane
+1
View File
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Point
@@ -0,0 +1 @@
::: specklepy.objects.geometry.PointCloud
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Polycurve
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Polyline
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Spiral
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Surface
@@ -0,0 +1 @@
::: specklepy.objects.geometry.Vector
@@ -0,0 +1,7 @@
::: specklepy.objects.graph_traversal.GraphTraversal
::: specklepy.objects.graph_traversal.TraversalContext
::: specklepy.objects.graph_traversal.TraversalRule
::: specklepy.objects.graph_traversal.DefaultRule
@@ -0,0 +1 @@
::: specklepy.objects.models.collections.collection.Collection
@@ -0,0 +1 @@
::: specklepy.objects.other.RenderMaterial
@@ -0,0 +1 @@
::: specklepy.objects.primitive.Interval
@@ -0,0 +1 @@
::: specklepy.objects.proxies.ColorProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.GroupProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.InstanceDefinitionProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.InstanceProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.LevelProxy
@@ -0,0 +1 @@
::: specklepy.objects.proxies.RenderMaterialProxy
@@ -0,0 +1 @@
::: specklepy.serialization.base_object_serializer.BaseObjectSerializer
+7
View File
@@ -0,0 +1,7 @@
::: specklepy.transports.abstract_transport.AbstractTransport
::: specklepy.transports.memory.MemoryTransport
::: specklepy.transports.sqlite.SQLiteTransport
::: specklepy.transports.server.ServerTransport
+304
View File
@@ -0,0 +1,304 @@
.md-content h1 {
font-size: 1.125rem;
font-weight: 700;
color: #111827;
letter-spacing: -0.025em;
line-height: 1.2;
}
@media (min-width: 640px) {
.md-content h1 {
font-size: 1.25rem;
}
}
[data-md-color-scheme="slate"] .md-content h1 {
color: #e5e7eb;
}
.md-content h2 {
font-size: 1rem;
font-weight: 700;
color: #111827;
letter-spacing: -0.025em;
margin-top: 2rem;
}
@media (min-width: 640px) {
.md-content h2 {
font-size: 1.125rem;
}
}
[data-md-color-scheme="slate"] .md-content h2 {
color: #e5e7eb;
}
.md-content h3 {
font-size: 0.9375rem;
font-weight: 600;
color: #1f2937;
margin-top: 1.5rem;
}
@media (min-width: 640px) {
.md-content h3 {
font-size: 1rem;
}
}
[data-md-color-scheme="slate"] .md-content h3 {
color: #d1d5db;
}
.md-content p,
.md-content li {
font-size: 0.875rem;
line-height: 1.6;
color: #374151;
margin-top: 0.5rem;
}
[data-md-color-scheme="slate"] .md-content p,
[data-md-color-scheme="slate"] .md-content li {
color: #d1d5db;
}
.doc.doc-object-name {
font-size: 1.125rem;
font-weight: 700;
color: #111827;
}
[data-md-color-scheme="slate"] .doc.doc-object-name {
color: #e5e7eb;
}
.doc.doc-heading {
font-size: 0.9375rem;
font-weight: 600;
color: #1f2937;
}
[data-md-color-scheme="slate"] .doc.doc-heading {
color: #d1d5db;
}
.md-content code {
font-size: 0.75rem;
color: #dc2626;
background-color: #f3f4f6;
padding: 0.125rem 0.375rem;
border-radius: 0.25rem;
}
[data-md-color-scheme="slate"] .md-content code {
color: #fca5a5;
background-color: #1f2937;
}
.md-content pre code {
font-size: 0.75rem;
color: inherit;
background-color: transparent;
padding: 0;
}
.md-content a {
color: #2563eb;
text-decoration: none;
}
.md-content a:hover {
color: #1d4ed8;
text-decoration: underline;
}
[data-md-color-scheme="slate"] .md-content a {
color: #60a5fa;
}
[data-md-color-scheme="slate"] .md-content a:hover {
color: #93c5fd;
}
[data-md-color-scheme="slate"] {
--md-default-bg-color: #000000;
--md-default-fg-color: #ffffff;
--md-code-bg-color: #0a0a0a;
--md-code-fg-color: #ffffff;
}
[data-md-color-scheme="slate"] .md-header {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-tabs {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-footer {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-sidebar {
background-color: #000000;
}
[data-md-color-scheme="slate"] .md-nav {
background-color: #000000;
}
.highlight pre,
.highlight code {
background-color: #f6f8fa !important;
color: #24292e !important;
}
.highlight .k,
.highlight .kc,
.highlight .kd,
.highlight .kn,
.highlight .kp,
.highlight .kr,
.highlight .kt {
color: #d73a49 !important;
}
.highlight .s,
.highlight .s1,
.highlight .s2,
.highlight .sb,
.highlight .sc,
.highlight .sd,
.highlight .se,
.highlight .sh,
.highlight .si,
.highlight .sx,
.highlight .sr,
.highlight .ss {
color: #032f62 !important;
}
.highlight .nf,
.highlight .fm,
.highlight .nc {
color: #6f42c1 !important;
}
.highlight .m,
.highlight .mf,
.highlight .mh,
.highlight .mi,
.highlight .mo {
color: #005cc5 !important;
}
.highlight .c,
.highlight .c1,
.highlight .cm,
.highlight .cp,
.highlight .cs {
color: #6a737d !important;
font-style: italic;
}
.highlight .o,
.highlight .ow {
color: #d73a49 !important;
}
.highlight .n,
.highlight .na,
.highlight .nb,
.highlight .nd,
.highlight .ni,
.highlight .nl,
.highlight .nn,
.highlight .nx,
.highlight .py,
.highlight .nt,
.highlight .nv,
.highlight .bp,
.highlight .vc,
.highlight .vg,
.highlight .vi {
color: #e36209 !important;
}
[data-md-color-scheme="slate"] .highlight pre,
[data-md-color-scheme="slate"] .highlight code {
background-color: #0d1117 !important;
color: #e6edf3 !important;
}
[data-md-color-scheme="slate"] .highlight .k,
[data-md-color-scheme="slate"] .highlight .kc,
[data-md-color-scheme="slate"] .highlight .kd,
[data-md-color-scheme="slate"] .highlight .kn,
[data-md-color-scheme="slate"] .highlight .kp,
[data-md-color-scheme="slate"] .highlight .kr,
[data-md-color-scheme="slate"] .highlight .kt {
color: #ff7b72 !important;
}
[data-md-color-scheme="slate"] .highlight .s,
[data-md-color-scheme="slate"] .highlight .s1,
[data-md-color-scheme="slate"] .highlight .s2,
[data-md-color-scheme="slate"] .highlight .sb,
[data-md-color-scheme="slate"] .highlight .sc,
[data-md-color-scheme="slate"] .highlight .sd,
[data-md-color-scheme="slate"] .highlight .se,
[data-md-color-scheme="slate"] .highlight .sh,
[data-md-color-scheme="slate"] .highlight .si,
[data-md-color-scheme="slate"] .highlight .sx,
[data-md-color-scheme="slate"] .highlight .sr,
[data-md-color-scheme="slate"] .highlight .ss {
color: #a5d6ff !important;
}
[data-md-color-scheme="slate"] .highlight .nf,
[data-md-color-scheme="slate"] .highlight .fm,
[data-md-color-scheme="slate"] .highlight .nc {
color: #d2a8ff !important;
}
[data-md-color-scheme="slate"] .highlight .m,
[data-md-color-scheme="slate"] .highlight .mf,
[data-md-color-scheme="slate"] .highlight .mh,
[data-md-color-scheme="slate"] .highlight .mi,
[data-md-color-scheme="slate"] .highlight .mo {
color: #79c0ff !important;
}
[data-md-color-scheme="slate"] .highlight .c,
[data-md-color-scheme="slate"] .highlight .c1,
[data-md-color-scheme="slate"] .highlight .cm,
[data-md-color-scheme="slate"] .highlight .cp,
[data-md-color-scheme="slate"] .highlight .cs {
color: #8b949e !important;
font-style: italic;
}
[data-md-color-scheme="slate"] .highlight .o,
[data-md-color-scheme="slate"] .highlight .ow {
color: #ff7b72 !important;
}
[data-md-color-scheme="slate"] .highlight .n,
[data-md-color-scheme="slate"] .highlight .na,
[data-md-color-scheme="slate"] .highlight .nb,
[data-md-color-scheme="slate"] .highlight .nd,
[data-md-color-scheme="slate"] .highlight .ni,
[data-md-color-scheme="slate"] .highlight .nl,
[data-md-color-scheme="slate"] .highlight .nn,
[data-md-color-scheme="slate"] .highlight .nx,
[data-md-color-scheme="slate"] .highlight .py,
[data-md-color-scheme="slate"] .highlight .nt,
[data-md-color-scheme="slate"] .highlight .nv,
[data-md-color-scheme="slate"] .highlight .bp,
[data-md-color-scheme="slate"] .highlight .vc,
[data-md-color-scheme="slate"] .highlight .vg,
[data-md-color-scheme="slate"] .highlight .vi {
color: #ffa657 !important;
}
+21
View File
@@ -1,6 +1,27 @@
[tools]
python = "3.13.7"
uv = "0.9.11"
[settings]
experimental = true
python.uv_venv_auto = true
[tasks.install]
run= "uv sync --all-extras --all-groups"
[tasks.install_docs]
run= "uv sync --group docs"
[tasks.build_docs]
description = "Build static docs "
run = "uv run mkdocs build"
depends = ['install_docs']
[tasks.test]
run = "uv run pytest"
[env]
IS_PUBLIC = "false"
+130
View File
@@ -0,0 +1,130 @@
site_name: specklepy Docs
theme:
name: material
font:
text: Inter
favicon: assets/speckle_logo.png
logo: assets/speckle_logo.png
features:
- navigation.tabs
palette:
# Palette toggle for light mode
- scheme: default
primary: white
toggle:
icon: material/weather-night
name: Switch to dark mode
# Palette toggle for dark mode
- scheme: slate
primary: black
logo: assets/logo_white.png
toggle:
icon: material/weather-sunny
name: Switch to light mode
markdown_extensions:
- pymdownx.highlight:
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences
extra_css:
- stylesheets/extra.css
plugins:
- search
- mkdocstrings:
handlers:
python:
paths: [src]
options:
parameter_headings: false
members_order: source
separate_signature: true
filters: ["!^_"] #Ignore _ prefixed properties
docstring_options:
ignore_init_summary: true
merge_init_into_class: true
show_signature_annotations: true
signature_crossrefs: true
show_if_no_docstring: true
show_labels: true
show_source: true
show_symbol_type_heading: true
show_symbol_type_toc: true
show_bases: false
heading_level: 3
inventories:
- url: https://docs.python.org/3/objects.inv
domains: [py, std]
nav:
- Home: index.md
- specklepy SDK:
- API:
- Client: specklepy/api/client.md
- Credentials: specklepy/api/credentials.md
- Operations: specklepy/api/operations.md
- Resources:
- ActiveUserResource: specklepy/api/resources/ActiveUserResource.md
- FileImportResource: specklepy/api/resources/FileImportResource.md
- ModelResource: specklepy/api/resources/ModelResource.md
- OtherUserResource: specklepy/api/resources/OtherUserResource.md
- ProjectInviteResource: specklepy/api/resources/ProjectInviteResource.md
- ProjectResource: specklepy/api/resources/ProjectResource.md
- ServerResource: specklepy/api/resources/ServerResource.md
- SubscriptionResource: specklepy/api/resources/SubscriptionResource.md
- VersionResource: specklepy/api/resources/VersionResource.md
- WorkspaceResource: specklepy/api/resources/WorkspaceResource.md
- Objects:
- Base: specklepy/objects/base.md
- Data Objects: specklepy/objects/data_objects.md
- Geometry:
- Arc: specklepy/objects/geometry/Arc.md
- Box: specklepy/objects/geometry/Box.md
- Circle: specklepy/objects/geometry/Circle.md
- ControlPoint: specklepy/objects/geometry/ControlPoint.md
- Curve: specklepy/objects/geometry/Curve.md
- Ellipse: specklepy/objects/geometry/Ellipse.md
- Line: specklepy/objects/geometry/Line.md
- Mesh: specklepy/objects/geometry/Mesh.md
- Plane: specklepy/objects/geometry/Plane.md
- Point: specklepy/objects/geometry/Point.md
- PointCloud: specklepy/objects/geometry/PointCloud.md
- Polycurve: specklepy/objects/geometry/Polycurve.md
- Polyline: specklepy/objects/geometry/Polyline.md
- Spiral: specklepy/objects/geometry/Spiral.md
- Surface: specklepy/objects/geometry/Surface.md
- Vector: specklepy/objects/geometry/Vector.md
- Primitives:
- Interval: specklepy/objects/primitives/interval.md
- Other:
- RenderMaterial: specklepy/objects/other/render_material.md
- Collection: specklepy/objects/other/collection.md
- Proxies:
- ColorProxy: specklepy/objects/proxies/ColorProxy.md
- GroupProxy: specklepy/objects/proxies/GroupProxy.md
- InstanceProxy: specklepy/objects/proxies/InstanceProxy.md
- InstanceDefinitionProxy: specklepy/objects/proxies/InstanceDefinitionProxy.md
- LevelProxy: specklepy/objects/proxies/LevelProxy.md
- RenderMaterialProxy: specklepy/objects/proxies/RenderMaterialProxy.md
- Graph Traversal: specklepy/objects/graph_traversal/traversal.md
- Transports: specklepy/transports/transports.md
- Serialization: specklepy/serialization/serializer.md
- Core API:
- Models: specklepy/core/api/models/models.md
- Inputs: specklepy/core/api/inputs/inputs.md
- Enums: specklepy/core/api/enums.md
- Logging:
- Exceptions: specklepy/logging/exceptions.md
- Speckle Automate:
- AutomationContext: speckle_automate/automation_context.md
- Runner: speckle_automate/runner.md
- Schema: speckle_automate/schema.md
+3
View File
@@ -0,0 +1,3 @@
[build]
command = "mise run build_docs"
publish = "site"
+9 -1
View File
@@ -22,6 +22,7 @@ dependencies = [
speckleifc = ["ifcopenshell>=0.8.3.post2"]
[dependency-groups]
dev = [
"commitizen>=4.1.0",
"devtools>=0.12.2",
@@ -32,11 +33,18 @@ dev = [
"pytest-asyncio>=0.25.2",
"pytest-cov>=6.0.0",
"pytest-ordering>=0.6",
"ruff>=0.9.2",
"pytest_httpserver >=1.1.3",
"ruff==0.9.2",
"types-deprecated>=1.2.15.20241117",
"types-requests>=2.32.0.20241016",
"types-ujson>=5.10.0.20240515",
]
docs = [
"mkdocs>=1.6.1",
"mkdocs-material>=9.6.5",
"mkdocstrings>=0.28.1",
"mkdocstrings-python>=1.15.0",
]
[project.urls]
repository = "https://github.com/specklesystems/specklepy"
+63 -26
View File
@@ -19,8 +19,12 @@ from speckle_automate.schema import (
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.version_inputs import CreateVersionInput
from specklepy.core.api.inputs.version_inputs import (
CreateVersionInput,
MarkReceivedVersionInput,
)
from specklepy.core.api.models.current import Model, Version
from specklepy.logging import metrics
from specklepy.logging.exceptions import SpeckleException
from specklepy.objects.base import Base
from specklepy.transports.memory import MemoryTransport
@@ -66,6 +70,7 @@ class AutomationContext:
if isinstance(automation_run_data, AutomationRunData)
else AutomationRunData.model_validate_json(automation_run_data)
)
metrics.set_host_app("automate")
speckle_client = SpeckleClient(
automation_run_data.speckle_server_url,
automation_run_data.speckle_server_url.startswith("https"),
@@ -100,6 +105,7 @@ class AutomationContext:
"""Receive the Speckle project version that triggered this automation run."""
# TODO: this is a quick hack to keep implementation consistency.
# Move to proper receive many versions
project_id = self.automation_run_data.project_id
version_id = self.automation_run_data.triggers[0].payload.version_id
try:
version = self.speckle_client.version.get(
@@ -109,7 +115,7 @@ class AutomationContext:
raise ValueError(
f"""Could not receive specified version.
Is your environment configured correctly?
project_id: {self.automation_run_data.project_id}
project_id: {project_id}
model_id: {self.automation_run_data.triggers[0].payload.model_id}
version_id: {self.automation_run_data.triggers[0].payload.version_id}
"""
@@ -124,6 +130,13 @@ class AutomationContext:
base = operations.receive(
version.referenced_object, self._server_transport, self._memory_transport
)
self.speckle_client.version.received(
MarkReceivedVersionInput(
version_id=version_id,
project_id=project_id,
source_application="automate_function",
)
)
# self._closure_tree = base["__closure"]
print(
f"It took {self.elapsed():.2f} seconds to receive",
@@ -245,24 +258,24 @@ class AutomationContext:
"""
)
if self.run_status in [AutomationStatus.SUCCEEDED, AutomationStatus.FAILED]:
object_results = {
"version": 2,
results_dict = self._automation_result.model_dump(by_alias=True)
results = {
"version": 3,
"values": {
"objectResults": self._automation_result.model_dump(by_alias=True)[
"objectResults"
],
"objectResults": results_dict["objectResults"],
"versionResult": results_dict["versionResult"],
"blobIds": self._automation_result.blobs,
},
}
else:
object_results = None
results = None
params = {
"projectId": self.automation_run_data.project_id,
"functionRunId": self.automation_run_data.function_run_id,
"status": self.run_status.value,
"statusMessage": self._automation_result.status_message,
"results": object_results,
"results": results,
"contextView": self._automation_result.result_view,
}
print(f"Reporting run status with content: {params}")
@@ -312,25 +325,49 @@ class AutomationContext:
return upload_response.upload_results[0].blob_id
def mark_run_failed(self, status_message: str) -> None:
"""Mark the current run a failure."""
self._mark_run(AutomationStatus.FAILED, status_message)
def mark_run_failed(
self, status_message: str, version_result: dict[str, Any] | None = None
) -> None:
"""
Mark the current run a failure.
Args:
status_message: Optional message to be displayed.
version_result: Optional data object,
that will be attached to the run results.
The dictionary should be JSON serializable
"""
self._mark_run(AutomationStatus.FAILED, status_message, version_result)
def mark_run_exception(self, status_message: str) -> None:
"""Mark the current run a failure."""
self._mark_run(AutomationStatus.EXCEPTION, status_message)
self._mark_run(AutomationStatus.EXCEPTION, status_message, None)
def mark_run_success(self, status_message: Optional[str]) -> None:
"""Mark the current run a success with an optional message."""
self._mark_run(AutomationStatus.SUCCEEDED, status_message)
def mark_run_success(
self, status_message: str | None, version_result: dict[str, Any] | None = None
) -> None:
"""
Mark the current run a success with an optional message.
Args:
status_message: Optional message to be displayed.
version_result: Optional data object,
that will be attached to the run results.
The dictionary should be JSON serializable
"""
self._mark_run(AutomationStatus.SUCCEEDED, status_message, version_result)
def _mark_run(
self, status: AutomationStatus, status_message: Optional[str]
self,
status: AutomationStatus,
status_message: str | None,
version_result: dict[str, Any] | None,
) -> None:
duration = self.elapsed()
self._automation_result.status_message = status_message
self._automation_result.run_status = status
self._automation_result.elapsed = duration
self._automation_result.version_result = version_result
msg = f"Automation run {status.value} after {duration:.2f} seconds."
print("\n".join([msg, status_message]) if status_message else msg)
@@ -456,29 +493,29 @@ class AutomationContext:
Args:
level: Result level.
category (str): A short tag for the event type.
affected_objects (Union[Base, List[Base]]): A single object or a list of
objects that are causing the info case.
affected_objects (Union[Base, List[Base]]): A single object, a list of
objects, or an empty list. When empty, a result case is still
appended with no object IDs (e.g. for skipped rules or version-level
messages).
message (Optional[str]): Optional message.
metadata: User provided metadata key value pairs
visual_overrides: Case specific 3D visual overrides.
"""
if isinstance(affected_objects, list):
if len(affected_objects) < 1:
raise ValueError(
f"Need atleast one object to report a(n) {level.value.upper()}"
)
object_list = affected_objects
else:
object_list = [affected_objects]
ids: Dict[str, Optional[str]] = {}
# When objects are provided, each must have an id (empty list allowed for
# version-level/skipped results).
for o in object_list:
# validate that the Base.id is not None. If its a None, throw an Exception
if not o.id:
if not getattr(o, "id", None):
raise Exception(
f"You can only attach {level} results to objects with an id."
)
ids[o.id] = o.applicationId
ids[o.id] = getattr(o, "applicationId", None)
print(
f"Created new {level.value.upper()}"
f" category: {category} caused by: {message}"
+5 -7
View File
@@ -88,10 +88,8 @@ def create_test_automation_run(
print(result)
return (
result.get("projectMutations")
.get("automationMutations")
.get("createTestAutomationRun")
return TestAutomationRunData.model_validate(
result["projectMutations"]["automationMutations"]["createTestAutomationRun"]
)
@@ -123,9 +121,9 @@ def create_test_automation_run_data(
project_id=test_automation_environment.project_id,
speckle_server_url=test_automation_environment.server_url,
automation_id=test_automation_environment.automation_id,
automation_run_id=test_automation_run_data["automationRunId"],
function_run_id=test_automation_run_data["functionRunId"],
triggers=test_automation_run_data["triggers"],
automation_run_id=test_automation_run_data.automation_run_id,
function_run_id=test_automation_run_data.function_run_id,
triggers=test_automation_run_data.triggers,
)
+12 -11
View File
@@ -1,7 +1,7 @@
""""""
from enum import Enum
from typing import Any, Dict, List, Literal, Optional
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field
from pydantic.alias_generators import to_camel
@@ -36,7 +36,7 @@ class AutomationRunData(BaseModel):
automation_run_id: str
function_run_id: str
triggers: List[VersionCreationTrigger]
triggers: list[VersionCreationTrigger]
model_config = ConfigDict(
alias_generator=to_camel, populate_by_name=True, protected_namespaces=()
@@ -49,7 +49,7 @@ class TestAutomationRunData(BaseModel):
automation_run_id: str
function_run_id: str
triggers: List[VersionCreationTrigger]
triggers: list[VersionCreationTrigger]
model_config = ConfigDict(
alias_generator=to_camel, populate_by_name=True, protected_namespaces=()
@@ -80,19 +80,20 @@ class ResultCase(AutomateBase):
category: str
level: ObjectResultLevel
object_app_ids: Dict[str, Optional[str]]
message: Optional[str]
metadata: Optional[Dict[str, Any]]
visual_overrides: Optional[Dict[str, Any]]
object_app_ids: dict[str, str | None]
message: str | None
metadata: dict[str, Any] | None
visual_overrides: dict[str, Any] | None
class AutomationResult(AutomateBase):
"""Schema accepted by the Speckle server as a result for an automation run."""
elapsed: float = 0
result_view: Optional[str] = None
result_versions: List[str] = Field(default_factory=list)
blobs: List[str] = Field(default_factory=list)
result_view: str | None = None
result_versions: list[str] = Field(default_factory=list)
blobs: list[str] = Field(default_factory=list)
run_status: AutomationStatus = AutomationStatus.RUNNING
status_message: Optional[str] = None
status_message: str | None = None
object_results: list[ResultCase] = Field(default_factory=list)
version_result: dict[str, Any] | None = None
+7 -4
View File
@@ -18,7 +18,7 @@ def cmd_line_import() -> None:
parser.add_argument("output_path")
parser.add_argument("project_id")
parser.add_argument("version_message")
parser.add_argument("model_id")
parser.add_argument("model_ingestion_id")
# parser.add_argument("model_name")
# parser.add_argument("region_name")
@@ -32,6 +32,8 @@ def cmd_line_import() -> None:
"ifc",
)
client: SpeckleClient | None = None
try:
client = SpeckleClient(SERVER_URL, use_ssl=not SERVER_URL.startswith("http://"))
client.authenticate_with_token(TOKEN)
@@ -41,13 +43,14 @@ def cmd_line_import() -> None:
args.file_path,
project,
args.version_message,
args.model_id,
args.model_ingestion_id,
client,
)
with open(args.output_path, "w") as f:
json.dump({"success": True, "commitId": version.id}, f)
except Exception as e:
error_msg = f"IFC Importer failed with exception:\n{traceback.format_exc()}"
stack_trace = traceback.format_exc()
error_msg = f"IFC Importer failed with exception:\n{stack_trace}"
print(error_msg)
# Write error result
@@ -58,4 +61,4 @@ def cmd_line_import() -> None:
if __name__ == "__main__":
start = time.time()
cmd_line_import()
print(f"Total time (including cleanup): {(time.time() - start) * 1000}ms")
print(f"Total time (including cleanup): {(time.time() - start):.3f}s")
@@ -12,12 +12,23 @@ def data_object_to_speckle(
step_element: entity_instance,
children: list[Base],
current_storey: str | None = None,
parent_element: entity_instance | None = None,
) -> DataObject:
guid = cast(str, step_element.GlobalId)
name = cast(str, step_element.Name or guid)
properties = extract_properties(step_element)
# Add parent ID only if element's parent is also a DataObject (not a Collection)
# Collections are: IfcProject and IfcSpatialStructureElement types
if (
parent_element
and hasattr(parent_element, "GlobalId")
and not parent_element.is_a("IfcProject")
and not parent_element.is_a("IfcSpatialStructureElement")
):
properties["parentApplicationId"] = parent_element.GlobalId
# Add building storey information if available and not a building storey itself
if current_storey and not step_element.is_a("IfcBuildingStorey"):
properties["Building Storey"] = current_storey
+7 -1
View File
@@ -51,4 +51,10 @@ def open_ifc(file_path: str) -> file:
def create_geometry_iterator(ifc_file: file | sqlite) -> iterator:
return iterator(_create_iterator_settings(), ifc_file, multiprocessing.cpu_count())
GEOMETRY_LIBRARY = "hybrid-opencascade-cgal" # First OCC then fallback to CGAL
return iterator(
_create_iterator_settings(),
ifc_file,
multiprocessing.cpu_count(),
geometry_library=GEOMETRY_LIBRARY, # type: ignore
)
+44 -8
View File
@@ -22,12 +22,15 @@ from specklepy.objects import Base
from specklepy.objects.data_objects import DataObject
from specklepy.objects.models.collections.collection import Collection
from specklepy.objects.proxies import InstanceProxy
from specklepy.progress.ingestion_progress import IngestionProgressManager
@dataclass
class ImportJob:
ifc_file: file
progress: IngestionProgressManager
_render_material_manager: RenderMaterialProxyManager = field(
default_factory=lambda: RenderMaterialProxyManager()
)
@@ -39,14 +42,19 @@ class ImportJob:
)
geometries_count: int = 0
geometries_used: int = 0
elements_converted: int = 0
_current_storey_data_object: DataObject | None = field(default=None, init=False)
_display_value_cache: dict[int, list[Base]] = field(default_factory=dict)
"""Maps an instance step ID to a list of instances"""
def convert_element(self, step_element: entity_instance) -> Base:
def convert_element(
self,
step_element: entity_instance,
parent_element: entity_instance | None = None,
) -> Base:
try:
return self._convert_element(step_element)
return self._convert_element(step_element, parent_element)
except SpeckleException:
raise
except Exception as ex:
@@ -54,14 +62,18 @@ class ImportJob:
f"Failed to convert {step_element.is_a()} #{step_element.id()}"
) from ex
def _convert_element(self, step_element: entity_instance) -> Base:
def _convert_element(
self,
step_element: entity_instance,
parent_element: entity_instance | None = None,
) -> Base:
# Track current storey context and store for level proxies
previous_storey_data_object = self._current_storey_data_object
if step_element.is_a("IfcBuildingStorey"):
# Convert the building storey to a DataObject for the level proxy
storey_display_value = self._display_value_cache.get(step_element.id(), [])
self._current_storey_data_object = data_object_to_speckle(
storey_display_value, step_element, []
storey_display_value, step_element, [], parent_element=None
)
children = self._convert_children(step_element)
@@ -86,7 +98,11 @@ class ImportJob:
)
else:
result = data_object_to_speckle(
display_value, step_element, children, current_storey_name
display_value,
step_element,
children,
current_storey_name,
parent_element,
)
# Associate non-spatial elements with current storey for level proxies
if self._current_storey_data_object is not None and result.applicationId:
@@ -96,11 +112,17 @@ class ImportJob:
# Restore previous storey context
self._current_storey_data_object = previous_storey_data_object
self.elements_converted += 1
if self.progress.should_report_progress():
self.progress.report(
f"Converted {self.elements_converted:,} elements", None
)
return result
def _convert_children(self, step_element: entity_instance) -> list[Base]:
return [
self.convert_element(i)
self.convert_element(i, parent_element=step_element)
for i in get_children(step_element)
if self._should_convert(i)
]
@@ -120,12 +142,16 @@ class ImportJob:
def convert(self) -> Base:
start = time.time()
self.pre_process_geometry()
print(f"Geometry conversion complete after {(time.time() - start) * 1000}ms")
print(
f"Geometry conversion complete after {(time.time() - start):.3f}s" # noqa: E501
)
print(f"Created {self.geometries_count} geometries")
start = time.time()
root = self._convert_project_tree()
print(f"Object tree conversion complete after {(time.time() - start) * 1000}ms")
print(
f"Element tree conversion complete after {(time.time() - start):.3f}s" # noqa: E501
)
print(f"Used {self.geometries_used} geometries")
return root
@@ -133,7 +159,10 @@ class ImportJob:
iterator = create_geometry_iterator(self.ifc_file)
if not iterator.initialize():
raise SpeckleException("Failed to find any geometry in file")
self.progress.report("Converting geometries", None)
self.geometries_count = 0
while True:
shape = cast(TriangulationElement, iterator.get())
self.geometries_count += 1
@@ -145,6 +174,11 @@ class ImportJob:
raise SpeckleException(
f"Failed to convert geometry with id: {id}"
) from ex
if self.progress.should_report_progress():
self.progress.report(
f"Converted {self.geometries_count:,} geometries", None
)
if not iterator.next():
break
@@ -185,6 +219,8 @@ class ImportJob:
raise SpeckleException("Expected exactly one IfcProject in file")
project = projects[0]
self.progress.report("Converting elements", None)
tree = self.convert_element(project)
if not isinstance(tree, Collection):
raise TypeError("Expected root object to convert to a Collection")
+105 -33
View File
@@ -1,61 +1,133 @@
import contextlib
import importlib.metadata
import time
import traceback
from pathlib import Path
from speckleifc.ifc_geometry_processing import open_ifc
from speckleifc.importer import ImportJob
from specklepy.core.api.client import SpeckleClient
from specklepy.core.api.inputs.version_inputs import CreateVersionInput
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionFailedInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
SourceDataInput,
)
from specklepy.core.api.models.current import Project, Version
from specklepy.core.api.operations import send
from specklepy.logging import metrics
from specklepy.progress.ingestion_progress import IngestionProgressManager
from specklepy.progress.progress_transport import ProgressTransport
from specklepy.transports.server import ServerTransport
# Since progress messages are currently blocking (no async), we're being extra coarse
# with progress updates to ensure we're not waisting time sending updates.
# We could maybe go a little lower, but for now I'm not risking degrading performance
PROGRESS_INTERVAL_SECONDS = 10
def open_and_convert_file(
file_path: str,
project: Project,
version_message: str | None,
model_id: str,
model_ingestion_id: str,
client: SpeckleClient,
) -> Version:
start = time.time()
very_start = start
try:
start = time.time()
very_start = start
path = Path(file_path)
account = client.account
server_url = account.serverInfo.url
assert server_url
remote_transport = ServerTransport(project.id, account=account)
specklepy_version = importlib.metadata.version("specklepy")
ingestion = client.model_ingestion.start_processing(
ModelIngestionStartProcessingInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
progress_message="Importing IFC file",
source_data=SourceDataInput(
file_name=path.name,
file_size_bytes=path.stat().st_size,
source_application_slug=metrics.HOST_APP,
source_application_version=specklepy_version,
),
)
)
progress = IngestionProgressManager(
client, ingestion, PROGRESS_INTERVAL_SECONDS
)
account = client.account
server_url = account.serverInfo.url
assert server_url
remote_transport = ServerTransport(project.id, account=account)
progress_transport = ProgressTransport(
progress,
)
ifc_file = open_ifc(file_path) # pyright: ignore[reportUnknownVariableType]
progress.report("Opening file", None)
ifc_file = open_ifc(file_path) # pyright: ignore[reportUnknownVariableType]
import_job = ImportJob(ifc_file) # pyright: ignore[reportUnknownArgumentType]
data = import_job.convert()
import_job = ImportJob(ifc_file, progress) # pyright: ignore[reportUnknownArgumentType]
data = import_job.convert()
print(f"File conversion complete after {(time.time() - start) * 1000}ms")
print(
f"File conversion complete after {(time.time() - start):.3f}s" # noqa: E501
)
start = time.time()
start = time.time()
root_id = send(data, transports=[remote_transport], use_default_cache=False)
print(f"Sending to speckle complete after: {(time.time() - start) * 1000}ms")
progress.report("Uploading objects", None)
root_id = send(
data,
transports=[remote_transport, progress_transport],
use_default_cache=False,
)
print(
f"Sending to speckle complete after: {(time.time() - start):.3f}s" # noqa: E501
)
start = time.time()
start = time.time()
create_version = CreateVersionInput(
object_id=root_id,
model_id=model_id,
project_id=project.id,
message=version_message,
source_application="ifc",
)
version = client.version.create(create_version)
end = time.time()
print(f"Version committed after: {(end - start) * 1000}ms")
version_id = client.model_ingestion.complete(
ModelIngestionSuccessInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
root_object_id=root_id,
version_message=version_message,
)
)
print(f"Total time (to commit): {(end - very_start) * 1000}ms")
del ifc_file
# needed to query version until ingestion api expands to serve it
version = client.version.get(version_id, project.id)
custom_properties = {"ui": "dui3", "actionSource": "import"}
if project.workspace_id:
custom_properties["workspace_id"] = project.workspace_id
metrics.track(metrics.SEND, account, custom_properties, send_sync=True)
end = time.time()
print(f"Version committed after: {(end - start):.3f}s")
return version
print(f"Total time (to commit): {(end - very_start):.3f}s")
del ifc_file
custom_properties = {"ui": "dui3", "actionSource": "import"}
if project.workspace_id:
custom_properties["workspace_id"] = project.workspace_id
metrics.track(
metrics.SEND,
account,
custom_properties,
send_sync=True,
track_email=True,
)
return version
except Exception as e:
stack_trace = traceback.format_exc()
with contextlib.suppress(Exception):
# make sure to not report process kills when we're cancelling
client.model_ingestion.fail_with_error(
ModelIngestionFailedInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
error_reason=str(e),
error_stacktrace=stack_trace,
)
)
raise e
+51
View File
@@ -0,0 +1,51 @@
import time
from speckleifc.main import open_and_convert_file
from specklepy.api.client import SpeckleClient
from specklepy.core.api.credentials import get_accounts_for_server
from specklepy.logging import metrics
def _manual_import() -> None:
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCreateInput,
SourceDataInput,
)
PROJECT_ID = "412a3c3927"
MODEL_ID = "223e61212d"
SERVER_URL = "latest.speckle.systems"
FILE_PATH = r"C:\Test Files\ifc\AC20-FZK-Haus.ifc" # noqa: E501
metrics.set_host_app(
"ifc",
)
account = get_accounts_for_server(SERVER_URL)[0]
client = SpeckleClient(SERVER_URL, use_ssl=not SERVER_URL.startswith("http://"))
client.authenticate_with_account(account)
ingestion = client.model_ingestion.create(
ModelIngestionCreateInput(
model_id=MODEL_ID,
project_id=PROJECT_ID,
progress_message="",
source_data=SourceDataInput(
source_application_slug="speckleifc",
source_application_version="0.0.0",
file_name=None,
file_size_bytes=None,
),
max_idle_timeout_seconds=2700, # 45mins
)
)
project = client.project.get(PROJECT_ID)
open_and_convert_file(FILE_PATH, project, None, ingestion.id, client)
if __name__ == "__main__":
start = time.time()
_manual_import()
print(f"Total time (including cleanup): {(time.time() - start):.3f}s")
+7
View File
@@ -4,6 +4,7 @@ from specklepy.api.credentials import Account
from specklepy.api.resources import (
ActiveUserResource,
FileImportResource,
ModelIngestionResource,
ModelResource,
OtherUserResource,
ProjectInviteResource,
@@ -119,6 +120,12 @@ class SpeckleClient(CoreSpeckleClient):
client=self.httpclient,
server_version=server_version,
)
self.model_ingestion = ModelIngestionResource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.file_import = FileImportResource(
account=self.account,
basepath=self.url,
+5
View File
@@ -1,5 +1,8 @@
from specklepy.api.resources.current.active_user_resource import ActiveUserResource
from specklepy.api.resources.current.file_import_resource import FileImportResource
from specklepy.api.resources.current.model_ingestion_resource import (
ModelIngestionResource,
)
from specklepy.api.resources.current.model_resource import ModelResource
from specklepy.api.resources.current.other_user_resource import OtherUserResource
from specklepy.api.resources.current.project_invite_resource import (
@@ -22,4 +25,6 @@ __all__ = [
"SubscriptionResource",
"VersionResource",
"WorkspaceResource",
"FileImportResource",
"ModelIngestionResource",
]
@@ -15,7 +15,7 @@ from specklepy.logging import metrics
class FileImportResource(CoreResource):
"""API Access class for projects"""
"""API Access class for file imports"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
@@ -0,0 +1,57 @@
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
)
from specklepy.core.api.models.current import (
ModelIngestion,
)
from specklepy.core.api.resources import (
ModelIngestionResource as CoreResource,
)
from specklepy.logging import metrics
class ModelIngestionResource(CoreResource):
"""API Access class for model ingestion"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(account, basepath, client, server_version)
def create(self, input: ModelIngestionCreateInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Create"})
return super().create(input)
def get_ingestion(self, project_id: str, model_ingestion_id: str) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Get"})
return super().get_ingestion(project_id, model_ingestion_id)
def update_progress(self, input: ModelIngestionUpdateInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Update"})
return super().update_progress(input)
def start_processing(
self, input: ModelIngestionStartProcessingInput
) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Start Processing"})
return super().start_processing(input)
def requeue(self, input: ModelIngestionRequeueInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Update"})
return super().requeue(input)
def complete(self, input: ModelIngestionSuccessInput) -> str:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion End"})
return super().complete(input)
def fail_with_error(self, input: ModelIngestionFailedInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Error"})
return super().fail_with_error(input)
def fail_with_cancel(self, input: ModelIngestionCancelledInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Cancel"})
return super().fail_with_cancel(input)
@@ -8,6 +8,10 @@ from specklepy.core.api.inputs.model_inputs import (
)
from specklepy.core.api.inputs.project_inputs import ProjectModelsFilter
from specklepy.core.api.models import Model, ModelWithVersions, ResourceCollection
from specklepy.core.api.models.current import (
ModelPermissionChecks,
PermissionCheckResult,
)
from specklepy.core.api.resources import ModelResource as CoreResource
from specklepy.logging import metrics
@@ -72,3 +76,17 @@ class ModelResource(CoreResource):
def update(self, input: UpdateModelInput) -> Model:
metrics.track(metrics.SDK, self.account, {"name": "Model Update"})
return super().update(input)
def get_permissions(self, model_id: str, project_id: str) -> ModelPermissionChecks:
metrics.track(metrics.SDK, self.account, {"name": "Model Get Permissions"})
return super().get_permissions(model_id, project_id)
def can_create_model_ingestion(
self, model_id: str, project_id: str
) -> PermissionCheckResult:
metrics.track(
metrics.SDK,
self.account,
{"name": "Model Get Permissions canCreateIngestion"},
)
return super().can_create_model_ingestion(model_id, project_id)
+7
View File
@@ -12,6 +12,7 @@ from specklepy.core.api.credentials import Account
from specklepy.core.api.resources import (
ActiveUserResource,
FileImportResource,
ModelIngestionResource,
ModelResource,
OtherUserResource,
ProjectInviteResource,
@@ -250,6 +251,12 @@ class SpeckleClient:
client=self.httpclient,
server_version=server_version,
)
self.model_ingestion = ModelIngestionResource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.subscription = SubscriptionResource(
account=self.account,
basepath=self.ws_url,
+16
View File
@@ -7,6 +7,7 @@ class ProjectVisibility(str, Enum):
PRIVATE = "PRIVATE"
PUBLIC = "PUBLIC"
UNLISTED = "UNLISTED"
"""Deprecated, use PUBLIC instead"""
WORKSPACE = "WORKSPACE"
@@ -30,3 +31,18 @@ class ProjectVersionsUpdatedMessageType(str, Enum):
CREATED = "CREATED"
DELETED = "DELETED"
UPDATED = "UPDATED"
class ProjectModelIngestionUpdatedMessageType(str, Enum):
CANCELLATION_REQUESTED = "cancellationRequested"
CREATED = "created"
DELETED = "deleted"
UPDATED = "updated"
class ModelIngestionStatus(str, Enum):
CANCELLED = "cancelled"
FAILED = "failed"
PROCESSING = "processing"
QUEUED = "queued"
SUCCESS = "success"
@@ -0,0 +1,78 @@
from specklepy.core.api.enums import ProjectModelIngestionUpdatedMessageType
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
class SourceDataInput(GraphQLBaseModel):
source_application_slug: str
source_application_version: str
file_name: str | None
file_size_bytes: int | None
class ModelIngestionCreateInput(GraphQLBaseModel):
model_id: str
project_id: str
progress_message: str
source_data: SourceDataInput
max_idle_timeout_seconds: int | None = None
class ModelIngestionStartProcessingInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress_message: str
source_data: SourceDataInput
class ModelIngestionRequeueInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress_message: str
class ModelIngestionUpdateInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress: float | None
progress_message: str
class ModelIngestionSuccessInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
root_object_id: str
version_message: str | None
class ModelIngestionFailedInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
error_reason: str
error_stacktrace: str | None
class ModelIngestionCancelledInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
cancellation_message: str
class ModelIngestionRequestCancellationInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
cancellation_message: str
class ModelIngestionReference(GraphQLBaseModel):
"""
`@oneOf` i.e. server expects **either** `ingestion_id` or `model_id`, but not both.
"""
ingestion_id: str | None
model_id: str | None
class ProjectModelIngestionSubscriptionInput(GraphQLBaseModel):
project_id: str
ingestion_reference: ModelIngestionReference
message_type: ProjectModelIngestionUpdatedMessageType | None = None
@@ -34,4 +34,8 @@ class MarkReceivedVersionInput(GraphQLBaseModel):
version_id: str
project_id: str
source_application: str
"""
IMPORTANT: this is meant to be the slug of the application that has done the
receiving, not to be confused with `Version.sourceApplication`
"""
message: Optional[str] = None
+25 -3
View File
@@ -1,7 +1,7 @@
from datetime import datetime
from typing import Generic, List, TypeVar
from specklepy.core.api.enums import ProjectVisibility
from specklepy.core.api.enums import ModelIngestionStatus, ProjectVisibility
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
from specklepy.logging.exceptions import WorkspacePermissionException
@@ -105,7 +105,7 @@ class PendingStreamCollaborator(GraphQLBaseModel):
project_name: str
title: str
role: str
invited_by: LimitedUser
invited_by: LimitedUser | None = None
user: LimitedUser | None = None
token: str | None
@@ -137,6 +137,12 @@ class Version(GraphQLBaseModel):
source_application: str | None
class ModelPermissionChecks(GraphQLBaseModel):
can_update: "PermissionCheckResult"
can_delete: "PermissionCheckResult"
can_create_version: "PermissionCheckResult"
class Model(GraphQLBaseModel):
author: LimitedUser | None
created_at: datetime
@@ -156,7 +162,6 @@ class ProjectPermissionChecks(GraphQLBaseModel):
can_create_model: "PermissionCheckResult"
can_delete: "PermissionCheckResult"
can_load: "PermissionCheckResult"
can_publish: "PermissionCheckResult"
class Project(GraphQLBaseModel):
@@ -244,3 +249,20 @@ class FileImport(GraphQLBaseModel):
class FileUploadUrl(GraphQLBaseModel):
url: str
file_id: str
class ModelIngestionStatusData(GraphQLBaseModel):
status: ModelIngestionStatus
progress_message: str | None = None
version_id: str | None = None
class ModelIngestion(GraphQLBaseModel):
id: str
created_at: datetime
updated_at: datetime
model_id: str
project_id: str
user_id: str
cancellation_requested: bool
status_data: ModelIngestionStatusData
@@ -1,12 +1,13 @@
from typing import Optional
from specklepy.core.api.enums import (
ProjectModelIngestionUpdatedMessageType,
ProjectModelsUpdatedMessageType,
ProjectUpdatedMessageType,
ProjectVersionsUpdatedMessageType,
UserProjectsUpdatedMessageType,
)
from specklepy.core.api.models.current import Model, Project, Version
from specklepy.core.api.models.current import Model, ModelIngestion, Project, Version
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
@@ -33,3 +34,8 @@ class ProjectVersionsUpdatedMessage(GraphQLBaseModel):
type: ProjectVersionsUpdatedMessageType
model_id: str
version: Optional[Version]
class ProjectModelIngestionUpdatedMessage(GraphQLBaseModel):
model_ingestion: ModelIngestion
type: ProjectModelIngestionUpdatedMessageType
@@ -1,5 +1,8 @@
from specklepy.core.api.resources.current.active_user_resource import ActiveUserResource
from specklepy.core.api.resources.current.file_import_resource import FileImportResource
from specklepy.core.api.resources.current.model_ingestion_resource import (
ModelIngestionResource,
)
from specklepy.core.api.resources.current.model_resource import ModelResource
from specklepy.core.api.resources.current.other_user_resource import OtherUserResource
from specklepy.core.api.resources.current.project_invite_resource import (
@@ -24,4 +27,6 @@ __all__ = [
"SubscriptionResource",
"VersionResource",
"WorkspaceResource",
"FileImportResource",
"ModelIngestionResource",
]
@@ -16,13 +16,15 @@ from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
from specklepy.logging.exceptions import SpeckleException
NAME = "file_import"
class UploadFileResponse(GraphQLBaseModel):
etag: str
class FileImportResource(ResourceBase):
"""API Access class for project invites"""
"""API Access class for file imports"""
def __init__(
self,
@@ -36,7 +38,7 @@ class FileImportResource(ResourceBase):
basepath=basepath,
client=client,
server_version=server_version,
name="file-import",
name=NAME,
)
def finish_file_import_job(self, input: FinishFileImportInput) -> bool:
@@ -0,0 +1,417 @@
from typing import Any, Tuple
from gql import Client, gql
from specklepy.api.credentials import Account
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequestCancellationInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
)
from specklepy.core.api.models.current import (
ModelIngestion,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
NAME = "ingestion"
class ModelIngestionResource(ResourceBase):
"""API Access class for model ingestion"""
def __init__(
self,
account: Account,
basepath: str,
client: Client,
server_version: Tuple[Any, ...] | None,
) -> None:
super().__init__(
account=account,
basepath=basepath,
client=client,
name=NAME,
server_version=server_version,
)
def get_ingestion(self, project_id: str, model_ingestion_id: str) -> ModelIngestion:
QUERY = gql(
"""
query Query($projectId: String!, $modelIngestionId: ID!) {
data:project(id: $projectId) {
data:ingestion(id: $modelIngestionId) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
... on ModelIngestionSuccessStatus
{
versionId
}
}
}
}
}
""" # noqa: E501
)
variables = {
"projectId": project_id,
"modelIngestionId": model_ingestion_id,
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[ModelIngestion]],
QUERY,
variables,
).data.data
def create(self, input: ModelIngestionCreateInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionCreate($input: ModelIngestionCreateInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: create(input: $input) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def start_processing(
self, input: ModelIngestionStartProcessingInput
) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionStartProcessing($input: ModelIngestionStartProcessingInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: startProcessing(input: $input) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def requeue(self, input: ModelIngestionRequeueInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionRequeue($input: ModelIngestionRequeueInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: requeue(input: $input) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def update_progress(self, input: ModelIngestionUpdateInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionUpdateProgress(
$input: ModelIngestionUpdateInput!
) {
data: projectMutations {
data: modelIngestionMutations {
data: updateProgress(input: $input) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def complete(self, input: ModelIngestionSuccessInput) -> str:
"""
Request that the server completes the ingestion by creating a version
If successful, the job will be in a terminal "successful" state.
For failed Ingestions, use `fail_with_error` instead
For user cancellation, use `fail_with_cancelled` instead
Arguments:
input {ModelIngestionSuccessInput} -- input variable
Returns:
str -- the id of the version that was just created to complete the ingestion
"""
QUERY = gql(
"""
mutation IngestionComplete($input: ModelIngestionSuccessInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: completeWithVersion(input: $input) {
data:statusData {
... on ModelIngestionSuccessStatus {
data:versionId
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[DataResponse[DataResponse[str]]]]],
QUERY,
variables,
).data.data.data.data.data
def fail_with_error(self, input: ModelIngestionFailedInput) -> ModelIngestion:
"""
Fail the job with an error.
For user requested cancellation, use `fail_with_cancelled` instead
"""
QUERY = gql(
"""
mutation IngestionFailWithError($input: ModelIngestionFailedInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: failWithError(input: $input) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
def fail_with_cancel(self, input: ModelIngestionCancelledInput) -> ModelIngestion:
"""
Fail the ingestion with a `cancelled` status.
This should only be done if the user has explicitly requested cancellation
Other forms of cancellation use `fail_with_error`
The ingestion should then enter a terminal "canceled" state
"""
QUERY = gql(
"""
mutation IngestionFailWithCancel($input: ModelIngestionCancelledInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: failWithCancel(input: $input) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
def request_cancellation(
self, input: ModelIngestionRequestCancellationInput
) -> ModelIngestion:
"""
Request that the ingestion is canceled.
Note: simply calling this mutation does not immediately cancel,
it doesn't even guarantee it will be canceled at all.
It's up to the client to observe this cancellation request
via `subscription.project_model_ingestion_cancellation_requested`
and report it as cancelled (via `ingestion.fail_with_cancel`
See "cooperative cancellation pattern"
"""
QUERY = gql(
"""
mutation IngestionRequestCancellation($input: ModelIngestionRequestCancellationInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: requestCancellation (input: $input) {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
@@ -10,6 +10,10 @@ from specklepy.core.api.inputs.model_inputs import (
)
from specklepy.core.api.inputs.project_inputs import ProjectModelsFilter
from specklepy.core.api.models import Model, ModelWithVersions, ResourceCollection
from specklepy.core.api.models.current import (
ModelPermissionChecks,
PermissionCheckResult,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
@@ -299,3 +303,71 @@ class ModelResource(ResourceBase):
return self.make_request_and_parse_response(
DataResponse[DataResponse[Model]], QUERY, variables
).data.data
def get_permissions(self, project_id: str, model_id: str) -> ModelPermissionChecks:
QUERY = gql(
"""
query ModelPermissions($projectId: String!, $modelId: String!) {
data:project(id: $projectId) {
data:model(id: $modelId) {
data:permissions {
canUpdate {
authorized
code
message
}
canDelete {
authorized
code
message
}
canCreateVersion {
authorized
code
message
}
}
}
}
}
"""
)
variables = {"projectId": project_id, "modelId": model_id}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelPermissionChecks]]],
QUERY,
variables,
).data.data.data
def can_create_model_ingestion(
self, project_id: str, model_id: str
) -> PermissionCheckResult:
QUERY = gql(
"""
query ModelPermissions($projectId: String!, $modelId: String!) {
data:project(id: $projectId) {
data:model(id: $modelId) {
data:permissions {
data:canCreateIngestion {
authorized
code
message
}
}
}
}
}
"""
)
variables = {"projectId": project_id, "modelId": model_id}
return self.make_request_and_parse_response(
DataResponse[
DataResponse[DataResponse[DataResponse[PermissionCheckResult]]]
],
QUERY,
variables,
).data.data.data.data
@@ -6,17 +6,25 @@ from graphql import DocumentNode
from pydantic import BaseModel
from typing_extensions import TypeVar
from specklepy.core.api.enums import ProjectModelIngestionUpdatedMessageType
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionReference,
ProjectModelIngestionSubscriptionInput,
)
from specklepy.core.api.models import (
ProjectModelsUpdatedMessage,
ProjectUpdatedMessage,
ProjectVersionsUpdatedMessage,
UserProjectsUpdatedMessage,
)
from specklepy.core.api.models.subscription_messages import (
ProjectModelIngestionUpdatedMessage,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
from specklepy.logging.exceptions import SpeckleException
NAME = "subscribe"
NAME = "subscription"
TEventArgs = TypeVar("TEventArgs", bound=BaseModel)
@@ -202,6 +210,72 @@ class SubscriptionResource(ResourceBase):
callback=lambda d: callback(d.data),
)
async def project_model_ingestion_updated(
self,
callback: Callable[[ProjectModelIngestionUpdatedMessage], None],
input: ProjectModelIngestionSubscriptionInput,
) -> None:
QUERY = gql(
"""
subscription IngestionUpdated($input: ProjectModelIngestionSubscriptionInput!) {
data: projectModelIngestionUpdated(input: $input) {
modelIngestion {
id
createdAt
updatedAt
modelId
projectId
userId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
... on ModelIngestionSuccessStatus
{
versionId
}
}
}
type
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(
warnings="error", by_alias=True, exclude_none=True
),
}
await self.subscribe_2(
DataResponse[ProjectModelIngestionUpdatedMessage],
QUERY,
variables,
callback=lambda d: callback(d.data),
)
async def project_model_ingestion_cancellation_requested(
self,
callback: Callable[[ProjectModelIngestionUpdatedMessage], None],
project_id: str,
ingestion_id: str,
) -> None:
await self.project_model_ingestion_updated(
callback,
ProjectModelIngestionSubscriptionInput(
project_id=project_id,
ingestion_reference=ModelIngestionReference(
ingestion_id=ingestion_id, model_id=None
),
message_type=ProjectModelIngestionUpdatedMessageType.CANCELLATION_REQUESTED,
),
)
@check_wsclient
async def subscribe_2(
self,
@@ -14,7 +14,7 @@ from specklepy.core.api.models import ResourceCollection, Version
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
NAME = "model"
NAME = "version"
class VersionResource(ResourceBase):
@@ -16,7 +16,7 @@ NAME = "workspace"
class WorkspaceResource(ResourceBase):
"""API Access class for models"""
"""API Access class for workspaces"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
+41 -7
View File
@@ -1,19 +1,20 @@
import contextlib
import getpass
import hashlib
import importlib.metadata
import logging
import platform
import queue
import sys
import threading
from typing import Any
from typing import Any, Literal
import requests
from specklepy.core.api.credentials import Account
"""
Anonymous telemetry to help us understand how to make a better Speckle.
Lightweight usage telemetry to help us understand how to make a better Speckle.
This really helps us to deliver a better open source project and product!
"""
TRACK = True
@@ -22,13 +23,14 @@ HOST_APP_VERSION = f"python {'.'.join(map(str, sys.version_info[:2]))}"
PLATFORMS = {"win32": "Windows", "cygwin": "Windows", "darwin": "Mac OS X"}
LOG = logging.getLogger(__name__)
METRICS_TRACKER = None
METRICS_TRACKER: "MetricsTracker | None" = None
# actions
SDK = "SDK Action"
CONNECTOR = "Connector Action"
RECEIVE = "Receive"
SEND = "Send"
ACTIONS = Literal["SDK Action", "Connector Action", "Receive", "Send"]
def disable():
@@ -48,15 +50,32 @@ def set_host_app(host_app: str, host_app_version: str | None = None):
def track(
action: str,
action: ACTIONS,
account: Account | None = None,
custom_props: dict | None = None,
send_sync: bool = False,
track_email: bool = False,
):
"""
:param action:
:type action: ACTIONS
:param account:
:type account: Account | None
:param custom_props:
:type custom_props: dict | None
:param send_sync: When `True`, the track event is executed synchronously,
and any exceptions will be raised.
When `False`, the track it is deferred to a queue, and any exceptions will be
swallowed and reported as warnings.
:type send_sync: bool
:param track_email: When `True`, the users plain text email address will be included
:type track_email: bool
"""
if not TRACK:
return
tracker = initialise_tracker(account)
event_params: dict[str, Any] = {
"event": action,
"properties": {
@@ -72,6 +91,18 @@ def track(
if custom_props:
event_params["properties"].update(custom_props)
if track_email:
event_params["properties"]["email"] = tracker.last_email
try:
specklepy_version = importlib.metadata.version("specklepy")
event_params["properties"]["core_version"] = specklepy_version
except importlib.metadata.PackageNotFoundError:
if send_sync:
raise
else:
LOG.warning("Failed to read specklepy's version number", exc_info=True)
if send_sync:
tracker.send_event(event_params)
else:
@@ -84,7 +115,7 @@ def initialise_tracker(account: Account | None = None) -> "MetricsTracker":
METRICS_TRACKER = MetricsTracker()
if account:
METRICS_TRACKER.set_last_user(account.userInfo.email)
METRICS_TRACKER.set_last_user_email(account.userInfo.email)
METRICS_TRACKER.set_last_server(account.serverInfo.url)
return METRICS_TRACKER
@@ -103,6 +134,7 @@ class MetricsTracker(metaclass=Singleton):
analytics_url: str = "https://analytics.speckle.systems/track?ip=1"
analytics_token: str = "acd87c5a50b56df91a795e999812a3a4"
last_user: str = ""
last_email: str = ""
last_server: str | None = None
platform: str
@@ -121,17 +153,19 @@ class MetricsTracker(metaclass=Singleton):
if node and user:
self.last_user = f"@{self.hash(f'{node}-{user}')}"
def set_last_user(self, email: str | None) -> None:
def set_last_user_email(self, email: str | None) -> None:
if not email:
return
self.last_user = f"@{self.hash(email)}"
self.last_email = email
def set_last_server(self, server: str | None) -> None:
if not server:
return
self.last_server = self.hash(server)
def hash(self, value: str) -> str:
@staticmethod
def hash(value: str) -> str:
inputList = value.lower().split("://")
input = inputList[len(inputList) - 1].split("/")[0].split("?")[0]
return hashlib.md5(input.encode("utf-8")).hexdigest().upper()
+24
View File
@@ -323,6 +323,30 @@ def _validate_type(t: Optional[type], value: Any) -> Tuple[bool, Any]:
@dataclass(kw_only=True)
class Base(_RegisteringBase, speckle_type="Base"):
"""Base class for all Speckle objects.
The base object class is the foundation of all data being
transferred with Speckle. Any custom data structure that you want to transfer via
Speckle should inherit from it.
Objects in Speckle are immutable for storage purposes. When any property changes,
the object gets a new identity (hash). This hash is stored in the `id` property
after serialization.
Attributes:
id: Unique identifier (hash) for the object. This is typically
set automatically during serialization and depends on the object's properties.
applicationId: Optional identifier for the application that created
this object, can store the host application's native object ID.
```py title="Example"
from specklepy.objects.base import Base
obj = Base(id="some-id", applicationId="my-app")
obj["custom_prop"] = 42 # Add a dynamic property
obj["@detached_prop"] = another_object # Add a detached property
```
"""
id: Union[str, None] = None
# totalChildrenCount: Union[int, None] = None
applicationId: Union[str, None] = None
+4
View File
@@ -18,6 +18,10 @@ class DataObject(
speckle_type="Objects.Data.DataObject",
detachable={"displayValue"},
):
"""
A generic data object that can hold arbitrary properties and display values.
"""
name: str
properties: Dict[str, object]
displayValue: List[Base]
+39
View File
@@ -9,6 +9,30 @@ from specklepy.objects.interfaces import ICurve, IHasUnits
@dataclass(kw_only=True)
class Arc(Base, IHasUnits, ICurve, speckle_type="Objects.Geometry.Arc"):
"""
An arc defined by a plane, start point, mid point and end point.
This class represents a circular arc in 3D space, defined by three points
and a plane. The arc is a portion of a circle that lies on the specified plane.
Attributes:
plane: The plane on which the arc lies
startPoint: The starting point of the arc
midPoint: A point on the arc between the start and end points
endPoint: The ending point of the arc.
```py title="Example"
from specklepy.objects.geometry.plane import Plane
from specklepy.objects.geometry.point import Point
plane = Plane(origin=Point(0, 0, 0), normal=Point(0, 0, 1))
start = Point(1, 0, 0)
mid = Point(0.7071, 0.7071, 0)
end = Point(0, 1, 0)
arc = Arc(plane=plane, startPoint=start, midPoint=mid, endPoint=end)
```
"""
plane: Plane
startPoint: Point
midPoint: Point
@@ -16,10 +40,20 @@ class Arc(Base, IHasUnits, ICurve, speckle_type="Objects.Geometry.Arc"):
@property
def radius(self) -> float:
"""Calculates the radius of the arc.
Returns:
The radius of the arc, as the distance from the start point to the origin.
"""
return self.startPoint.distance_to(self.plane.origin)
@property
def length(self) -> float:
"""Calculates the length of the arc.
Returns:
The length of the arc.
"""
start_to_mid = self.startPoint.distance_to(self.midPoint)
mid_to_end = self.midPoint.distance_to(self.endPoint)
r = self.radius
@@ -30,6 +64,11 @@ class Arc(Base, IHasUnits, ICurve, speckle_type="Objects.Geometry.Arc"):
@property
def measure(self) -> float:
"""Calculates the angular measure of the arc in radians.
Returns:
The angular measure of the arc in radians.
"""
start_to_mid = self.startPoint.distance_to(self.midPoint)
mid_to_end = self.midPoint.distance_to(self.endPoint)
r = self.radius
+33 -1
View File
@@ -9,7 +9,29 @@ from specklepy.objects.primitive import Interval
@dataclass(kw_only=True)
class Box(Base, IHasUnits, IHasArea, IHasVolume, speckle_type="Objects.Geometry.Box"):
"""
a 3-dimensional box oriented on a plane
A 3-dimensional box oriented on a plane.
This class represents a rectangular prism in 3D space, defined by a base plane and
three intervals specifying its dimensions along the x, y, and z axes.
Attributes:
basePlane: The plane on which the box is oriented
xSize: The interval defining the box's size along the x-axis
ySize: The interval defining the box's size along the y-axis
zSize: The interval defining the box's size along the z-axis
```py title="Example"
from specklepy.objects.geometry.plane import Plane
from specklepy.objects.geometry.point import Point
from specklepy.objects.primitive import Interval
base_plane = Plane(origin=Point(0, 0, 0), normal=Point(0, 0, 1))
x_size = Interval(start=0, end=10)
y_size = Interval(start=0, end=5)
z_size = Interval(start=0, end=3)
box = Box(basePlane=base_plane, xSize=x_size, ySize=y_size, zSize=z_size)
```
"""
basePlane: Plane
@@ -29,6 +51,11 @@ class Box(Base, IHasUnits, IHasArea, IHasVolume, speckle_type="Objects.Geometry.
@property
def area(self) -> float:
"""Calculates the surface area of the box.
Returns:
The total surface area of the box.
"""
return 2 * (
self.xSize.length * self.ySize.length
+ self.xSize.length * self.zSize.length
@@ -37,4 +64,9 @@ class Box(Base, IHasUnits, IHasArea, IHasVolume, speckle_type="Objects.Geometry.
@property
def volume(self) -> float:
"""Calculates the volume of the box.
Returns:
The volume of the box.
"""
return self.xSize.length * self.ySize.length * self.zSize.length
@@ -0,0 +1,13 @@
from specklepy.objects.graph_traversal.traversal import (
DefaultRule,
GraphTraversal,
TraversalContext,
TraversalRule,
)
__all__ = [
"GraphTraversal",
"TraversalContext",
"TraversalRule",
"DefaultRule",
]
@@ -23,6 +23,9 @@ class ITraversalRule(Protocol):
@final
@define(slots=True, frozen=True)
class DefaultRule:
def should_return(self) -> bool:
return False
def get_members_to_traverse(self, _) -> Set[str]:
return set()
@@ -47,6 +50,15 @@ class TraversalContext:
class GraphTraversal:
_rules: List[ITraversalRule]
def _get_active_rule(self, o: Base) -> Optional[ITraversalRule]:
for rule in self._rules:
if rule.does_rule_hold(o):
return rule
return None
def _get_active_rule_or_default_rule(self, o: Base) -> ITraversalRule:
return self._get_active_rule(o) or _default_rule
def traverse(self, root: Base) -> Iterator[TraversalContext]:
stack: List[TraversalContext] = []
@@ -110,15 +122,6 @@ class GraphTraversal:
for o in GraphTraversal.traverse_member(obj):
yield o
def _get_active_rule_or_default_rule(self, o: Base) -> ITraversalRule:
return self._get_active_rule(o) or _default_rule
def _get_active_rule(self, o: Base) -> Optional[ITraversalRule]:
for rule in self._rules:
if rule.does_rule_hold(o):
return rule
return None
@final
@define(slots=True, frozen=True)
@@ -0,0 +1,68 @@
from time import monotonic
from specklepy.core.api.client import SpeckleClient
from specklepy.core.api.inputs.model_ingestion_inputs import ModelIngestionUpdateInput
from specklepy.core.api.models.current import ModelIngestion
class IngestionProgressManager:
"""
Provides a performant way to report ingestion progress.
Allows callers to throttle ingestion progress messages based on an update interval.
Throttling prevents callers from overwhelming the server with constant progress updates,
and minimises blocking high-speed operations with too frequent progress updates.
Callers can use this pattern for reporting throttled progress
```
if progress.should_report_progress():
progress.report(f"Converting geometries ({current:,}/{total:,})", current - total)
```
And for unthrottled progress (e.g. between phases)
```
progress.report(f"Next phases has started (0/{total:,})", 0)
This class is similar to the `IngestionProgressManager` in the .NET SDK
Unlike in .NET, we recommend using a very coarse `update_interval_seconds`; since we're not using async messages,
they are blocking and will degrade performance if used too frequently.
```
""" # noqa: E501
def __init__(
self,
speckle_client: SpeckleClient,
ingestion: ModelIngestion,
update_interval_seconds: float,
):
self.speckle_client = speckle_client
self.ingestion = ingestion
self.update_interval = update_interval_seconds
self._last_updated_at = 0.0
def report(self, progress_message: str, progress: float | None) -> ModelIngestion:
"""
Reports a progress update
"""
self._last_updated_at = monotonic()
formatted_progress = f"{progress:.0%}" if progress else ""
print(f"Progress update: {progress_message} {formatted_progress}")
return self.speckle_client.model_ingestion.update_progress(
ModelIngestionUpdateInput(
ingestion_id=self.ingestion.id,
project_id=self.ingestion.project_id,
progress_message=progress_message,
progress=progress,
)
)
def should_report_progress(self) -> bool:
"""
Returns `true` if it's time for an update,
`false` if it's too soon since the last update
"""
elapsed = monotonic() - self._last_updated_at
return elapsed >= self.update_interval
@@ -0,0 +1,64 @@
from typing import Dict, List
from specklepy.progress.ingestion_progress import IngestionProgressManager
from specklepy.transports.abstract_transport import AbstractTransport
class ProgressTransport(AbstractTransport):
"""
This transport does not persist objects anywhere,
instead it simply reacts to save_object being called,
and reports throttled progress.
"""
def __init__(
self,
progress: IngestionProgressManager,
name="Progress",
progress_message_template="Uploading objects {:,}",
) -> None:
super().__init__()
self._name = name
self._progress = progress
self._progress_message_template = progress_message_template
self.saved_object_count = 0
def _throttle_progress(self) -> None:
if self._progress.should_report_progress():
self._progress.report(
self._progress_message_template.format(self.saved_object_count), None
)
@property
def name(self) -> str:
return self._name
def __repr__(self) -> str:
return f"ProgressTransport(objects: {self.saved_object_count})"
def save_object(self, id: str, serialized_object: str) -> None:
self.saved_object_count += 1
self._throttle_progress()
def save_object_from_transport(
self, id: str, source_transport: AbstractTransport
) -> None:
self.saved_object_count += 1
self._throttle_progress()
def get_object(self, id: str) -> str | None:
return None
def has_objects(self, id_list: List[str]) -> Dict[str, bool]:
return {id: False for id in id_list}
def begin_write(self) -> None:
self.saved_object_count = 0
def end_write(self) -> None:
pass
def copy_object_and_children(
self, id: str, target_transport: AbstractTransport
) -> str:
raise NotImplementedError

Some files were not shown because too many files have changed in this diff Show More