Compare commits

...

21 Commits

Author SHA1 Message Date
Jedd Morgan f4863a89d8 Trying my best to clean up the mess 2026-04-14 18:06:03 +01:00
Jedd Morgan 856f12e57c Don't type check unions recursivly 2026-04-14 16:05:27 +01:00
Jedd Morgan 10e639d19a fail fast false 2026-04-07 20:24:50 +01:00
Jedd Morgan c209cdaec4 test public too 2026-04-07 18:51:17 +01:00
Jedd Morgan 169dd00fac Skip broken test 2026-04-07 18:50:34 +01:00
Jonathon Broughton 58190c378a Add Python version 3.14 to CI workflow 2026-03-30 18:18:31 +01:00
Jonathon Broughton aa16234e7f feat(automate): allow automation results with no affected objects (#488)
* allow empty affected objects

* adds unit tests for `attach_result_to_objects` method

Introduces tests for handling empty object lists and objects with IDs.

Enhances error handling for cases where objects lack IDs, ensuring robustness in the functionality.

Confirms that the method correctly appends results under various scenarios.

* line length
2026-02-24 20:21:59 +00:00
Jonathon Broughton c1f82fa0d2 fix(tests): Update broken test cases for StreamWrapper URLs (#489)
* Update test cases for StreamWrapper URLs

* Update branch name in StreamWrapper test

* Update project URLs in test_wrapper.py

* Uncomment URLs in test_to_string function

Uncommented specific URLs in the test case to enable testing.
2026-02-23 11:29:35 +01:00
Jedd Morgan c53a51c8ad Jrm/can create model ingestion (#486)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* Add `canCreateModelIngestion` model permission check

* format

* oops
2026-01-29 14:23:44 +00:00
Jedd Morgan c1f27b78f9 feat(api)!: Add model permission checks (#485)
* Add model permission checks

* test_public

* This is the real fix

* mistake

* public api resource
2026-01-29 12:04:21 +01:00
Jedd Morgan 49d4b7d44d doc: MarkReceivedVersionInput clarification (#484)
* MarkReceivedVersionInput clarification

* Reformat
2026-01-27 19:52:30 +03:00
Jedd Morgan 7181f50dda update nullability of invitedBy (#483) 2026-01-15 20:06:13 +03:00
Mucahit Bilal GOKER 2f84214786 feat(ifc): add parentId to nested objects (#481)
* add parentId to nested objects

* rename to parentApplicationId

* implement jedd's feedback

* ruff check

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2026-01-08 09:04:12 +03:00
Jedd Morgan 0fe1af8e75 Update PostgreSQL connection string in docker-compose (#482) 2026-01-07 15:54:26 +00:00
Gergő Jedlicska 6297943fe1 gergo/version message for ingestion (#480)
* feat: use mise for docs build

* feat(modelingestion): add version message reporting
2026-01-05 11:46:31 +00:00
Gergő Jedlicska 428bbe2c3d gergo/queryIngestionFix (#479)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* feat: use mise for docs build

* fix: getting the ingestion query needs to use model ingestion id
2025-12-11 10:44:36 +01:00
Jedd Morgan 0ca22891bc fallback to cgal (#476) 2025-12-10 10:09:00 +00:00
Jedd Morgan fd8c2a32f9 chore(speckleifc): changed ifc status messages (#478)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* remove this function

* Changed progress messages
2025-12-09 17:27:26 +00:00
Jedd Morgan ba8c356d82 chore(speckleifc): Ifc metrics slug tweaks (#477)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* ifc metrics

* add http server tests for metrics

* clean up tests

* change back to localhost:3000

* comment

* renamed wrapper for clarity

* fix unrelated model_ingestion
2025-12-09 16:18:21 +01:00
Gergő Jedlicska 8249cd2184 Jedd/cxpla 340 specklepy (#475)
Publish Python Package / test (push) Has been cancelled
Publish Python Package / Build and Publish Python Package (push) Has been cancelled
* First pass

* add tests

* Add cancellation

* fix

* status changes

* fixes

* test fixes

* tests(subscriptions): fix model ingestion tests

* feat(modelingestion): rename resource and add some more tests

* feat(ifcimport): use new modelingestion api

* feat: wrap up new ingestion

* fix: model ingestion payload and test server url

* fix: test port was 3000

* fix: remove version message from model ingestion success input

* fix: test subs cancelled

* ci: signal public of private envv in ci

---------

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-12-08 13:25:54 +01:00
Gergő Jedlicska 7c108a9d43 feat(speckle_automate): version receive metrics (#470)
make sure the automate metrics are attributed to automate host app use
the version metrics to report the version received into automate.

Co-authored-by: Jedd Morgan <45512892+JR-Morgan@users.noreply.github.com>
2025-11-27 13:27:25 +00:00
44 changed files with 1893 additions and 171 deletions
+6
View File
@@ -13,12 +13,14 @@ jobs:
name: Test (internal)
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
steps:
- uses: actions/checkout@v4
@@ -59,13 +61,17 @@ jobs:
test-public: # Run integration tests against the public server image
name: Test (public)
runs-on: ubuntu-latest
env:
IS_PUBLIC: "true"
strategy:
fail-fast: false
matrix:
python-version:
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
steps:
- uses: actions/checkout@v4
+1
View File
@@ -0,0 +1 @@
words = ["specklepy"]
+1 -4
View File
@@ -97,10 +97,7 @@ services:
STRATEGY_LOCAL: "true"
POSTGRES_URL: "postgres"
POSTGRES_USER: "speckle"
POSTGRES_PASSWORD: "speckle"
POSTGRES_DB: "speckle"
POSTGRES_URL: 'postgres://speckle:speckle@postgres:5432/speckle'
ENABLE_MP: "false"
LOG_PRETTY: "true"
+8 -1
View File
@@ -8,7 +8,7 @@ python.uv_venv_auto = true
[tasks.install]
run= "uv sync --all-groups"
run= "uv sync --all-extras --all-groups"
[tasks.install_docs]
@@ -18,3 +18,10 @@ run= "uv sync --group docs"
description = "Build static docs "
run = "uv run mkdocs build"
depends = ['install_docs']
[tasks.test]
run = "uv run pytest"
[env]
IS_PUBLIC = "false"
+2
View File
@@ -22,6 +22,7 @@ dependencies = [
speckleifc = ["ifcopenshell>=0.8.3.post2"]
[dependency-groups]
dev = [
"commitizen>=4.1.0",
"devtools>=0.12.2",
@@ -32,6 +33,7 @@ dev = [
"pytest-asyncio>=0.25.2",
"pytest-cov>=6.0.0",
"pytest-ordering>=0.6",
"pytest_httpserver >=1.1.3",
"ruff==0.9.2",
"types-deprecated>=1.2.15.20241117",
"types-requests>=2.32.0.20241016",
+24 -11
View File
@@ -19,8 +19,12 @@ from speckle_automate.schema import (
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.version_inputs import CreateVersionInput
from specklepy.core.api.inputs.version_inputs import (
CreateVersionInput,
MarkReceivedVersionInput,
)
from specklepy.core.api.models.current import Model, Version
from specklepy.logging import metrics
from specklepy.logging.exceptions import SpeckleException
from specklepy.objects.base import Base
from specklepy.transports.memory import MemoryTransport
@@ -66,6 +70,7 @@ class AutomationContext:
if isinstance(automation_run_data, AutomationRunData)
else AutomationRunData.model_validate_json(automation_run_data)
)
metrics.set_host_app("automate")
speckle_client = SpeckleClient(
automation_run_data.speckle_server_url,
automation_run_data.speckle_server_url.startswith("https"),
@@ -100,6 +105,7 @@ class AutomationContext:
"""Receive the Speckle project version that triggered this automation run."""
# TODO: this is a quick hack to keep implementation consistency.
# Move to proper receive many versions
project_id = self.automation_run_data.project_id
version_id = self.automation_run_data.triggers[0].payload.version_id
try:
version = self.speckle_client.version.get(
@@ -109,7 +115,7 @@ class AutomationContext:
raise ValueError(
f"""Could not receive specified version.
Is your environment configured correctly?
project_id: {self.automation_run_data.project_id}
project_id: {project_id}
model_id: {self.automation_run_data.triggers[0].payload.model_id}
version_id: {self.automation_run_data.triggers[0].payload.version_id}
"""
@@ -124,6 +130,13 @@ class AutomationContext:
base = operations.receive(
version.referenced_object, self._server_transport, self._memory_transport
)
self.speckle_client.version.received(
MarkReceivedVersionInput(
version_id=version_id,
project_id=project_id,
source_application="automate_function",
)
)
# self._closure_tree = base["__closure"]
print(
f"It took {self.elapsed():.2f} seconds to receive",
@@ -480,29 +493,29 @@ class AutomationContext:
Args:
level: Result level.
category (str): A short tag for the event type.
affected_objects (Union[Base, List[Base]]): A single object or a list of
objects that are causing the info case.
affected_objects (Union[Base, List[Base]]): A single object, a list of
objects, or an empty list. When empty, a result case is still
appended with no object IDs (e.g. for skipped rules or version-level
messages).
message (Optional[str]): Optional message.
metadata: User provided metadata key value pairs
visual_overrides: Case specific 3D visual overrides.
"""
if isinstance(affected_objects, list):
if len(affected_objects) < 1:
raise ValueError(
f"Need atleast one object to report a(n) {level.value.upper()}"
)
object_list = affected_objects
else:
object_list = [affected_objects]
ids: Dict[str, Optional[str]] = {}
# When objects are provided, each must have an id (empty list allowed for
# version-level/skipped results).
for o in object_list:
# validate that the Base.id is not None. If its a None, throw an Exception
if not o.id:
if not getattr(o, "id", None):
raise Exception(
f"You can only attach {level} results to objects with an id."
)
ids[o.id] = o.applicationId
ids[o.id] = getattr(o, "applicationId", None)
print(
f"Created new {level.value.upper()}"
f" category: {category} caused by: {message}"
+6 -3
View File
@@ -18,7 +18,7 @@ def cmd_line_import() -> None:
parser.add_argument("output_path")
parser.add_argument("project_id")
parser.add_argument("version_message")
parser.add_argument("model_id")
parser.add_argument("model_ingestion_id")
# parser.add_argument("model_name")
# parser.add_argument("region_name")
@@ -32,6 +32,8 @@ def cmd_line_import() -> None:
"ifc",
)
client: SpeckleClient | None = None
try:
client = SpeckleClient(SERVER_URL, use_ssl=not SERVER_URL.startswith("http://"))
client.authenticate_with_token(TOKEN)
@@ -41,13 +43,14 @@ def cmd_line_import() -> None:
args.file_path,
project,
args.version_message,
args.model_id,
args.model_ingestion_id,
client,
)
with open(args.output_path, "w") as f:
json.dump({"success": True, "commitId": version.id}, f)
except Exception as e:
error_msg = f"IFC Importer failed with exception:\n{traceback.format_exc()}"
stack_trace = traceback.format_exc()
error_msg = f"IFC Importer failed with exception:\n{stack_trace}"
print(error_msg)
# Write error result
@@ -12,12 +12,23 @@ def data_object_to_speckle(
step_element: entity_instance,
children: list[Base],
current_storey: str | None = None,
parent_element: entity_instance | None = None,
) -> DataObject:
guid = cast(str, step_element.GlobalId)
name = cast(str, step_element.Name or guid)
properties = extract_properties(step_element)
# Add parent ID only if element's parent is also a DataObject (not a Collection)
# Collections are: IfcProject and IfcSpatialStructureElement types
if (
parent_element
and hasattr(parent_element, "GlobalId")
and not parent_element.is_a("IfcProject")
and not parent_element.is_a("IfcSpatialStructureElement")
):
properties["parentApplicationId"] = parent_element.GlobalId
# Add building storey information if available and not a building storey itself
if current_storey and not step_element.is_a("IfcBuildingStorey"):
properties["Building Storey"] = current_storey
+7 -1
View File
@@ -51,4 +51,10 @@ def open_ifc(file_path: str) -> file:
def create_geometry_iterator(ifc_file: file | sqlite) -> iterator:
return iterator(_create_iterator_settings(), ifc_file, multiprocessing.cpu_count())
GEOMETRY_LIBRARY = "hybrid-opencascade-cgal" # First OCC then fallback to CGAL
return iterator(
_create_iterator_settings(),
ifc_file,
multiprocessing.cpu_count(),
geometry_library=GEOMETRY_LIBRARY, # type: ignore
)
+18 -6
View File
@@ -44,9 +44,13 @@ class ImportJob:
_display_value_cache: dict[int, list[Base]] = field(default_factory=dict)
"""Maps an instance step ID to a list of instances"""
def convert_element(self, step_element: entity_instance) -> Base:
def convert_element(
self,
step_element: entity_instance,
parent_element: entity_instance | None = None,
) -> Base:
try:
return self._convert_element(step_element)
return self._convert_element(step_element, parent_element)
except SpeckleException:
raise
except Exception as ex:
@@ -54,14 +58,18 @@ class ImportJob:
f"Failed to convert {step_element.is_a()} #{step_element.id()}"
) from ex
def _convert_element(self, step_element: entity_instance) -> Base:
def _convert_element(
self,
step_element: entity_instance,
parent_element: entity_instance | None = None,
) -> Base:
# Track current storey context and store for level proxies
previous_storey_data_object = self._current_storey_data_object
if step_element.is_a("IfcBuildingStorey"):
# Convert the building storey to a DataObject for the level proxy
storey_display_value = self._display_value_cache.get(step_element.id(), [])
self._current_storey_data_object = data_object_to_speckle(
storey_display_value, step_element, []
storey_display_value, step_element, [], parent_element=None
)
children = self._convert_children(step_element)
@@ -86,7 +94,11 @@ class ImportJob:
)
else:
result = data_object_to_speckle(
display_value, step_element, children, current_storey_name
display_value,
step_element,
children,
current_storey_name,
parent_element,
)
# Associate non-spatial elements with current storey for level proxies
if self._current_storey_data_object is not None and result.applicationId:
@@ -100,7 +112,7 @@ class ImportJob:
def _convert_children(self, step_element: entity_instance) -> list[Base]:
return [
self.convert_element(i)
self.convert_element(i, parent_element=step_element)
for i in get_children(step_element)
if self._should_convert(i)
]
+101 -34
View File
@@ -1,9 +1,19 @@
import contextlib
import importlib.metadata
import time
import traceback
from pathlib import Path
from speckleifc.ifc_geometry_processing import open_ifc
from speckleifc.importer import ImportJob
from specklepy.core.api.client import SpeckleClient
from specklepy.core.api.inputs.version_inputs import CreateVersionInput
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionFailedInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
SourceDataInput,
)
from specklepy.core.api.models.current import Project, Version
from specklepy.core.api.operations import send
from specklepy.logging import metrics
@@ -13,49 +23,106 @@ from specklepy.transports.server import ServerTransport
def open_and_convert_file(
file_path: str,
project: Project,
version_message: str | None,
model_id: str,
version_message: str,
model_ingestion_id: str,
client: SpeckleClient,
) -> Version:
start = time.time()
very_start = start
try:
start = time.time()
very_start = start
path = Path(file_path)
account = client.account
server_url = account.serverInfo.url
assert server_url
remote_transport = ServerTransport(project.id, account=account)
specklepy_version = importlib.metadata.version("specklepy")
client.model_ingestion.start_processing(
ModelIngestionStartProcessingInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
progress_message="Importing IFC file",
source_data=SourceDataInput(
file_name=path.name,
file_size_bytes=path.stat().st_size,
source_application_slug=metrics.HOST_APP,
source_application_version=specklepy_version,
),
)
)
ifc_file = open_ifc(file_path) # pyright: ignore[reportUnknownVariableType]
account = client.account
server_url = account.serverInfo.url
assert server_url
remote_transport = ServerTransport(project.id, account=account)
import_job = ImportJob(ifc_file) # pyright: ignore[reportUnknownArgumentType]
data = import_job.convert()
ifc_file = open_ifc(file_path) # pyright: ignore[reportUnknownVariableType]
print(f"File conversion complete after {(time.time() - start) * 1000}ms")
client.model_ingestion.update_progress(
ModelIngestionUpdateInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
progress_message="Converting file",
progress=None,
)
)
import_job = ImportJob(ifc_file) # pyright: ignore[reportUnknownArgumentType]
data = import_job.convert()
start = time.time()
print(f"File conversion complete after {(time.time() - start) * 1000}ms")
root_id = send(data, transports=[remote_transport], use_default_cache=False)
print(f"Sending to speckle complete after: {(time.time() - start) * 1000}ms")
start = time.time()
start = time.time()
client.model_ingestion.update_progress(
ModelIngestionUpdateInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
progress_message="Uploading objects",
progress=None,
)
)
root_id = send(data, transports=[remote_transport], use_default_cache=False)
print(f"Sending to speckle complete after: {(time.time() - start) * 1000}ms")
create_version = CreateVersionInput(
object_id=root_id,
model_id=model_id,
project_id=project.id,
message=version_message,
source_application="ifc",
)
version = client.version.create(create_version)
end = time.time()
print(f"Version committed after: {(end - start) * 1000}ms")
start = time.time()
print(f"Total time (to commit): {(end - very_start) * 1000}ms")
del ifc_file
version_id = client.model_ingestion.complete(
ModelIngestionSuccessInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
root_object_id=root_id,
version_message=version_message,
)
)
custom_properties = {"ui": "dui3", "actionSource": "import"}
if project.workspace_id:
custom_properties["workspace_id"] = project.workspace_id
metrics.track(metrics.SEND, account, custom_properties, send_sync=True)
# needed to query version until ingestion api expands to serve it
version = client.version.get(version_id, project.id)
return version
end = time.time()
print(f"Version committed after: {(end - start) * 1000}ms")
print(f"Total time (to commit): {(end - very_start) * 1000}ms")
del ifc_file
custom_properties = {"ui": "dui3", "actionSource": "import"}
if project.workspace_id:
custom_properties["workspace_id"] = project.workspace_id
metrics.track(
metrics.SEND,
account,
custom_properties,
send_sync=True,
track_email=True,
)
return version
except Exception as e:
stack_trace = traceback.format_exc()
with contextlib.suppress(Exception):
# make sure to not report process kills when we're cancelling
client.model_ingestion.fail_with_error(
ModelIngestionFailedInput(
project_id=project.id,
ingestion_id=model_ingestion_id,
error_reason=str(e),
error_stacktrace=stack_trace,
)
)
raise e
+7
View File
@@ -4,6 +4,7 @@ from specklepy.api.credentials import Account
from specklepy.api.resources import (
ActiveUserResource,
FileImportResource,
ModelIngestionResource,
ModelResource,
OtherUserResource,
ProjectInviteResource,
@@ -119,6 +120,12 @@ class SpeckleClient(CoreSpeckleClient):
client=self.httpclient,
server_version=server_version,
)
self.model_ingestion = ModelIngestionResource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.file_import = FileImportResource(
account=self.account,
basepath=self.url,
+5
View File
@@ -1,5 +1,8 @@
from specklepy.api.resources.current.active_user_resource import ActiveUserResource
from specklepy.api.resources.current.file_import_resource import FileImportResource
from specklepy.api.resources.current.model_ingestion_resource import (
ModelIngestionResource,
)
from specklepy.api.resources.current.model_resource import ModelResource
from specklepy.api.resources.current.other_user_resource import OtherUserResource
from specklepy.api.resources.current.project_invite_resource import (
@@ -22,4 +25,6 @@ __all__ = [
"SubscriptionResource",
"VersionResource",
"WorkspaceResource",
"FileImportResource",
"ModelIngestionResource",
]
@@ -15,7 +15,7 @@ from specklepy.logging import metrics
class FileImportResource(CoreResource):
"""API Access class for projects"""
"""API Access class for file imports"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
@@ -0,0 +1,57 @@
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
)
from specklepy.core.api.models.current import (
ModelIngestion,
)
from specklepy.core.api.resources import (
ModelIngestionResource as CoreResource,
)
from specklepy.logging import metrics
class ModelIngestionResource(CoreResource):
"""API Access class for model ingestion"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(account, basepath, client, server_version)
def create(self, input: ModelIngestionCreateInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Create"})
return super().create(input)
def get_ingestion(self, project_id: str, model_ingestion_id: str) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Get"})
return super().get_ingestion(project_id, model_ingestion_id)
def update_progress(self, input: ModelIngestionUpdateInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Update"})
return super().update_progress(input)
def start_processing(
self, input: ModelIngestionStartProcessingInput
) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Start Processing"})
return super().start_processing(input)
def requeue(self, input: ModelIngestionRequeueInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Update"})
return super().requeue(input)
def complete(self, input: ModelIngestionSuccessInput) -> str:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion End"})
return super().complete(input)
def fail_with_error(self, input: ModelIngestionFailedInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Error"})
return super().fail_with_error(input)
def fail_with_cancel(self, input: ModelIngestionCancelledInput) -> ModelIngestion:
metrics.track(metrics.SDK, self.account, {"name": "Ingestion Cancel"})
return super().fail_with_cancel(input)
@@ -8,6 +8,10 @@ from specklepy.core.api.inputs.model_inputs import (
)
from specklepy.core.api.inputs.project_inputs import ProjectModelsFilter
from specklepy.core.api.models import Model, ModelWithVersions, ResourceCollection
from specklepy.core.api.models.current import (
ModelPermissionChecks,
PermissionCheckResult,
)
from specklepy.core.api.resources import ModelResource as CoreResource
from specklepy.logging import metrics
@@ -72,3 +76,17 @@ class ModelResource(CoreResource):
def update(self, input: UpdateModelInput) -> Model:
metrics.track(metrics.SDK, self.account, {"name": "Model Update"})
return super().update(input)
def get_permissions(self, model_id: str, project_id: str) -> ModelPermissionChecks:
metrics.track(metrics.SDK, self.account, {"name": "Model Get Permissions"})
return super().get_permissions(model_id, project_id)
def can_create_model_ingestion(
self, model_id: str, project_id: str
) -> PermissionCheckResult:
metrics.track(
metrics.SDK,
self.account,
{"name": "Model Get Permissions canCreateIngestion"},
)
return super().can_create_model_ingestion(model_id, project_id)
+7
View File
@@ -12,6 +12,7 @@ from specklepy.core.api.credentials import Account
from specklepy.core.api.resources import (
ActiveUserResource,
FileImportResource,
ModelIngestionResource,
ModelResource,
OtherUserResource,
ProjectInviteResource,
@@ -250,6 +251,12 @@ class SpeckleClient:
client=self.httpclient,
server_version=server_version,
)
self.model_ingestion = ModelIngestionResource(
account=self.account,
basepath=self.url,
client=self.httpclient,
server_version=server_version,
)
self.subscription = SubscriptionResource(
account=self.account,
basepath=self.ws_url,
+16
View File
@@ -7,6 +7,7 @@ class ProjectVisibility(str, Enum):
PRIVATE = "PRIVATE"
PUBLIC = "PUBLIC"
UNLISTED = "UNLISTED"
"""Deprecated, use PUBLIC instead"""
WORKSPACE = "WORKSPACE"
@@ -30,3 +31,18 @@ class ProjectVersionsUpdatedMessageType(str, Enum):
CREATED = "CREATED"
DELETED = "DELETED"
UPDATED = "UPDATED"
class ProjectModelIngestionUpdatedMessageType(str, Enum):
CANCELLATION_REQUESTED = "cancellationRequested"
CREATED = "created"
DELETED = "deleted"
UPDATED = "updated"
class ModelIngestionStatus(str, Enum):
CANCELLED = "cancelled"
FAILED = "failed"
PROCESSING = "processing"
QUEUED = "queued"
SUCCESS = "success"
@@ -0,0 +1,77 @@
from specklepy.core.api.enums import ProjectModelIngestionUpdatedMessageType
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
class SourceDataInput(GraphQLBaseModel):
source_application_slug: str
source_application_version: str
file_name: str | None
file_size_bytes: int | None
class ModelIngestionCreateInput(GraphQLBaseModel):
model_id: str
project_id: str
progress_message: str
source_data: SourceDataInput
class ModelIngestionStartProcessingInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress_message: str
source_data: SourceDataInput
class ModelIngestionRequeueInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress_message: str
class ModelIngestionUpdateInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
progress: float | None
progress_message: str
class ModelIngestionSuccessInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
root_object_id: str
version_message: str | None
class ModelIngestionFailedInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
error_reason: str
error_stacktrace: str | None
class ModelIngestionCancelledInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
cancellation_message: str
class ModelIngestionRequestCancellationInput(GraphQLBaseModel):
ingestion_id: str
project_id: str
cancellation_message: str
class ModelIngestionReference(GraphQLBaseModel):
"""
`@oneOf` i.e. server expects **either** `ingestion_id` or `model_id`, but not both.
"""
ingestion_id: str | None
model_id: str | None
class ProjectModelIngestionSubscriptionInput(GraphQLBaseModel):
project_id: str
ingestion_reference: ModelIngestionReference
message_type: ProjectModelIngestionUpdatedMessageType | None = None
@@ -34,4 +34,8 @@ class MarkReceivedVersionInput(GraphQLBaseModel):
version_id: str
project_id: str
source_application: str
"""
IMPORTANT: this is meant to be the slug of the application that has done the
receiving, not to be confused with `Version.sourceApplication`
"""
message: Optional[str] = None
+22 -3
View File
@@ -1,7 +1,7 @@
from datetime import datetime
from typing import Generic, List, TypeVar
from specklepy.core.api.enums import ProjectVisibility
from specklepy.core.api.enums import ModelIngestionStatus, ProjectVisibility
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
from specklepy.logging.exceptions import WorkspacePermissionException
@@ -105,7 +105,7 @@ class PendingStreamCollaborator(GraphQLBaseModel):
project_name: str
title: str
role: str
invited_by: LimitedUser
invited_by: LimitedUser | None = None
user: LimitedUser | None = None
token: str | None
@@ -137,6 +137,12 @@ class Version(GraphQLBaseModel):
source_application: str | None
class ModelPermissionChecks(GraphQLBaseModel):
can_update: "PermissionCheckResult"
can_delete: "PermissionCheckResult"
can_create_version: "PermissionCheckResult"
class Model(GraphQLBaseModel):
author: LimitedUser | None
created_at: datetime
@@ -156,7 +162,6 @@ class ProjectPermissionChecks(GraphQLBaseModel):
can_create_model: "PermissionCheckResult"
can_delete: "PermissionCheckResult"
can_load: "PermissionCheckResult"
can_publish: "PermissionCheckResult"
class Project(GraphQLBaseModel):
@@ -244,3 +249,17 @@ class FileImport(GraphQLBaseModel):
class FileUploadUrl(GraphQLBaseModel):
url: str
file_id: str
class ModelIngestionStatusData(GraphQLBaseModel):
status: ModelIngestionStatus
progress_message: str | None = None
class ModelIngestion(GraphQLBaseModel):
id: str
created_at: datetime
updated_at: datetime
cancellation_requested: bool
model_id: str
status_data: ModelIngestionStatusData
@@ -1,12 +1,13 @@
from typing import Optional
from specklepy.core.api.enums import (
ProjectModelIngestionUpdatedMessageType,
ProjectModelsUpdatedMessageType,
ProjectUpdatedMessageType,
ProjectVersionsUpdatedMessageType,
UserProjectsUpdatedMessageType,
)
from specklepy.core.api.models.current import Model, Project, Version
from specklepy.core.api.models.current import Model, ModelIngestion, Project, Version
from specklepy.core.api.models.graphql_base_model import GraphQLBaseModel
@@ -33,3 +34,8 @@ class ProjectVersionsUpdatedMessage(GraphQLBaseModel):
type: ProjectVersionsUpdatedMessageType
model_id: str
version: Optional[Version]
class ProjectModelIngestionUpdatedMessage(GraphQLBaseModel):
model_ingestion: ModelIngestion
type: ProjectModelIngestionUpdatedMessageType
@@ -1,5 +1,8 @@
from specklepy.core.api.resources.current.active_user_resource import ActiveUserResource
from specklepy.core.api.resources.current.file_import_resource import FileImportResource
from specklepy.core.api.resources.current.model_ingestion_resource import (
ModelIngestionResource,
)
from specklepy.core.api.resources.current.model_resource import ModelResource
from specklepy.core.api.resources.current.other_user_resource import OtherUserResource
from specklepy.core.api.resources.current.project_invite_resource import (
@@ -24,4 +27,6 @@ __all__ = [
"SubscriptionResource",
"VersionResource",
"WorkspaceResource",
"FileImportResource",
"ModelIngestionResource",
]
@@ -16,13 +16,15 @@ from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
from specklepy.logging.exceptions import SpeckleException
NAME = "file_import"
class UploadFileResponse(GraphQLBaseModel):
etag: str
class FileImportResource(ResourceBase):
"""API Access class for project invites"""
"""API Access class for file imports"""
def __init__(
self,
@@ -36,7 +38,7 @@ class FileImportResource(ResourceBase):
basepath=basepath,
client=client,
server_version=server_version,
name="file-import",
name=NAME,
)
def finish_file_import_job(self, input: FinishFileImportInput) -> bool:
@@ -0,0 +1,397 @@
from typing import Any, Tuple
from gql import Client, gql
from specklepy.api.credentials import Account
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequestCancellationInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
)
from specklepy.core.api.models.current import (
ModelIngestion,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
NAME = "ingestion"
class ModelIngestionResource(ResourceBase):
"""API Access class for model ingestion"""
def __init__(
self,
account: Account,
basepath: str,
client: Client,
server_version: Tuple[Any, ...] | None,
) -> None:
super().__init__(
account=account,
basepath=basepath,
client=client,
name=NAME,
server_version=server_version,
)
def get_ingestion(self, project_id: str, model_ingestion_id: str) -> ModelIngestion:
QUERY = gql(
"""
query Query($projectId: String!, $modelIngestionId: ID!) {
data:project(id: $projectId) {
data:ingestion(id: $modelIngestionId) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
""" # noqa: E501
)
variables = {
"projectId": project_id,
"modelIngestionId": model_ingestion_id,
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[ModelIngestion]],
QUERY,
variables,
).data.data
def create(self, input: ModelIngestionCreateInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionCreate($input: ModelIngestionCreateInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: create(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def start_processing(
self, input: ModelIngestionStartProcessingInput
) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionStartProcessing($input: ModelIngestionStartProcessingInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: startProcessing(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def requeue(self, input: ModelIngestionRequeueInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionRequeue($input: ModelIngestionRequeueInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: requeue(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def update_progress(self, input: ModelIngestionUpdateInput) -> ModelIngestion:
QUERY = gql(
"""
mutation IngestionUpdateProgress(
$input: ModelIngestionUpdateInput!
) {
data: projectMutations {
data: modelIngestionMutations {
data: updateProgress(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]], QUERY, variables
).data.data.data
def complete(self, input: ModelIngestionSuccessInput) -> str:
"""
Request that the server completes the ingestion by creating a version
If successful, the job will be in a terminal "successful" state.
For failed Ingestions, use `fail_with_error` instead
For user cancellation, use `fail_with_cancelled` instead
Arguments:
input {ModelIngestionSuccessInput} -- input variable
Returns:
str -- the id of the version that was just created to complete the ingestion
"""
QUERY = gql(
"""
mutation IngestionComplete($input: ModelIngestionSuccessInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: completeWithVersion(input: $input) {
data:statusData {
... on ModelIngestionSuccessStatus {
data:versionId
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[DataResponse[DataResponse[str]]]]],
QUERY,
variables,
).data.data.data.data.data
def fail_with_error(self, input: ModelIngestionFailedInput) -> ModelIngestion:
"""
Fail the job with an error.
For user requested cancellation, use `fail_with_cancelled` instead
"""
QUERY = gql(
"""
mutation IngestionFailWithError($input: ModelIngestionFailedInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: failWithError(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
def fail_with_cancel(self, input: ModelIngestionCancelledInput) -> ModelIngestion:
"""
Fail the ingestion with a `cancelled` status.
This should only be done if the user has explicitly requested cancellation
Other forms of cancellation use `fail_with_error`
The ingestion should then enter a terminal "canceled" state
"""
QUERY = gql(
"""
mutation IngestionFailWithCancel($input: ModelIngestionCancelledInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: failWithCancel(input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
"""
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
def request_cancellation(
self, input: ModelIngestionRequestCancellationInput
) -> ModelIngestion:
"""
Request that the ingestion is canceled.
Note: simply calling this mutation does not immediately cancel,
it doesn't even guarantee it will be canceled at all.
It's up to the client to observe this cancellation request
via `subscription.project_model_ingestion_cancellation_requested`
and report it as cancelled (via `ingestion.fail_with_cancel`
See "cooperative cancellation pattern"
"""
QUERY = gql(
"""
mutation IngestionRequestCancellation($input: ModelIngestionRequestCancellationInput!) {
data: projectMutations {
data: modelIngestionMutations {
data: requestCancellation (input: $input) {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
}
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(warnings="error", by_alias=True),
}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelIngestion]]],
QUERY,
variables,
).data.data.data
@@ -10,6 +10,10 @@ from specklepy.core.api.inputs.model_inputs import (
)
from specklepy.core.api.inputs.project_inputs import ProjectModelsFilter
from specklepy.core.api.models import Model, ModelWithVersions, ResourceCollection
from specklepy.core.api.models.current import (
ModelPermissionChecks,
PermissionCheckResult,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
@@ -299,3 +303,71 @@ class ModelResource(ResourceBase):
return self.make_request_and_parse_response(
DataResponse[DataResponse[Model]], QUERY, variables
).data.data
def get_permissions(self, project_id: str, model_id: str) -> ModelPermissionChecks:
QUERY = gql(
"""
query ModelPermissions($projectId: String!, $modelId: String!) {
data:project(id: $projectId) {
data:model(id: $modelId) {
data:permissions {
canUpdate {
authorized
code
message
}
canDelete {
authorized
code
message
}
canCreateVersion {
authorized
code
message
}
}
}
}
}
"""
)
variables = {"projectId": project_id, "modelId": model_id}
return self.make_request_and_parse_response(
DataResponse[DataResponse[DataResponse[ModelPermissionChecks]]],
QUERY,
variables,
).data.data.data
def can_create_model_ingestion(
self, project_id: str, model_id: str
) -> PermissionCheckResult:
QUERY = gql(
"""
query ModelPermissions($projectId: String!, $modelId: String!) {
data:project(id: $projectId) {
data:model(id: $modelId) {
data:permissions {
data:canCreateIngestion {
authorized
code
message
}
}
}
}
}
"""
)
variables = {"projectId": project_id, "modelId": model_id}
return self.make_request_and_parse_response(
DataResponse[
DataResponse[DataResponse[DataResponse[PermissionCheckResult]]]
],
QUERY,
variables,
).data.data.data.data
@@ -6,17 +6,25 @@ from graphql import DocumentNode
from pydantic import BaseModel
from typing_extensions import TypeVar
from specklepy.core.api.enums import ProjectModelIngestionUpdatedMessageType
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionReference,
ProjectModelIngestionSubscriptionInput,
)
from specklepy.core.api.models import (
ProjectModelsUpdatedMessage,
ProjectUpdatedMessage,
ProjectVersionsUpdatedMessage,
UserProjectsUpdatedMessage,
)
from specklepy.core.api.models.subscription_messages import (
ProjectModelIngestionUpdatedMessage,
)
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
from specklepy.logging.exceptions import SpeckleException
NAME = "subscribe"
NAME = "subscription"
TEventArgs = TypeVar("TEventArgs", bound=BaseModel)
@@ -202,6 +210,66 @@ class SubscriptionResource(ResourceBase):
callback=lambda d: callback(d.data),
)
async def project_model_ingestion_updated(
self,
callback: Callable[[ProjectModelIngestionUpdatedMessage], None],
input: ProjectModelIngestionSubscriptionInput,
) -> None:
QUERY = gql(
"""
subscription IngestionUpdated($input: ProjectModelIngestionSubscriptionInput!) {
data: projectModelIngestionUpdated(input: $input) {
modelIngestion {
id
createdAt
updatedAt
modelId
cancellationRequested
statusData {
... on HasModelIngestionStatus {
status
}
... on HasProgressMessage {
progressMessage
}
}
}
type
}
}
""" # noqa: E501
)
variables = {
"input": input.model_dump(
warnings="error", by_alias=True, exclude_none=True
),
}
await self.subscribe_2(
DataResponse[ProjectModelIngestionUpdatedMessage],
QUERY,
variables,
callback=lambda d: callback(d.data),
)
async def project_model_ingestion_cancellation_requested(
self,
callback: Callable[[ProjectModelIngestionUpdatedMessage], None],
project_id: str,
ingestion_id: str,
) -> None:
await self.project_model_ingestion_updated(
callback,
ProjectModelIngestionSubscriptionInput(
project_id=project_id,
ingestion_reference=ModelIngestionReference(
ingestion_id=ingestion_id, model_id=None
),
message_type=ProjectModelIngestionUpdatedMessageType.CANCELLATION_REQUESTED,
),
)
@check_wsclient
async def subscribe_2(
self,
@@ -14,7 +14,7 @@ from specklepy.core.api.models import ResourceCollection, Version
from specklepy.core.api.resource import ResourceBase
from specklepy.core.api.responses import DataResponse
NAME = "model"
NAME = "version"
class VersionResource(ResourceBase):
@@ -16,7 +16,7 @@ NAME = "workspace"
class WorkspaceResource(ResourceBase):
"""API Access class for models"""
"""API Access class for workspaces"""
def __init__(self, account, basepath, client, server_version) -> None:
super().__init__(
+41 -7
View File
@@ -1,19 +1,20 @@
import contextlib
import getpass
import hashlib
import importlib.metadata
import logging
import platform
import queue
import sys
import threading
from typing import Any
from typing import Any, Literal
import requests
from specklepy.core.api.credentials import Account
"""
Anonymous telemetry to help us understand how to make a better Speckle.
Lightweight usage telemetry to help us understand how to make a better Speckle.
This really helps us to deliver a better open source project and product!
"""
TRACK = True
@@ -22,13 +23,14 @@ HOST_APP_VERSION = f"python {'.'.join(map(str, sys.version_info[:2]))}"
PLATFORMS = {"win32": "Windows", "cygwin": "Windows", "darwin": "Mac OS X"}
LOG = logging.getLogger(__name__)
METRICS_TRACKER = None
METRICS_TRACKER: "MetricsTracker | None" = None
# actions
SDK = "SDK Action"
CONNECTOR = "Connector Action"
RECEIVE = "Receive"
SEND = "Send"
ACTIONS = Literal["SDK Action", "Connector Action", "Receive", "Send"]
def disable():
@@ -48,15 +50,32 @@ def set_host_app(host_app: str, host_app_version: str | None = None):
def track(
action: str,
action: ACTIONS,
account: Account | None = None,
custom_props: dict | None = None,
send_sync: bool = False,
track_email: bool = False,
):
"""
:param action:
:type action: ACTIONS
:param account:
:type account: Account | None
:param custom_props:
:type custom_props: dict | None
:param send_sync: When `True`, the track event is executed synchronously,
and any exceptions will be raised.
When `False`, the track it is deferred to a queue, and any exceptions will be
swallowed and reported as warnings.
:type send_sync: bool
:param track_email: When `True`, the users plain text email address will be included
:type track_email: bool
"""
if not TRACK:
return
tracker = initialise_tracker(account)
event_params: dict[str, Any] = {
"event": action,
"properties": {
@@ -72,6 +91,18 @@ def track(
if custom_props:
event_params["properties"].update(custom_props)
if track_email:
event_params["properties"]["email"] = tracker.last_email
try:
specklepy_version = importlib.metadata.version("specklepy")
event_params["properties"]["core_version"] = specklepy_version
except importlib.metadata.PackageNotFoundError:
if send_sync:
raise
else:
LOG.warning("Failed to read specklepy's version number", exc_info=True)
if send_sync:
tracker.send_event(event_params)
else:
@@ -84,7 +115,7 @@ def initialise_tracker(account: Account | None = None) -> "MetricsTracker":
METRICS_TRACKER = MetricsTracker()
if account:
METRICS_TRACKER.set_last_user(account.userInfo.email)
METRICS_TRACKER.set_last_user_email(account.userInfo.email)
METRICS_TRACKER.set_last_server(account.serverInfo.url)
return METRICS_TRACKER
@@ -103,6 +134,7 @@ class MetricsTracker(metaclass=Singleton):
analytics_url: str = "https://analytics.speckle.systems/track?ip=1"
analytics_token: str = "acd87c5a50b56df91a795e999812a3a4"
last_user: str = ""
last_email: str = ""
last_server: str | None = None
platform: str
@@ -121,17 +153,19 @@ class MetricsTracker(metaclass=Singleton):
if node and user:
self.last_user = f"@{self.hash(f'{node}-{user}')}"
def set_last_user(self, email: str | None) -> None:
def set_last_user_email(self, email: str | None) -> None:
if not email:
return
self.last_user = f"@{self.hash(email)}"
self.last_email = email
def set_last_server(self, server: str | None) -> None:
if not server:
return
self.last_server = self.hash(server)
def hash(self, value: str) -> str:
@staticmethod
def hash(value: str) -> str:
inputList = value.lower().split("://")
input = inputList[len(inputList) - 1].split("/")[0].split("?")[0]
return hashlib.md5(input.encode("utf-8")).hexdigest().upper()
+31 -30
View File
@@ -1,7 +1,7 @@
import contextlib
from dataclasses import dataclass, field
from enum import Enum
from inspect import isclass
from types import UnionType
from typing import (
Any,
ClassVar,
@@ -13,11 +13,13 @@ from typing import (
Tuple,
Type,
Union,
get_origin,
get_type_hints,
)
from warnings import warn
from pydantic.alias_generators import to_pascal
from typing_extensions import get_args
from specklepy.logging.exceptions import SpeckleException
from specklepy.transports.memory import MemoryTransport
@@ -220,32 +222,28 @@ def _validate_type(t: Optional[type], value: Any) -> Tuple[bool, Any]:
if value in t._value2member_map_:
return True, t(value)
if getattr(t, "__module__", None) == "typing":
if isinstance(t, ForwardRef):
return True, value
if isinstance(t, ForwardRef):
return True, value
origin = t.__origin__
# below is what in nicer for >= py38
# origin = get_origin(t)
if getattr(t, "__module__", None) in ["typing", "types"]:
origin = get_origin(t)
args = get_args(t)
# recursive validation for Unions on both types preferring the fist type
if origin is Union:
# below is what in nicer for >= py38
# t_1, t_2 = get_args(t)
args = t.__args__ # type: ignore
if origin is Union or isinstance(t, UnionType):
for arg_t in args:
t_success, t_value = _validate_type(arg_t, value)
if t_success:
return True, t_value
ok, v = _validate_type(arg_t, value)
if ok:
return True, v
return False, value
if origin is dict:
if not isinstance(value, dict):
return False, value
if value == {}:
if not value:
return True, value
if not getattr(t, "__args__", None):
if not args:
return True, value
t_key, t_value = t.__args__ # type: ignore
t_key, t_value = args
if (
getattr(t_key, "__name__", None),
@@ -265,11 +263,11 @@ def _validate_type(t: Optional[type], value: Any) -> Tuple[bool, Any]:
if origin is list:
if not isinstance(value, list):
return False, value
if value == []:
if not value:
return True, value
if not hasattr(t, "__args__"):
if not args:
return True, value
t_items = t.__args__[0] # type: ignore
t_items = args[0]
if getattr(t_items, "__name__", None) == "T":
return True, value
first_item_valid, _ = _validate_type(t_items, value[0])
@@ -280,10 +278,10 @@ def _validate_type(t: Optional[type], value: Any) -> Tuple[bool, Any]:
if origin is tuple:
if not isinstance(value, tuple):
return False, value
if not hasattr(t, "__args__"):
if not args:
return True, value
args = t.__args__ # type: ignore
if args == tuple():
if not args:
return True, value
# we're not checking for empty tuple, cause tuple lengths must match
if len(args) != len(value):
@@ -299,7 +297,7 @@ def _validate_type(t: Optional[type], value: Any) -> Tuple[bool, Any]:
if origin is set:
if not isinstance(value, set):
return False, value
if not hasattr(t, "__args__"):
if not args:
return True, value
t_items = t.__args__[0] # type: ignore
first_item_valid, _ = _validate_type(t_items, next(iter(value)))
@@ -310,13 +308,16 @@ def _validate_type(t: Optional[type], value: Any) -> Tuple[bool, Any]:
if isinstance(value, t):
return True, value
with contextlib.suppress(ValueError, TypeError):
if t is float and value is not None:
return True, float(value)
# TODO: dafuq, i had to add this not list check
# but it would also fail for objects and other complex values
if t is str and value and not isinstance(value, list):
return True, str(value)
if t is float and type(value) is int:
return True, float(value)
# with contextlib.suppress(ValueError, TypeError):
# if t is float and value is not None:
# return True, float(value)
# # TODO: dafuq, i had to add this not list check
# # but it would also fail for objects and other complex values
# if t is str and value and not isinstance(value, list):
# return True, str(value)
return False, value
@@ -50,12 +50,10 @@ class TestActiveUserResourcePermissions:
assert hasattr(permissions, "can_create_model")
assert hasattr(permissions, "can_delete")
assert hasattr(permissions, "can_load")
assert hasattr(permissions, "can_publish")
assert permissions.can_create_model.authorized is True
assert permissions.can_delete.authorized is True
assert permissions.can_load.authorized is True
assert permissions.can_publish.authorized is True
def test_active_user_get_projects_with_permissions_with_filter(
self, client: SpeckleClient, test_project: Project
@@ -0,0 +1,278 @@
from datetime import datetime
import pytest
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.core.api.enums import ModelIngestionStatus
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCancelledInput,
ModelIngestionCreateInput,
ModelIngestionFailedInput,
ModelIngestionRequeueInput,
ModelIngestionStartProcessingInput,
ModelIngestionSuccessInput,
ModelIngestionUpdateInput,
SourceDataInput,
)
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.project_inputs import ProjectCreateInput
from specklepy.core.api.models.current import (
Model,
ModelIngestion,
ModelIngestionStatusData,
Project,
ProjectVisibility,
Version,
)
from specklepy.logging.exceptions import GraphQLException
from specklepy.objects.base import Base
from specklepy.transports.server.server import ServerTransport
from tests.integration.conftest import is_public
@pytest.mark.run()
@pytest.mark.skipif(is_public(), reason="The public API does not support these tests")
class TestIngestionResource:
@pytest.fixture
def project(self, client: SpeckleClient):
return client.project.create(
ProjectCreateInput(
name="test", description=None, visibility=ProjectVisibility.PUBLIC
)
)
@pytest.fixture
def model(self, client: SpeckleClient, project: Project):
return client.model.create(
CreateModelInput(name="test", description=None, project_id=project.id)
)
@pytest.fixture()
def ingestion(
self, client: SpeckleClient, model: Model, project: Project
) -> ModelIngestion:
input = ModelIngestionCreateInput(
model_id=model.id,
project_id=project.id,
progress_message="Starting processing",
source_data=SourceDataInput(
source_application_slug="pytest",
source_application_version="0.0.0",
file_name=None,
file_size_bytes=None,
),
)
ingestion = client.model_ingestion.create(input)
assert isinstance(ingestion, ModelIngestion)
assert isinstance(ingestion.id, str)
assert isinstance(ingestion.created_at, datetime)
assert isinstance(ingestion.updated_at, datetime)
assert isinstance(ingestion.cancellation_requested, bool)
assert isinstance(ingestion.model_id, str)
assert isinstance(ingestion.status_data, ModelIngestionStatusData)
assert isinstance(ingestion.status_data.progress_message, str | None)
assert ingestion.status_data.status == ModelIngestionStatus.PROCESSING
assert not ingestion.cancellation_requested
assert ingestion.model_id == model.id
return ingestion
def test_get_ingestion(
self, client: SpeckleClient, project: Project, ingestion: ModelIngestion
):
queried_ingestion = client.model_ingestion.get_ingestion(
project.id, ingestion.id
)
assert queried_ingestion.id == ingestion.id
assert queried_ingestion.status_data.status == ingestion.status_data.status
def test_update_progress(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
def update(progress: float | None, message: str):
input = ModelIngestionUpdateInput(
ingestion_id=ingestion.id,
project_id=project.id,
progress=progress,
progress_message=message,
)
res = client.model_ingestion.update_progress(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.PROCESSING
update(None, "None")
update(0.1, "0.1")
update(0.5, "Whoa-oh! We're half way there!")
update(1, "Finished")
update(0.2, "Back to processing again")
def test_start_processing_and_requeue(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
# just setting the baseline state
assert ingestion.status_data.status == ModelIngestionStatus.PROCESSING
def requeue(message: str):
input = ModelIngestionRequeueInput(
project_id=project.id,
ingestion_id=ingestion.id,
progress_message=message,
)
res = client.model_ingestion.requeue(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.QUEUED
def start_processing(message: str):
input = ModelIngestionStartProcessingInput(
ingestion_id=ingestion.id,
project_id=project.id,
progress_message=message,
source_data=SourceDataInput(
source_application_slug="test",
source_application_version="test",
file_name="test",
file_size_bytes=1,
),
)
res = client.model_ingestion.start_processing(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.PROCESSING
requeue("put it back in there")
start_processing("go for it")
requeue("and again")
start_processing("run forest run")
def update(progress: float | None, message: str):
input = ModelIngestionUpdateInput(
ingestion_id=ingestion.id,
project_id=project.id,
progress=progress,
progress_message=message,
)
res = client.model_ingestion.update_progress(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message == message
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.PROCESSING
update(None, "None")
update(0.1, "0.1")
update(0.5, "Whoa-oh! We're half way there!")
update(1, "Finished")
update(0.2, "Back to processing again")
def test_error(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
input = ModelIngestionFailedInput(
ingestion_id=ingestion.id,
project_id=project.id,
error_reason="Failed to integration test an error",
error_stacktrace="over here in test_error",
)
res = client.model_ingestion.fail_with_error(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message is None
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.FAILED
# trying to fail for a second time should throw
# with pytest.raises(GraphQLException):
# _ = client.ingestion.fail_with_error(input)
def test_complete(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
remote = ServerTransport(project.id, client)
object_id = operations.send(
Base(applicationId="ASDFGHJKL"), [remote], use_default_cache=False
)
input = ModelIngestionSuccessInput(
ingestion_id=ingestion.id,
root_object_id=object_id,
project_id=project.id,
version_message=None,
)
res = client.model_ingestion.complete(input)
assert isinstance(res, str)
version = client.version.get(res, project.id)
assert isinstance(version, Version)
# trying to complete for a second time should throw
# with pytest.raises(GraphQLException):
# _ = client.ingestion.complete(input)
def test_cancel(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
input = ModelIngestionCancelledInput(
ingestion_id=ingestion.id,
project_id=project.id,
cancellation_message="This was cancelled for testing purposes",
)
res = client.model_ingestion.fail_with_cancel(input)
assert isinstance(res, ModelIngestion)
assert res.status_data.progress_message is None
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.CANCELLED
# Cancel again, should be idempotent
res = client.model_ingestion.fail_with_cancel(input)
assert res.status_data.progress_message is None
assert not res.cancellation_requested
assert res.status_data.status == ModelIngestionStatus.CANCELLED
def test_error_non_existent_ingestion(
self, client: SpeckleClient, project: Project
):
input = ModelIngestionFailedInput(
ingestion_id="Non-existent-ingestion",
project_id=project.id,
error_reason="Failed to integration test an error",
error_stacktrace="over here in test_error",
)
with pytest.raises(GraphQLException):
_ = client.model_ingestion.fail_with_error(input)
def test_complete_failed_non_existent_ingestion(
self, client: SpeckleClient, project: Project
):
input = ModelIngestionFailedInput(
ingestion_id="Non-existent-ingestion",
project_id=project.id,
error_reason="Failed to integration test an error",
error_stacktrace="over here in test_error",
)
with pytest.raises(GraphQLException):
_ = client.model_ingestion.fail_with_error(input)
@pytest.mark.skip(reason="TEST FAILS - server behaviour was changed")
def test_complete_non_existent_root_object(
self, client: SpeckleClient, ingestion: ModelIngestion, project: Project
):
input = ModelIngestionSuccessInput(
ingestion_id=ingestion.id,
root_object_id="asdfasdfasdfasfd",
project_id=project.id,
version_message=None,
)
with pytest.raises(GraphQLException):
_ = client.model_ingestion.complete(input)
@@ -1,6 +1,7 @@
import pytest
from specklepy.api.client import SpeckleClient
from specklepy.core.api.enums import ProjectVisibility
from specklepy.core.api.inputs.model_inputs import (
CreateModelInput,
DeleteModelInput,
@@ -12,11 +13,14 @@ from specklepy.core.api.inputs.project_inputs import (
)
from specklepy.core.api.models.current import (
Model,
ModelPermissionChecks,
PermissionCheckResult,
Project,
ProjectWithModels,
ResourceCollection,
)
from specklepy.logging.exceptions import GraphQLException
from tests.integration.conftest import is_internal, is_public
@pytest.mark.run()
@@ -24,7 +28,9 @@ class TestModelResource:
@pytest.fixture()
def test_project(self, client: SpeckleClient) -> Project:
project = client.project.create(
ProjectCreateInput(name="Test project", description="", visibility=None)
ProjectCreateInput(
name="Test project", description="", visibility=ProjectVisibility.PUBLIC
)
)
return project
@@ -149,3 +155,52 @@ class TestModelResource:
with pytest.raises(GraphQLException):
client.model.delete(delete_data)
def test_model_get_permissions(
self,
client: SpeckleClient,
second_client: SpeckleClient,
test_project: Project,
test_model: Model,
):
result = client.model.get_permissions(test_project.id, test_model.id)
assert isinstance(result, ModelPermissionChecks)
assert result.can_update.authorized is True
assert result.can_create_version.authorized is True
assert result.can_delete.authorized is True
guest = second_client.model.get_permissions(test_project.id, test_model.id)
assert isinstance(guest, ModelPermissionChecks)
assert guest.can_update.authorized is False
assert guest.can_create_version.authorized is False
assert guest.can_delete.authorized is False
@pytest.mark.skipif(
is_public(), reason="API only available on server versions 3.0.11 or greater"
)
def test_can_create_model_ingestion_internal_server(
self,
client: SpeckleClient,
test_project: Project,
test_model: Model,
):
result = client.model.can_create_model_ingestion(test_project.id, test_model.id)
assert isinstance(result, PermissionCheckResult)
assert result.authorized is True
@pytest.mark.skipif(
is_internal(),
reason="API only available on server versions 3.0.11 or greater",
)
def test_can_create_model_ingestion_public_server(
self,
client: SpeckleClient,
test_project: Project,
test_model: Model,
):
with pytest.raises(GraphQLException) as ex:
_ = client.model.can_create_model_ingestion(test_project.id, test_model.id)
assert "GRAPHQL_VALIDATION_FAILED" in str(ex.value)
@@ -24,6 +24,17 @@ class TestProjectResource:
)
return project
@pytest.fixture()
def test_public_project(self, client: SpeckleClient) -> Project:
project = client.project.create(
ProjectCreateInput(
name="test project123",
description="desc",
visibility=ProjectVisibility.PUBLIC,
)
)
return project
@pytest.mark.parametrize(
"name, description, visibility",
[
@@ -50,7 +61,7 @@ class TestProjectResource:
assert result.id is not None
assert result.name == name
assert result.description == (description or "")
# we've disabled creation of public projects for now, they fall back to unlisted
# we've disabled creation of unlisted projects for now, they fall back to public
if visibility == ProjectVisibility.UNLISTED:
assert result.visibility == ProjectVisibility.PUBLIC
else:
@@ -67,13 +78,32 @@ class TestProjectResource:
assert result.created_at == test_project.created_at
def test_project_get_permissions(
self, client: SpeckleClient, test_project: Project
self,
client: SpeckleClient,
second_client: SpeckleClient,
test_project: Project,
test_public_project: Project,
):
result = client.project.get_permissions(test_project.id)
result_private = client.project.get_permissions(test_project.id)
assert isinstance(result_private, ProjectPermissionChecks)
assert result_private.can_create_model.authorized is True
assert result_private.can_delete.authorized is True
assert result_private.can_load.authorized is True
result = client.project.get_permissions(test_public_project.id)
assert isinstance(result, ProjectPermissionChecks)
assert result.can_create_model.authorized is True
assert result.can_delete.authorized is True
assert result.can_load.authorized is True
guest = second_client.project.get_permissions(test_public_project.id)
assert isinstance(result, ProjectPermissionChecks)
assert guest.can_create_model.authorized is False
assert guest.can_delete.authorized is False
assert guest.can_load.authorized is False
def test_project_update(self, client: SpeckleClient, test_project: Project):
new_name = "MY new name"
@@ -1,15 +1,26 @@
import asyncio
from typing import Dict, Optional
from sys import platform
from typing import Dict
import pytest
from specklepy.api.client import SpeckleClient
from specklepy.core.api.enums import (
ModelIngestionStatus,
ProjectModelIngestionUpdatedMessageType,
ProjectModelsUpdatedMessageType,
ProjectUpdatedMessageType,
ProjectVersionsUpdatedMessageType,
UserProjectsUpdatedMessageType,
)
from specklepy.core.api.inputs.model_ingestion_inputs import (
ModelIngestionCreateInput,
ModelIngestionReference,
ModelIngestionRequestCancellationInput,
ModelIngestionUpdateInput,
ProjectModelIngestionSubscriptionInput,
SourceDataInput,
)
from specklepy.core.api.inputs.model_inputs import CreateModelInput
from specklepy.core.api.inputs.project_inputs import (
ProjectCreateInput,
@@ -24,9 +35,16 @@ from specklepy.core.api.models import (
UserProjectsUpdatedMessage,
Version,
)
from tests.integration.conftest import create_client, create_version
from specklepy.core.api.models.current import ModelIngestion
from specklepy.core.api.models.subscription_messages import (
ProjectModelIngestionUpdatedMessage,
)
from tests.integration.conftest import create_client, create_version, is_public
WAIT_PERIOD = 0.4 # time in seconds
# WSL is slow AF, so for local runs, we're being extra generous
# For CI runs on linux,m a much smaller wait time is acceptable
SETUP_TIME_SECONDS = 1 if platform == "linux" else 4
MAX_WAIT_TIME_SECONDS = 0.75 if platform == "linux" else 5
@pytest.mark.run()
@@ -55,48 +73,68 @@ class TestSubscriptionResource:
)
return model1
@pytest.fixture
def test_model_ingestion(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model: Model,
) -> ModelIngestion:
project = subscription_client.model_ingestion.create(
ModelIngestionCreateInput(
project_id=test_project.id,
model_id=test_model.id,
progress_message="",
source_data=SourceDataInput(
source_application_slug="pytest",
source_application_version="0.0.0",
file_name=None,
file_size_bytes=None,
),
)
)
return project
@pytest.mark.asyncio
async def test_user_projects_updated(
self,
subscription_client: SpeckleClient,
) -> None:
message: Optional[UserProjectsUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[UserProjectsUpdatedMessage] = loop.create_future()
def callback(d: UserProjectsUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.user_projects_updated(callback)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
input = ProjectCreateInput(name=None, description=None, visibility=None)
created = subscription_client.project.create(input)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, UserProjectsUpdatedMessage)
assert message.id == created.id
assert message.type == UserProjectsUpdatedMessageType.ADDED
assert isinstance(message.project, Project)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
async def test_project_models_updated(
self, subscription_client: SpeckleClient, test_project: Project
) -> None:
message: Optional[ProjectModelsUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelsUpdatedMessage] = loop.create_future()
def callback(d: ProjectModelsUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_models_updated(
@@ -104,51 +142,53 @@ class TestSubscriptionResource:
)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
input = CreateModelInput(
name="my model", description="myDescription", project_id=test_project.id
)
created = subscription_client.model.create(input)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectModelsUpdatedMessage)
assert message.id == created.id
assert message.type == ProjectModelsUpdatedMessageType.CREATED
assert isinstance(message.model, Model)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
async def test_project_updated(
self, subscription_client: SpeckleClient, test_project: Project
) -> None:
message: Optional[ProjectUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectUpdatedMessage] = loop.create_future()
def callback(d: ProjectUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_updated(callback, test_project.id)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(
SETUP_TIME_SECONDS
) # Give time for subscription to be triggered
input = ProjectUpdateInput(id=test_project.id, name="This is my new name")
created = subscription_client.project.update(input)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectUpdatedMessage)
assert message.id == created.id
assert message.type == ProjectUpdatedMessageType.UPDATED
assert isinstance(message.project, Project)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
async def test_project_versions_updated(
@@ -157,13 +197,12 @@ class TestSubscriptionResource:
test_project: Project,
test_model: Model,
) -> None:
message: Optional[ProjectVersionsUpdatedMessage] = None
task = None
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectVersionsUpdatedMessage] = loop.create_future()
def callback(d: ProjectVersionsUpdatedMessage):
nonlocal message
message = d
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_versions_updated(
@@ -171,15 +210,181 @@ class TestSubscriptionResource:
)
)
await asyncio.sleep(WAIT_PERIOD) # Give time to subscription to be setup
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
created = create_version(subscription_client, test_project.id, test_model.id)
await asyncio.sleep(WAIT_PERIOD) # Give time for subscription to be triggered
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectVersionsUpdatedMessage)
assert message.id == created.id
assert message.type == ProjectVersionsUpdatedMessageType.CREATED
assert isinstance(message.version, Version)
task.cancel()
await task
if not task.cancel():
await task
@pytest.mark.asyncio
@pytest.mark.skipif(
is_public(), reason="The public API does not support these tests"
)
async def test_project_model_ingestion_cancellation(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model_ingestion: ModelIngestion,
) -> None:
assert not test_model_ingestion.cancellation_requested
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelIngestionUpdatedMessage] = (
loop.create_future()
)
def callback(d: ProjectModelIngestionUpdatedMessage):
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_model_ingestion_cancellation_requested(
callback, test_project.id, ingestion_id=test_model_ingestion.id
)
)
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
cancellation_request = ModelIngestionRequestCancellationInput(
ingestion_id=test_model_ingestion.id,
project_id=test_project.id,
cancellation_message="Please cancel",
)
created = subscription_client.model_ingestion.request_cancellation(
cancellation_request
)
assert created.id == test_model_ingestion.id
assert created.cancellation_requested
assert created.status_data.status == ModelIngestionStatus.PROCESSING
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectModelIngestionUpdatedMessage)
assert message.model_ingestion.id == created.id
assert message.model_ingestion.cancellation_requested
assert (
message.type
== ProjectModelIngestionUpdatedMessageType.CANCELLATION_REQUESTED
)
assert created.status_data.status == ModelIngestionStatus.PROCESSING
if not task.cancel():
await task
@pytest.mark.asyncio
@pytest.mark.skipif(
is_public(), reason="The public API does not support these tests"
)
async def test_project_model_ingestion_cancellation_isnt_triggered_by_updates(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model_ingestion: ModelIngestion,
) -> None:
assert not test_model_ingestion.cancellation_requested
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelIngestionUpdatedMessage] = (
loop.create_future()
)
def callback(d: ProjectModelIngestionUpdatedMessage):
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_model_ingestion_cancellation_requested(
callback, test_project.id, ingestion_id=test_model_ingestion.id
)
)
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
cancellation_request = ModelIngestionUpdateInput(
ingestion_id=test_model_ingestion.id,
project_id=test_project.id,
progress=None,
progress_message="this is just an ordinary update",
)
created = subscription_client.model_ingestion.update_progress(
cancellation_request
)
assert created.id == test_model_ingestion.id
assert not created.cancellation_requested
assert created.status_data.status == ModelIngestionStatus.PROCESSING
await asyncio.sleep(MAX_WAIT_TIME_SECONDS)
assert (
not future.done()
) # make sure the sub did not call back and resolve the future
if not task.cancel():
await task
@pytest.mark.asyncio
@pytest.mark.skipif(
is_public(), reason="The public API does not support these tests"
)
async def test_project_model_ingestion_updates(
self,
subscription_client: SpeckleClient,
test_project: Project,
test_model_ingestion: ModelIngestion,
) -> None:
assert not test_model_ingestion.cancellation_requested
loop = asyncio.get_running_loop()
future: asyncio.Future[ProjectModelIngestionUpdatedMessage] = (
loop.create_future()
)
def callback(d: ProjectModelIngestionUpdatedMessage):
nonlocal future
future.set_result(d)
task = asyncio.create_task(
subscription_client.subscription.project_model_ingestion_updated(
callback,
input=ProjectModelIngestionSubscriptionInput(
project_id=test_project.id,
ingestion_reference=ModelIngestionReference(
ingestion_id=test_model_ingestion.id, model_id=None
),
),
# ingestion_id=test_model_ingestion.id,
)
)
await asyncio.sleep(SETUP_TIME_SECONDS) # Give time to subscription to be setup
progress_message = "this is just an ordinary update"
cancellation_request = ModelIngestionUpdateInput(
ingestion_id=test_model_ingestion.id,
project_id=test_project.id,
progress=None,
progress_message=progress_message,
)
created = subscription_client.model_ingestion.update_progress(
cancellation_request
)
assert created.id == test_model_ingestion.id
assert not created.cancellation_requested
assert created.status_data.status == ModelIngestionStatus.PROCESSING
message = await asyncio.wait_for(future, timeout=MAX_WAIT_TIME_SECONDS)
assert isinstance(message, ProjectModelIngestionUpdatedMessage)
assert message.model_ingestion.id == created.id
assert not message.model_ingestion.cancellation_requested
assert message.type == ProjectModelIngestionUpdatedMessageType.UPDATED
assert message.model_ingestion.status_data.progress_message == progress_message
assert created.status_data.status == ModelIngestionStatus.PROCESSING
if not task.cancel():
await task
+9
View File
@@ -1,3 +1,4 @@
import os
import random
import uuid
from typing import Dict
@@ -29,6 +30,14 @@ def host() -> str:
return "localhost:3000"
def is_public() -> bool:
return os.getenv("IS_PUBLIC", "false").lower() == "true"
def is_internal() -> bool:
return not is_public()
def seed_user(host: str) -> Dict[str, str]:
seed = uuid.uuid4().hex
user_dict = {
+92
View File
@@ -0,0 +1,92 @@
from typing import Any, Callable
import pytest
from pytest_httpserver import HTTPServer
from requests import HTTPError
from werkzeug import Request, Response
from specklepy.core.api.client import SpeckleClient
from specklepy.logging import metrics
PATH = "/"
def assert_common_properties(payload: Any) -> None:
assert payload["event"] == "SDK Action"
assert payload["properties"]["token"] == "acd87c5a50b56df91a795e999812a3a4"
assert payload["properties"]["type"] == "action"
assert payload["properties"]["server_id"]
assert payload["properties"]["distinct_id"]
assert payload["properties"]["hostApp"] == "python"
assert payload["properties"]["hostAppVersion"]
assert payload["properties"]["core_version"]
def handler(extra_check: Callable[[Any], bool]) -> Callable[[Request], Response]:
def inner(request: Request) -> Response:
json = request.get_json()
payload = json[0]
assert_common_properties(payload)
assert extra_check(payload)
return Response("", 200)
return inner
def test_metrics_track(httpserver: HTTPServer, client: SpeckleClient):
with ScopedMetricsSetup(httpserver.url_for(PATH)) as _:
# Test No email
httpserver.expect_oneshot_request(PATH, "post").respond_with_handler(
handler(lambda payload: "email" not in payload["properties"])
)
metrics.track("SDK Action", client.account, None, True, False)
# Test With email
httpserver.expect_oneshot_request(PATH, "post").respond_with_handler(
handler(
lambda payload: payload["properties"]["email"]
== client.account.userInfo.email
)
)
metrics.track("SDK Action", client.account, None, True, True)
# Test With custom value
httpserver.expect_oneshot_request(PATH, "post").respond_with_handler(
handler(
lambda payload: payload["properties"]["myCustomProp"] == "myCustomValue"
)
)
metrics.track(
"SDK Action", client.account, {"myCustomProp": "myCustomValue"}, True, True
)
def test_metrics_errors(httpserver: HTTPServer):
with ScopedMetricsSetup(httpserver.url_for(PATH)) as _:
httpserver.expect_oneshot_request(PATH, "post").respond_with_data("", 400)
# Expect send_sync == true to mean mean it will raise
with pytest.raises(HTTPError):
metrics.track("SDK Action", send_sync=True)
# Expect send_sync == false to mean mean it won't
metrics.track("SDK Action")
class ScopedMetricsSetup:
"""
Scoped setup and tear down for enabling metrics tracking
"""
tracker: metrics.MetricsTracker
def __init__(self, metrics_url: str):
self.tracker = metrics.initialise_tracker()
self.tracker.analytics_url = metrics_url
def __enter__(self):
metrics.enable()
def __exit__(self, exc_type, exc_value, traceback):
metrics.disable()
metrics.METRICS_TRACKER = None
+9 -9
View File
@@ -157,10 +157,10 @@ def test_parse_project():
def test_parse_model():
wrap = StreamWrapper(
"https://latest.speckle.systems/projects/843d07eb10/models/d9eb4918c8"
"https://app.speckle.systems/projects/8be1007be1/models/cc7578012d"
)
assert wrap.branch_name == "building wrapper"
assert wrap.branch_name == "speckle tower revit 2025"
assert wrap.type == "branch"
@@ -191,10 +191,10 @@ def test_parse_object_fe2():
def test_parse_version():
wrap = StreamWrapper(
"https://latest.speckle.systems/projects/843d07eb10/models/4e7345c838@c42d5cbac1"
"https://app.speckle.systems/projects/8be1007be1/models/cc7578012d@7199443eff"
)
wrap_quoted = StreamWrapper(
"https://latest.speckle.systems/projects/843d07eb10/models/4e7345c838%40c42d5cbac1"
"https://app.speckle.systems/projects/8be1007be1/models/cc7578012d%407199443eff"
)
assert wrap.type == "commit"
assert wrap_quoted.type == "commit"
@@ -208,11 +208,11 @@ def test_to_string():
"https://testing.speckle.dev/streams/0c6ad366c4/globals/abd3787893",
"https://testing.speckle.dev/streams/4c3ce1459c/commits/8b9b831792",
"https://testing.speckle.dev/streams/a75ab4f10f/objects/5530363e6d51c904903dafc3ea1d2ec6",
"https://latest.speckle.systems/projects/843d07eb10",
"https://latest.speckle.systems/projects/843d07eb10/models/4e7345c838",
"https://latest.speckle.systems/projects/843d07eb10/models/4e7345c838@c42d5cbac1",
"https://latest.speckle.systems/projects/843d07eb10/models/4e7345c838%40c42d5cbac1",
"https://latest.speckle.systems/projects/24c3741255/models/b48d1b10f5a732f4ca4144286391282c",
"https://app.speckle.systems/projects/8be1007be1",
"https://app.speckle.systems/projects/8be1007be1/models/cc7578012d",
"https://app.speckle.systems/projects/8be1007be1/models/cc7578012d@7199443eff",
"https://app.speckle.systems/projects/8be1007be1/models/cc7578012d%407199443eff",
"https://app.speckle.systems/projects/8be1007be1/models/9b5e57dca804a923a8d42d55dcc0191a",
]
for url in urls:
wrap = StreamWrapper(url)
+2 -2
View File
@@ -35,7 +35,7 @@ def sample_text_all_properties(sample_point: Point, sample_plane: Plane) -> Text
alignmentH=AlignmentHorizontal.Center,
alignmentV=AlignmentVertical.Center,
plane=sample_plane,
maxWidth=20,
maxWidth=20.0,
units=Units.m,
)
@@ -56,7 +56,7 @@ def test_text_creation_minimal(sample_point: Point):
def test_text_creation_extended(sample_point: Point, sample_plane: Plane):
text_value = "text"
max_width = 20
max_width = 20.0
text_obj = Text(
value=text_value,
@@ -0,0 +1,110 @@
"""Unit tests for AutomationContext.attach_result_to_objects contract."""
from unittest.mock import MagicMock
import pytest
from speckle_automate import AutomationContext
from speckle_automate.schema import (
AutomationRunData,
ObjectResultLevel,
VersionCreationTrigger,
VersionCreationTriggerPayload,
)
from specklepy.objects.base import Base
def _minimal_automation_context() -> AutomationContext:
run_data = AutomationRunData(
project_id="p",
speckle_server_url="http://localhost",
automation_id="a",
automation_run_id="r",
function_run_id="f",
triggers=[
VersionCreationTrigger(
trigger_type="versionCreation",
payload=VersionCreationTriggerPayload(model_id="m", version_id="v"),
)
],
)
return AutomationContext(
automation_run_data=run_data,
speckle_client=MagicMock(),
_server_transport=MagicMock(),
_speckle_token="",
)
def test_attach_result_to_objects_accepts_empty_list() -> None:
"""Empty affected_objects appends one result case with no object IDs."""
ctx = _minimal_automation_context()
assert len(ctx._automation_result.object_results) == 0
ctx.attach_result_to_objects(
ObjectResultLevel.WARNING,
"SkippedRule",
[],
message="No elements to check.",
)
assert len(ctx._automation_result.object_results) == 1
case = ctx._automation_result.object_results[0]
assert case.level == ObjectResultLevel.WARNING
assert case.category == "SkippedRule"
assert case.object_app_ids == {}
assert case.message == "No elements to check."
def test_attach_result_to_objects_with_objects_appends_case_with_ids() -> None:
"""Single or multiple objects with id produce result case with object_app_ids."""
ctx = _minimal_automation_context()
obj1 = Base()
obj1.id = "id-one"
obj1.applicationId = "app-one"
obj2 = Base()
obj2.id = "id-two"
ctx.attach_result_to_objects(
ObjectResultLevel.ERROR,
"BadType",
[obj1, obj2],
message="Invalid type.",
)
assert len(ctx._automation_result.object_results) == 1
case = ctx._automation_result.object_results[0]
assert case.level == ObjectResultLevel.ERROR
assert case.category == "BadType"
assert case.object_app_ids == {"id-one": "app-one", "id-two": None}
assert case.message == "Invalid type."
def test_attach_result_to_objects_raises_when_object_has_no_id() -> None:
"""At least one object without id raises."""
ctx = _minimal_automation_context()
obj = Base()
obj.id = None
with pytest.raises(Exception, match="results to objects with an id"):
ctx.attach_result_to_objects(
ObjectResultLevel.ERROR,
"Bad",
obj,
message="No id.",
)
assert len(ctx._automation_result.object_results) == 0
def test_attach_info_to_objects_accepts_empty_list() -> None:
"""attach_info_to_objects (convenience method) also accepts empty list."""
ctx = _minimal_automation_context()
ctx.attach_info_to_objects("VersionLevel", [], message="No levels in model.")
assert len(ctx._automation_result.object_results) == 1
case = ctx._automation_result.object_results[0]
assert case.level == ObjectResultLevel.INFO
assert case.category == "VersionLevel"
assert case.object_app_ids == {}
+1 -1
View File
@@ -143,7 +143,7 @@ def test_type_checking() -> None:
order = FrozenYoghurt()
order.servings = 2
order.price = "7" # type: ignore - it will get converted
order.price = 7
order.customer = "izzy"
order.dietary = DietaryRestrictions.VEGAN
order.tag = "preorder"
+5 -6
View File
@@ -31,13 +31,13 @@ fake_bases = [FakeBase("foo"), FakeBase("bar")]
@pytest.mark.parametrize(
"input_type, value, is_valid, return_value",
[
(str, 10, True, "10"),
(str, 10, False, 10),
(str, "foo_bar", True, "foo_bar"),
(
str,
{"foo": "bar"},
True,
"{'foo': 'bar'}",
False,
{"foo": "bar"},
),
(float, 1, True, 1),
# why are we allowing this??? We're lying to our users and ourselves too.
@@ -85,9 +85,8 @@ fake_bases = [FakeBase("foo"), FakeBase("bar")]
(Dict[int, Base], {1: test_base}, True, {1: test_base}),
(Tuple[int, str, str], (1, "foo", "bar"), True, (1, "foo", "bar")),
(Tuple, (1, "foo", "bar"), True, (1, "foo", "bar")),
# given our current rules, this is the reality. Its just sad...
(Tuple[str, str, str], (1, "foo", "bar"), True, ("1", "foo", "bar")),
(Tuple[str, Optional[str], str], (1, None, "bar"), True, ("1", None, "bar")),
(Tuple[str, str, str], (1, "foo", "bar"), False, (1, "foo", "bar")),
(Tuple[str, Optional[str], str], (1, None, "bar"), False, (1, None, "bar")),
(Set[bool], set([1, 2]), False, set([1, 2])),
(Set[int], set([1, 2]), True, set([1, 2])),
(Set[int], set([None, 2]), True, set([None, 2])),
Generated
+27 -1
View File
@@ -526,7 +526,7 @@ name = "exceptiongroup"
version = "1.3.0"
source = { registry = "https://pypi.org/simple/" }
dependencies = [
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
{ name = "typing-extensions", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0b/9f/a65090624ecf468cdca03533906e7c69ed7588582240cfe7cc9e770b50eb/exceptiongroup-1.3.0.tar.gz", hash = "sha256:b241f5885f560bc56a59ee63ca4c6a8bfa46ae4ad651af316d4e81817bb9fd88", size = 29749, upload-time = "2025-05-10T17:42:51.123Z" }
wheels = [
@@ -1867,6 +1867,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/ee/49/1377b49de7d0c1ce41292161ea0f721913fa8722c19fb9c1e3aa0367eecb/pytest_cov-7.0.0-py3-none-any.whl", hash = "sha256:3b8e9558b16cc1479da72058bdecf8073661c7f57f7d3c5f22a1c23507f2d861", size = 22424, upload-time = "2025-09-09T10:57:00.695Z" },
]
[[package]]
name = "pytest-httpserver"
version = "1.1.3"
source = { registry = "https://pypi.org/simple/" }
dependencies = [
{ name = "werkzeug" },
]
sdist = { url = "https://files.pythonhosted.org/packages/f1/d8/def15ba33bd696dd72dd4562a5287c0cba4d18a591eeb82e0b08ab385afc/pytest_httpserver-1.1.3.tar.gz", hash = "sha256:af819d6b533f84b4680b9416a5b3f67f1df3701f1da54924afd4d6e4ba5917ec", size = 68870, upload-time = "2025-04-10T08:17:15.6Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0d/d2/dfc2f25f3905921c2743c300a48d9494d29032f1389fc142e718d6978fb2/pytest_httpserver-1.1.3-py3-none-any.whl", hash = "sha256:5f84757810233e19e2bb5287f3826a71c97a3740abe3a363af9155c0f82fdbb9", size = 21000, upload-time = "2025-04-10T08:17:13.906Z" },
]
[[package]]
name = "pytest-ordering"
version = "0.6"
@@ -2222,6 +2234,7 @@ dev = [
{ name = "pytest" },
{ name = "pytest-asyncio" },
{ name = "pytest-cov" },
{ name = "pytest-httpserver" },
{ name = "pytest-ordering" },
{ name = "ruff" },
{ name = "types-deprecated" },
@@ -2259,6 +2272,7 @@ dev = [
{ name = "pytest", specifier = ">=8.3.4" },
{ name = "pytest-asyncio", specifier = ">=0.25.2" },
{ name = "pytest-cov", specifier = ">=6.0.0" },
{ name = "pytest-httpserver", specifier = ">=1.1.3" },
{ name = "pytest-ordering", specifier = ">=0.6" },
{ name = "ruff", specifier = "==0.9.2" },
{ name = "types-deprecated", specifier = ">=1.2.15.20241117" },
@@ -2622,6 +2636,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/47/96/9d5749106ff57629b54360664ae7eb9afd8302fad1680ead385383e33746/websockets-11.0.3-py3-none-any.whl", hash = "sha256:6681ba9e7f8f3b19440921e99efbb40fc89f26cd71bf539e45d8c8a25c976dc6", size = 118056, upload-time = "2023-05-07T14:25:18.508Z" },
]
[[package]]
name = "werkzeug"
version = "3.1.4"
source = { registry = "https://pypi.org/simple/" }
dependencies = [
{ name = "markupsafe" },
]
sdist = { url = "https://files.pythonhosted.org/packages/45/ea/b0f8eeb287f8df9066e56e831c7824ac6bab645dd6c7a8f4b2d767944f9b/werkzeug-3.1.4.tar.gz", hash = "sha256:cd3cd98b1b92dc3b7b3995038826c68097dcb16f9baa63abe35f20eafeb9fe5e", size = 864687, upload-time = "2025-11-29T02:15:22.841Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/2f/f9/9e082990c2585c744734f85bec79b5dae5df9c974ffee58fe421652c8e91/werkzeug-3.1.4-py3-none-any.whl", hash = "sha256:2ad50fb9ed09cc3af22c54698351027ace879a0b60a3b5edf5730b2f7d876905", size = 224960, upload-time = "2025-11-29T02:15:21.13Z" },
]
[[package]]
name = "wrapt"
version = "2.0.1"