Compare commits

...

39 Commits

Author SHA1 Message Date
dependabot[bot] e194177a98 Bump actions/setup-python from 5 to 6
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-04 10:49:07 +00:00
Jonathon Broughton 3f5880156b Update rule_processor.py (#67)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
2025-06-05 15:29:02 +01:00
Jonathon Broughton 6032306cc2 Update rule_processor.py (#66)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
2025-06-05 15:23:23 +01:00
Jonathon Broughton c7171a54cb v3 (#65)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
2025-06-05 14:03:44 +01:00
Jonathon Broughton 0019667302 Amend README.md to reflect shift of companion application (#64)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
* Improves rule number handling

Adds a fallback mechanism for retrieving rule numbers.

This ensures the system can handle cases where the primary
"Rule Number" field is missing or empty, defaulting to "Rule #"
to maintain data integrity.
Also corrects some docstring formatting.

* Improves rule processing efficiency

Avoids unnecessary rule processing by checking rule severity against the minimum configured severity level. Also ensures that results are only attached to failed objects if they exist and meet the minimum severity criteria. Addresses a potential issue where rules with no "Report Severity" column could cause errors, by considering an alternative "Severity" column.

* Adds Python compatibility inspection

Ensures that the project is compatible with Python 3 by adding a compatibility inspection setting.

This will help to identify and address any potential compatibility issues early on.

* Updates integration test URL and severity.

Updates the default URL used in the integration test to a new speckle model checker endpoint.

Changes the minimum severity level from warning to info, increasing the detail of reported results.

* Updates Model Checker documentation

Updates the documentation to reflect the shift from a spreadsheet-based rule definition to the new Model Checker Application.

Explains how to access the application, create rules, and configure automations.

Also introduces the alternative TSV file format for programmatically generating rules or version controlling rules.
2025-05-12 17:12:45 +01:00
Jonathon Broughton 129132dd3a Fixes (#63)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
* Improves rule number handling

Adds a fallback mechanism for retrieving rule numbers.

This ensures the system can handle cases where the primary
"Rule Number" field is missing or empty, defaulting to "Rule #"
to maintain data integrity.
Also corrects some docstring formatting.

* Improves rule processing efficiency

Avoids unnecessary rule processing by checking rule severity against the minimum configured severity level. Also ensures that results are only attached to failed objects if they exist and meet the minimum severity criteria. Addresses a potential issue where rules with no "Report Severity" column could cause errors, by considering an alternative "Severity" column.

* Adds Python compatibility inspection

Ensures that the project is compatible with Python 3 by adding a compatibility inspection setting.

This will help to identify and address any potential compatibility issues early on.

* Updates integration test URL and severity.

Updates the default URL used in the integration test to a new speckle model checker endpoint.

Changes the minimum severity level from warning to info, increasing the detail of reported results.
2025-05-12 15:53:33 +01:00
Chuck Driesler f902f9c23f Merge pull request #62 from specklesystems/cdriesler-patch-1
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
Update main.yml
2025-04-23 00:45:46 +01:00
Chuck Driesler 7158d0576d Update main.yml 2025-04-23 00:45:34 +01:00
Jonathon Broughton bb87a7b932 Newmain (#61)
* Added over the top levels of documentation for future developers

* Update README with clearer instructions

- Updated the template spreadsheet link.
- Changed steps for rule publishing and automation creation.
- Improved formatting of supported predicates table.
- Added a new support contact method via community forum.
2025-03-01 10:26:01 +00:00
Jonathon Broughton f1c4e65d72 Readmeimprovement (#60)
* Added over the top levels of documentation for future developers

* Update README with clearer instructions

- Updated the template spreadsheet link.
- Changed steps for rule publishing and automation creation.
- Improved formatting of supported predicates table.
- Added a new support contact method via community forum.
2025-03-01 10:22:44 +00:00
Jonathon Broughton 1fa7bcb31a Comprehensive Documentation Update for User and Developer Guides (#59)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
* Added over the top levels of documentation for future developers

* Update README for Speckle Checker functionality

Expanded the overview of the Speckle Checker, detailing its purpose and how it simplifies validation through spreadsheets. Updated usage instructions to include step-by-step guidance on preparing rule spreadsheets and creating automations. Added sections on rule definition format, supported predicates, and example rules for clarity. Enhanced support information at the end.

* Update developer guide for Checker function

Expanded the developer guide with detailed setup instructions, project overview, and testing procedures. Added sections on test automation environment, integration tests, and TDD workflow for rule development. Included troubleshooting tips and future development ideas to enhance functionality.
2025-02-28 16:05:21 +00:00
Jonathon Broughton 66312e1cdd Added over the top levels of documentation for future developers (#58) 2025-02-28 14:59:25 +00:00
Jonathon Broughton 38d2073dbb Refactor footgun (#57)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
- Added traceback import for better error logging.
- Enhanced exception handling to include traceback details.
- Commented out non-integer rule number checks for now.
2025-02-21 13:21:08 +00:00
Jonathon Broughton 091a272185 Add new predicates for value checks (#54)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
- Removed "matches" and "not equal" predicates.
- Added "not equal to" and "does not contain" predicates.
- Introduced a method to check if a parameter value does not contain a substring.
2025-02-20 18:30:05 +00:00
Jonathon Broughton 0e95f3998a Update rules.py (#53)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
2025-02-20 17:32:29 +00:00
Jonathon Broughton 05a5383060 Add 'contains' predicate method (#52)
- Introduced a new predicate for checking if a parameter value contains a substring.
- Added the corresponding method to handle the logic and error management.
2025-02-20 17:22:02 +00:00
Jonathon Broughton f3c56a48b5 Rule numbers and metdata validation (#51)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
* Update Python version and clean up dependencies

- Bumped required Python version from 3.12 to 3.13.
- Removed the lock file for dependency management.
- Cleaned up unnecessary dependency groups in the configuration.

* Refine rule number processing and type conversion

- Updated rule number handling to preserve original identifiers.
- Improved logic for generating missing rule numbers.
- Changed duplicate handling to keep original values intact.
- Adjusted column type conversion to replace NaN with empty strings.

* Refactor message handling in rule processor

- Introduced a new function to format rule messages.
- Replaced direct access to the message with the new formatting function.
- Cleaned up metadata creation for better readability.

* Enhance metadata generation with JSON validation

- Added JSON serialisability check for generated metadata
- Included error handling to log issues during creation
- Updated docstring to clarify function purpose and arguments
2025-02-20 15:41:33 +00:00
Jonathon Broughton a704aded80 Refactor parameter value checks to use float conversion (#49)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
- Updated methods to convert parameter values to float.
- Removed type checks and exceptions for non-numeric types.
- Simplified return logic for invalid conversions.
2025-02-20 12:23:43 +00:00
Jonathon Broughton 90c5051fc6 Update test_user_entry.py (#44) 2025-02-18 20:37:06 +00:00
Jonathon Broughton ec6bdf3485 Update requirements.txt (#43)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
2025-02-18 19:15:21 +00:00
Jonathon Broughton ceaa75d40a Update requirements.txt (#41) 2025-02-18 19:12:26 +00:00
renovate[bot] 0566f7d890 Update python Docker tag to v3.13 (#38)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-18 19:09:55 +00:00
Jonathon Broughton b431662031 Update rule_processor.py (#39) 2025-02-18 19:08:25 +00:00
renovate[bot] e520d9bc91 Update python Docker tag to v3.13 (#37)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-18 18:59:18 +00:00
renovate[bot] b6dcfe57df Update dependency httpx to v0.28.1 (#26)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-18 18:58:52 +00:00
renovate[bot] ba8443ce92 Update dependency pydantic-core to v2.29.0 (#27)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-18 18:58:36 +00:00
renovate[bot] 0bab18d2f2 Update python Docker tag to v3.13 (#28)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-18 18:58:21 +00:00
renovate[bot] dffb7ea7ba Update dependency attrs to v25 (#29)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-18 18:57:59 +00:00
renovate[bot] 4420fd31f4 Update dependency websockets to v15 (#30)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-18 18:57:42 +00:00
Jonathon Broughton 168a1f517a Robust rule validation and UX (#36)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
* Add minimum severity and hide skipped options

- Introduced `minimum_severity` to filter reported results.
- Added `hide_skipped` option to control visibility of skipped tests.
- Updated rule application logic to respect new parameters.

* Improve rule processing and validation

- Enhanced rule reading to group rules and handle validation messages.
- Added functions to process and validate rule numbers in the spreadsheet.
- Improved error handling when reading TSV files.

* Improve docstrings and clean up code

- Added docstring for `PropertyMatchMode` enum.
- Removed unnecessary blank lines in `FunctionInputs`.
- Commented out unused `property_match_mode` field with explanation.

* Improve rule validation logic

Added a new function to validate the structure of rule groups. Key updates include:
- Checks for required columns and conditions.
- Ensures rules start with "WHERE" and have at most one "CHECK".
- Validates that "CHECK" is the last condition if present.
- Raises descriptive errors for invalid structures.

* Refactor rule processing logic

- Added a new function to separate filters and final checks.
- Simplified condition evaluation by using the new function.
- Improved handling of empty rule groups and speckle objects.
- Enhanced clarity in filtering logic for better maintainability.

* Refactor number conversion logic in comparison

- Updated boolean comparison method call for clarity.
- Introduced a helper function to safely convert strings to numbers, handling whitespace and negative values.
- Simplified the type-checking process for string-to-float conversion.

* Update comparison logic in parameter checks

Flipped the comparison logic to match user expectations for parameter value checks. Updated docstrings for better clarity on UX perspective. Added error handling for non-numeric values to improve robustness.

* Refactor method names for clarity

Updated method names to remove leading underscores for better readability and consistency. Added a docstring to describe the purpose of the rules module.

* Add tests for comparison and rule processing

- Introduced a new test suite for value comparisons, covering numeric strings, boolean strings, case sensitivity in string comparisons, and float comparison with tolerance.
- Created a test suite for parameter handling functionality to validate the existence and retrieval of parameters in v2 and v3 objects.
- Added tests for rule processing functionality using both explicit CHECK format and legacy format rules.
2025-02-18 18:56:02 +00:00
Jonathon Broughton e49bf225ec Update main.yml (#35) 2025-02-18 09:23:57 +00:00
Jonathon Broughton f3987fced9 Update main.yml (#34) 2025-02-18 08:21:30 +00:00
Jonathon Broughton 1ae3372f42 Update main.yml (#33)
* Update main.yml

* Update Dockerfile
2025-02-18 08:07:09 +00:00
Jonathon Broughton b071380a4f Update main.yml (#32) 2025-02-18 00:07:30 +00:00
Jonathon Broughton 460b21772a Update Dockerfile (#31)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
2025-02-17 23:57:56 +00:00
renovate[bot] bb40f185b5 Update dependency numpy to v2.2.3 (#24)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-17 23:52:05 +00:00
renovate[bot] ee12143504 Update dependency specklepy to v2.21.3 (#25)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-02-17 23:51:50 +00:00
Jonathon Broughton 8582444e56 Add V3 compatibility (#23)
* Update project configuration and name

- Excluded virtual environment folder from module content.
- Changed JDK name to "WSL Checker" for clarity.
- Updated project name in the configuration file.

* Update project configuration and dependencies

- Added source folders for `src` and `tests`.
- Excluded `.devcontainer` and `.idea` directories.
- Updated Black settings in the project configuration.
- Changed Ruff version from 0.9.5 to 0.9.6.
- Enhanced rule processing by grouping rules before application.
- Introduced a new property match mode for better parameter matching.
- Removed unused test files and configurations to clean up the codebase.

* Shifting to local uv - editing toml

* Update CI workflow and project dependencies

- Added steps to the CI workflow for better clarity.
- Switched from Poetry to pip for dependency management.
- Removed poetry.lock file as it's no longer needed.
- Updated Python version specification to 3.12.
- Adjusted project configuration files for new environment settings.

* Update dependencies for pydantic and related packages

- Added pydantic version 2.11.0a2 to dependencies.
- Updated pydantic-core to version 2.29.0.
- Adjusted URLs and hashes for the new package versions in lock file.
- Ensured compatibility with existing requirements in project configuration.

* Update dependencies to stable versions

- Downgraded Pydantic from 2.11.0a2 to 2.10.6
- Updated Pydantic-core version from 2.29.0 to 2.27.2
- Adjusted URLs and hashes for the new package versions in lock file

* Update dependencies to latest versions

- Upgraded Pydantic from 2.10.6 to 2.11.0a2
- Updated Pydantic Core from 2.27.2 to 2.29.0
- Adjusted source URLs and hashes for new package versions

* Update dependencies to stable versions

- Downgraded Pydantic from 2.11.0a2 to 2.10.6
- Updated Pydantic-core version from 2.29.0 to 2.27.2
- Adjusted URLs and hashes for the new package versions in lock file

* Add pre-commit hook and setup script

- Introduced a pre-commit hook to manage dependencies.
- Automatically installs and exports requirements before commits.
- Added a setup script to copy the hook and set permissions.

* Update project config and clean up unused files

- Removed Black configuration options from the project settings.
- Changed Ruff to run on save instead of reformat.
- Deleted example function inputs JSON file.
- Removed flatten helper module as it's no longer needed.
- Updated pyproject.toml with new line length and ignore rules.
- Cleaned up import statements in local test file.

* Update line length and tidy up code comments

- Increased the maximum line length from 100 to 120.
- Cleaned up import statements for better readability.
- Improved docstrings for clarity and consistency.
- Removed unnecessary blank lines and commented-out code.

* Refactor rules handling and improve documentation

- Updated class names from `RevitRules` to `PropertyRules`.
- Adjusted method calls to reflect the new class structure.
- Cleaned up code formatting for better readability.
- Enhanced developer README with clearer setup instructions.

* Add filtering and rule processing features

- Introduced a new filter function to sort objects by category.
- Created a predicates module for mapping spreadsheet predicates to rule methods.
- Refactored rules handling, simplifying parameter checks and value retrievals.
- Enhanced the main automation function with improved input handling and context management.

* Past WIP rules for processing Speckle parameters

- Introduced a new class to manage parameter rules.
- Added methods to check and retrieve display values from Speckle objects.
- Implemented filtering for displayable objects based on defined criteria.
- Created functions to handle Revit parameter checks and value retrievals.
- Included various utility methods for comparing parameter values against thresholds, ranges, and lists.

* Add new JSON test data for building elements

- Introduced a new JSON file containing detailed specifications for various building elements.
- Included parameters such as ID, type, level details, and multiple attributes related to structural components.
- Added information on dimensions, materials, and classifications relevant to the construction context.

* Rename test files and add new parameter tests

- Renamed localtest.py to test_function.py for clarity.
- Added a new test file for parameter checks, covering:
  - Object deserialization structure
  - Parameter existence and value retrieval in v2 and v3 objects
  - Numeric comparisons and pattern matching on parameters
  - List-based and boolean parameter checks

* Enhance helper functions and add rule processing

- Updated helper functions for better item retrieval and checks.
- Added new functions to evaluate conditions and process rules against Speckle objects.
- Improved logging with a dedicated print function.
- Streamlined flattening logic in existing methods.

* Add test runner config and new web resources

- Added pytest as the project test runner.
- Created a new XML file for web resource paths.
- Included pytest-assertcount in development dependencies.

* Add robust value comparison methods

- Introduced new static methods for comparing values:
  - `is_equal_value`: Compares two values with type handling and optional case sensitivity.
  - `is_not_equal_value`: Checks if two values are not equal.
  - `is_identical_value`: Verifies if two values are exactly identical, considering case sensitivity and no tolerance for floats.
  - `is_not_identical_value`: Checks if two values are not identical.
- Enhanced handling of strings that represent numbers.

* Refactor rules and parameter handling

- Commented out large sections of code for clarity

* Update predicates and enhance tests

- Renamed equality-related methods for clarity.
- Added new methods for "not equal" and "not identical" checks.
- Updated test to return specific objects instead of walls.
- Changed property access in tests to use `.keys()`.
- Introduced multiple tests for stringified number comparisons, case sensitivity, and floating-point precision.

* Add tests for severity level conversion

Created a new test file to validate the `get_severity` function.
- Added parameterised tests for various input cases including valid and invalid severity strings, empty inputs, and non-string types.
- Ensured that all edge cases default to the correct enum value.

* Refactor parameter tests for clarity and efficiency

- Simplified assertions in parameter existence tests.
- Added parameterisation to reduce redundancy across tests.
- Enhanced error messages for better debugging.
- Consolidated similar test cases for v2 and v3 objects.
- Improved structure of boolean and numeric comparison tests.

* Enhance spreadsheet data handling

- Added a function to convert mixed column types.
- Updated the main reading function to use this new conversion.
- Improved error handling for reading TSV files.

* Update version handling in automation function

- Added a global VERSION variable for tracking.
- Updated comments to reflect changes from 'version' to 'VERSION'.
- Adjusted logic to retrieve the version from the root object.
- Enhanced success message to include the current version.

* Improve boolean handling and property search

- Added None handling in boolean conversion.
- Enhanced `find_property` to return raw values if needed.
- Updated traversal logic to avoid revisiting objects.
- Refined comparison methods for better type handling and tolerance.
- Introduced new methods for strict value comparisons with detailed arguments.

* Update predicate method mappings

Switched from string references to actual method names in the predicate mapping. This improves clarity and reduces errors by directly linking predicates to their corresponding methods in PropertyRules. Adjusted some predicate phrases for consistency as well.

* Enhance rule evaluation with severity handling

- Added optional parameters for rule and case numbers in condition evaluation.
- Introduced an Enum for severity levels to standardise reporting.
- Created a function to convert string severities into the SeverityLevel enum, handling various input formats.
- Refactored metadata generation to use the new severity function.
2025-02-17 23:50:34 +00:00
Jonathon Broughton f2e06f165e Update Result Annotations (#20)
build and deploy Speckle functions / publish-automate-function-version (push) Has been cancelled
* Add project configuration and refactor logic

- Created .gitignore to exclude IDE files.
- Added project module configuration for Python.
- Set up inspection profiles for code quality checks.
- Refactored main function logic into separate modules for better organisation.
- Introduced helper functions for object manipulation and rule processing.
- Implemented spreadsheet reading functionality to dynamically load rules.
- Added tests for integration with the Speckle server.

* Update import paths for consistency

- Changed relative imports to absolute imports for clarity.
- Ensured all module references are consistent across files.

* Update package versions and add new dependencies

Bumped several package versions to their latest releases:
- Updated `anyio` to 4.8.0
- Updated `certifi` to 2025.1.31
- Updated `charset-normalizer` to 3.4.1
- Updated `click` to 8.1.8
- Updated `deprecated` to 1.2.18
- Updated `graphql-core` to 3.2.6

Added a new dependency:
- Introduced `levenshtein` version 0.26.1.

Removed some unnecessary extras from the dev dependencies for cleaner management.

* Add Ruff configuration file

Set up a new configuration for Ruff with options to run on save and use the server.

* Refactor rules processing for Data Analysis

- Enhanced the metadata structure in the results attachment function for better data analysis.
- Improved logic handling in result attachment based on rule pass/fail status.
2025-02-07 19:22:14 +00:00
39 changed files with 7043 additions and 2667 deletions
+30
View File
@@ -0,0 +1,30 @@
# Use the official Python 3.13 slim image as the base
FROM python:3.13-slim
# Change to UK mirror for better reliability (robust for missing files)
RUN find /etc/apt/ -name '*.list' -exec sed -i 's|http://deb.debian.org|http://ftp.uk.debian.org|g' {} + || true
# Force apt to use IPv4 to avoid CDN/network issues
RUN echo 'Acquire::ForceIPv4 "true";' > /etc/apt/apt.conf.d/99force-ipv4
# Install system dependencies
RUN apt-get update && apt-get install -y \
git \
&& rm -rf /var/lib/apt/lists/*
# Set the working directory
WORKDIR /home/speckle
# Create a non-root user
RUN useradd -ms /bin/bash vscode
USER vscode
# Set environment variables
ENV PYTHONPATH=/home/speckle
ENV PYTHONUNBUFFERED=1
# Install Python dependencies
COPY requirements.txt requirements-dev.txt pyproject.toml ./
RUN pip install --no-cache-dir -r requirements.txt && \
pip install --no-cache-dir -r requirements-dev.txt && \
echo 'export PATH=$PATH:$HOME/.local/bin' >> ~/.bashrc
+42 -36
View File
@@ -1,43 +1,49 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/python
{
"name": "Python 3",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"image": "mcr.microsoft.com/devcontainers/python:1-3.11-bullseye",
"features": {
"ghcr.io/devcontainers-contrib/features/poetry:2": {}
},
"remoteEnv": {
"SPECKLE_TOKEN": "foobar"
},
"containerEnv": {
"SPECKLE_TOKEN": "asdfasdf"
},
// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "cp .env.example .env && POETRY_VIRTUALENVS_IN_PROJECT=true poetry install --no-root",
// Configure tool-specific properties.
"name": "Model Checker - An Automate Function",
"dockerFile": "Dockerfile",
"context": "..",
"workspaceFolder": "/home/speckle",
"runArgs": [
"--network",
"host"
],
"mounts": [
"source=${localWorkspaceFolder},target=/home/speckle,type=bind,consistency=cached"
],
"customizations": {
"vscode": {
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"ms-python.vscode-pylance",
"ms-python.python",
"ms-python.vscode-pylance",
"ms-python.black-formatter",
"streetsidesoftware.code-spell-checker",
"mikestead.dotenv"
]
"ms-python.isort",
"ms-python.flake8",
"littlefoxteam.vscode-python-test-adapter",
"ms-azuretools.vscode-docker",
"charliermarsh.ruff"
],
"settings": {
"python.defaultInterpreterPath": "/usr/local/bin/python",
"python.linting.enabled": true,
"python.linting.flake8Enabled": true,
"python.formatting.provider": "black",
"python.testing.pytestEnabled": true,
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestArgs": [
"tests"
],
"python.testing.autoTestDiscoverOnSaveEnabled": true,
"python.testing.cwd": "${workspaceFolder}",
"[python]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
}
}
}
}
}
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
},
"postCreateCommand": "sh -c \"mkdir -p ~/.pip && echo '[global]\nprefer-ipv4 = true' > ~/.pip/pip.conf\"",
"postStartCommand": "echo 'Container started successfully!'"
}
+21 -11
View File
@@ -11,27 +11,37 @@ jobs:
FUNCTION_SCHEMA_FILE_NAME: functionSchema.json
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4.2.2
- uses: actions/setup-python@v5
# Step 1: Checkout the repository
- name: Checkout Repository
uses: actions/checkout@v4.2.2
# Step 2: Set up Python
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: '3.11'
- name: Install poetry
# Step 3: Install dependencies using pip
- name: Install Dependencies
run: |
pip install poetry==1.8.4 &&
poetry config virtualenvs.create false &&
poetry config virtualenvs.in-project false &&
poetry config installer.parallel true
- name: Restore dependencies
run: poetry install --no-root
python -m pip install --upgrade pip
pip install -r requirements.txt
# Step 4: Generate the schema
- name: Extract functionInputSchema
id: extract_schema
run: |
python main.py generate_schema ${HOME}/${{ env.FUNCTION_SCHEMA_FILE_NAME }}
python main.py generate_schema "${HOME}/${{ env.FUNCTION_SCHEMA_FILE_NAME }}"
echo "Checking if functionSchema.json exists after generation..."
ls -lah "${HOME}/${{ env.FUNCTION_SCHEMA_FILE_NAME }}"
# Step 5: Build and publish the Speckle function
- name: Speckle Automate Function - Build and Publish
uses: specklesystems/speckle-automate-github-composite-action@0.9.0
with:
speckle_automate_url: ${{ env.SPECKLE_AUTOMATE_URL || vars.SPECKLE_AUTOMATE_URL || 'https://automate.speckle.dev' }}
speckle_automate_url: ${{ env.SPECKLE_AUTOMATE_URL || 'https://automate.speckle.dev' }}
speckle_token: ${{ secrets.SPECKLE_FUNCTION_TOKEN }}
speckle_function_id: ${{ secrets.SPECKLE_FUNCTION_ID }}
speckle_function_input_schema_file_path: ${{ env.FUNCTION_SCHEMA_FILE_NAME }}
speckle_function_command: 'python -u main.py run'
speckle_function_recommended_memory_mi: 5000
+9 -2
View File
@@ -1,12 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$" />
<orderEntry type="jdk" jdkName="Python 3.11.4 WSL (UbuntuDev): (/home/jsdbroughton/.virtualenvs/speckle_automate-mesh_density_checker/bin/python)" jdkType="Python SDK" />
<content url="file://$MODULE_DIR$">
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
<sourceFolder url="file://$MODULE_DIR$/tests" isTestSource="true" />
<excludeFolder url="file://$MODULE_DIR$/.venv" />
</content>
<orderEntry type="jdk" jdkName="uv (Checker)" jdkType="Python SDK" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
<component name="PyDocumentationSettings">
<option name="format" value="GOOGLE" />
<option name="myDocStringFormat" value="Google" />
</component>
<component name="TestRunnerService">
<option name="PROJECT_TEST_RUNNER" value="py.test" />
</component>
</module>
+5 -2
View File
@@ -1,7 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="Black">
<option name="sdkName" value="Python 3.11.4 WSL (UbuntuDev): (/home/jsdbroughton/.virtualenvs/speckle_automate-mesh_density_checker/bin/python)" />
<option name="sdkName" value="WSL Checker" />
</component>
<component name="ProjectRootManager" version="2" project-jdk-name="uv (Checker)" project-jdk-type="Python SDK" />
<component name="PythonCompatibilityInspectionAdvertiser">
<option name="version" value="3" />
</component>
<component name="ProjectRootManager" version="2" project-jdk-name="Python 3.11.4 WSL (UbuntuDev): (/home/jsdbroughton/.virtualenvs/speckle_automate-mesh_density_checker/bin/python)" project-jdk-type="Python SDK" />
</project>
+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="RuffConfigService">
<option name="enableLsp" value="false" />
<option name="runRuffOnSave" value="true" />
<option name="useRuffFormat" value="true" />
<option name="useRuffServer" value="true" />
</component>
</project>
+14
View File
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="WebResourcesPaths">
<contentEntries>
<entry url="file://$PROJECT_DIR$">
<entryData>
<resourceRoots>
<path value="file://$PROJECT_DIR$/test_data" />
</resourceRoots>
</entryData>
</entry>
</contentEntries>
</component>
</project>
+1
View File
@@ -0,0 +1 @@
3.13
+24 -2
View File
@@ -5,5 +5,27 @@
"stringcase",
"typer"
],
"python.defaultInterpreterPath": ".venv/bin/python"
}
"python.defaultInterpreterPath": ".venv/bin/python",
"python.testing.pytestEnabled": true,
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestArgs": [
"tests"
],
"python.testing.autoTestDiscoverOnSaveEnabled": true,
"python.testing.cwd": "${workspaceFolder}",
"editor.formatOnSave": true,
"editor.rulers": [
79
],
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
},
"[python]": {
"editor.defaultFormatter": "charliermarsh.ruff",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports.ruff": "explicit"
}
}
}
+352 -29
View File
@@ -1,54 +1,377 @@
# Checker Function Development Guide
# Checker Function Developer Guide
## Setup
This document provides technical details for developers working on the Speckle Checker Automate function.
## Project Overview
The Checker function enables validation of Speckle objects against user-defined rules in a spreadsheet. It's designed to
be flexible, supporting various object schemas including both v2 and v3 Speckle APIs.
## Setup Development Environment
### Prerequisites
- Python 3.10+
- Poetry for dependency management
### Installation
1. Clone the repository
2. Install dependencies:
```bash
poetry install
```
3. Activate the virtual environment:
```bash
poetry shell
```
### Test Automation Environment
The project uses Speckle's [Test Automation feature](https://speckle.guide/automate/function-testing.html) to run
integration tests against real Speckle data. This provides a sandboxed environment to validate the function's business
logic without triggering actual automations.
#### Setting Up a Test Automation
1. Navigate to your Speckle project
2. Go to the **Automations** tab
3. Click **New Automation**
4. Select **Create Test Automation** in the bottom left
5. Follow the configuration steps
Note: To create a test automation, you must:
- Be an owner of the Speckle project
- Have published this function to the Function Library
- Have at least one release for the function
#### Environment Configuration
For local integration testing, create a `.env` file in the project root with these variables:
1. Install dependencies:
```bash
poetry shell && poetry install
```
2. Configure `.env`:
```
# Your Personal Access Token from Speckle
SPECKLE_TOKEN=your_speckle_token
SPECKLE_SERVER_URL=app.speckle.systems
# The Speckle server URL
SPECKLE_SERVER_URL=https://app.speckle.systems
# From the test automation URL: /projects/[project-id]/automations/[automation-id]
SPECKLE_PROJECT_ID=your_project_id
SPECKLE_AUTOMATION_ID=your_automation_id
```
Get test automation details from app.speckle.systems
This configuration allows the test suite to:
1. Connect to your test automation via the Speckle API
2. Run the function locally against real Speckle data
3. Submit results to the test automation for validation
For detailed instructions, refer to
the [official documentation on function testing](https://speckle.guide/automate/function-testing.html#how-to-create-a-test-automation).
#### Running Integration Tests
With the `.env` file configured:
```bash
# Run the integration tests
pytest test_function.py
```
The SDK utilities will automatically:
- Connect to your test automation
- Execute your function with the specified test data
- Submit results back to Speckle
Test results will be visible on the automation page in the Speckle UI.
#### Unit Tests
For unit tests that don't require a full Speckle connection, you can run:
```bash
# Run unit tests only
pytest test_comparisons.py test_rule_processing.py
```
Note: The `.env` file should never be committed to version control (it's included in .gitignore)
## Project Structure
- `function.py`: Main business logic
- `rules.py`: Rule definitions and processing
- `inputs.py`: Function input schema
- `helpers.py`: Utility functions
- `spreadsheet.py`: TSV handling
```
├── main.py # Entry point for Automate
├── src/
│ ├── function.py # Main function logic
│ ├── inputs.py # Function input schema
│ ├── helpers.py # Utility functions
│ ├── filters.py # Object filtering functions
│ ├── rules.py # Rule definitions and property handling
│ ├── predicates.py # Predicate mapping for spreadsheet values
│ ├── rule_processor.py # Rule application and result handling
│ └── spreadsheet.py # TSV file parsing
├── tests/
│ ├── conftest.py # Test fixtures
│ ├── test_function.py # Main function tests
│ ├── test_comparisons.py # Value comparison tests
│ ├── test_parameters.py # Parameter handling tests
│ └── test_rule_processing.py # Rule processing tests
├── pyproject.toml # Project dependencies
└── poetry.lock # Locked dependencies
```
## Core Components
### 1. Function Execution Flow
The main execution flow is defined in `function.py`:
1. `automate_function()` receives context and inputs from Automate
2. Retrieves Speckle objects via `automate_context.receive_version()`
3. Flattens the object tree using `flatten_base()`
4. Loads rules from the spreadsheet URL via `read_rules_from_spreadsheet()`
5. Applies rules to objects using `apply_rules_to_objects()`
6. Reports results via the Automate context
### 2. Rule Processing
Rules are processed through several stages:
1. **Spreadsheet Parsing** (`spreadsheet.py`):
- Reads TSV data
- Groups rules by rule number
- Validates rule structure
2. **Rule Application** (`rule_processor.py`):
- Processes rule logic (WHERE, AND, CHECK)
- Evaluates conditions against objects
- Attaches results to objects in Automate context
3. **Property Rules** (`rules.py`):
- Handles property lookups in objects
- Implements comparison logic
- Supports both v2 and v3 Speckle schemas
### 3. Property Access System
The system uses a flexible property access mechanism that works with different Speckle schemas:
- **V2 Schema**: Properties in `parameters` dictionary with internal definition names
- **V3 Schema**: Properties in nested `properties.Parameters` structure
The `PropertyRules` class provides methods to:
- Find properties by path or name
- Extract values with appropriate type conversion
- Perform comparisons with tolerance and type handling
## Test-Driven Development for Rules
The test infrastructure is designed to support Test-Driven Development (TDD) when creating new rules or extending
functionality. This approach is especially powerful for rule development as it allows you to verify behavior against
known test objects.
### Using Test Fixtures for Rule Development
The `conftest.py` file contains test fixtures that provide sample Speckle objects for testing:
```python
@pytest.fixture
def v2_wall():
"""Creates a v2-style Speckle wall object"""
wall = Base()
wall.id = "cdb18060dc48281909e94f0f1d8d3cc0"
wall.type = "W30(Fc24)"
wall.units = "mm"
wall.family = "Basic Wall"
wall.height = 1400
wall.flipped = False
wall.category = "Walls"
# ... more properties
return wall
@pytest.fixture
def v3_wall():
"""Creates a v3-style Speckle wall object"""
wall = Base()
wall.id = "46f06fef727d64a0bbcbd7ced51e0cd2"
wall.name = "Walls - W30(Fc24)"
wall.type = "W30(Fc24)"
wall.units = "mm"
wall.family = "Basic Wall"
# ... more properties
return wall
```
These fixtures create standardized test objects that represent different Speckle schema versions, allowing you to test
rule behavior consistently.
### TDD Workflow for New Rules
When developing a new rule or predicate, follow this TDD approach:
1. **Add test fixtures**: First, expand `conftest.py` with representative objects that your rule will process
2. **Write tests first**: Create test cases in a test file (e.g., `test_my_rule.py`):
```python
def test_new_wall_rule(v2_wall, v3_wall):
"""Test a new rule that checks wall thickness requirements"""
# Test with v2 schema
assert PropertyRules.is_new_wall_check(v2_wall, "width", "300")
# Test with v3 schema
assert PropertyRules.is_new_wall_check(v3_wall, "Width", "300")
# Test failure case
v2_wall.parameters["WALL_ATTR_WIDTH_PARAM"].value = 200
assert not PropertyRules.is_new_wall_check(v2_wall, "width", "300")
```
3. **Implement the rule**: Add the new rule method to the `PropertyRules` class in `rules.py`:
```python
@staticmethod
def is_new_wall_check(speckle_object: Base, parameter_name: str, expected_value: str) -> bool:
"""Checks if a wall meets specific thickness requirements"""
parameter_value = PropertyRules.get_parameter_value(speckle_object, parameter_name)
# Implement rule logic
return result
```
4. **Add to predicate mapping**: Register your new rule in `predicates.py`:
```python
PREDICATE_METHOD_MAP = {
# Existing predicates...
"new_wall_check": PropertyRules.is_new_wall_check.__name__,
}
```
5. **Run tests to verify**:
```bash
pytest test_my_rule.py -v
```
### Creating Comprehensive Test Objects
For the most effective testing, your test objects in `conftest.py` should:
1. **Include diverse objects**: Walls, columns, beams, etc.
2. **Cover edge cases**: Null values, missing properties, special characters
3. **Represent both schemas**: Include both v2 and v3 format objects
4. **Include real-world examples**: Extract sample objects from actual projects
You can extract real objects for testing using:
```python
# Example code to extract and save real objects for test fixtures
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.transports.server import ServerTransport
client = SpeckleClient(host="app.speckle.systems")
client.authenticate_with_token(token)
transport = ServerTransport(client=client, stream_id="stream_id")
obj = operations.receive("object_id", transport)
# Print structure to help with fixture creation
print(obj.get_member_names())
print(obj.get_dynamic_member_names())
```
By following this TDD approach and maintaining comprehensive test fixtures, you can develop robust rules that work
reliably across different object schemas and handle edge cases appropriately.
## Testing
### Running Tests
```bash
poetry run pytest
# Run all tests
pytest
# Run specific test file
pytest tests/test_parameters.py
# Run with coverage
pytest --cov=src
```
## Extending Rules
### Test Data
1. Add new predicate to `input_predicate_mapping` in `rules.py`
2. Create corresponding method in `RevitRules` class
3. Update tests
Test fixtures in `conftest.py` provide sample objects:
## Building
- `v2_wall`: Wall object in v2 schema
- `v3_wall`: Wall object in v3 schema
The function is packaged as a Docker container:
### Manual Testing with Real Data
```bash
docker build -f ./Dockerfile -t checker .
```
For testing with real Speckle data:
## Local Testing
```python
from specklepy.api import operations
from specklepy.api.client import SpeckleClient
from specklepy.api.credentials import get_account_from_token
from specklepy.transports.server import ServerTransport
```bash
docker run --rm checker python -u main.py run [automation_data] [parameters] [token]
client = SpeckleClient(host="app.speckle.systems")
account = get_account_from_token(token, "app.speckle.systems")
client.authenticate_with_account(account)
transport = ServerTransport(client=client, stream_id="your_stream_id")
commit = client.commit.get(stream_id="your_stream_id", commit_id="your_commit_id")
obj = operations.receive(commit.referencedObject, transport)
```
## Deployment
Create a GitHub release to trigger deployment to Speckle Automate.
The function is deployed through GitHub Actions:
1. Create a GitHub release to trigger the build workflow
2. The workflow builds the necessary artifacts and pushes them to the Speckle Automate registry
3. The function becomes available in the Speckle Automate UI
## Performance Considerations
- **Large Object Trees**: When processing large models, use aggressive filtering with WHERE clauses
- **Rule Complexity**: Minimize the number of nested property lookups
- **Memory Usage**: Be aware of object reference handling and avoid deep copies
## Troubleshooting
### Common Issues
1. **Rule not matching expected objects**:
- Check property paths for the specific object type
- Verify data types (strings vs. numbers)
- Enable debug logging
2. **Slow performance**:
- Check for inefficient property lookups
- Add more specific WHERE filters to reduce object set
3. **Docker build failures**:
- Check dependency compatibility
- Verify Python version requirements
## Contributing
1. Create a branch for your feature or fix
2. Add tests for new functionality
3. Update documentation
4. Submit a pull request
5. Ensure CI tests pass
## Future Development
Potential improvements:
- Support for more complex rule logic (OR conditions)
- UI-based rule editor
- Result visualization tools
- Performance optimizations for large models
- Support for referencing other objects in rules
+9 -10
View File
@@ -1,16 +1,15 @@
# We use the official Python 3.11 image as our base image and will add our code to it. For more details, see https://hub.docker.com/_/python
# Use the official Python 3.13 slim image as the base
FROM python:3.13-slim
# We install poetry to generate a list of dependencies which will be required by our application
RUN pip install poetry==1.8.4
# We set the working directory to be the /home/speckle directory; all of our files will be copied here.
# Set the working directory inside the container
WORKDIR /home/speckle
# Copy all of our code and assets from the local directory into the /home/speckle directory of the container.
# We also ensure that the user 'speckle' owns these files, so it can access them
# This assumes that the Dockerfile is in the same directory as the rest of the code
# Copy the application files to the working directory
COPY . /home/speckle
# Using poetry, we generate a list of requirements, save them to requirements.txt, and then use pip to install them
RUN poetry export --format requirements.txt --output /home/speckle/requirements.txt && pip install --requirement /home/speckle/requirements.txt
# Upgrade pip and install dependencies using requirements.txt
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -r /home/speckle/requirements.txt
# Set the entrypoint for running the Speckle function
CMD ["python", "-u", "main.py", "run"]
+117 -23
View File
@@ -1,36 +1,130 @@
# Public Function: Checker
# Model Checker
Validate Speckle objects against configurable rules using spreadsheet definitions.
Model Checker is an Automate function that validates Speckle objects against configurable rules. This approach provides
a flexible way to implement quality checks and maintain consistent standards across projects.
## Usage
## Overview
1. Access the template Google Sheet [link needed]
2. Make a copy to your Google Drive using File > Make a copy
3. Define your rules in your sheet
4. Click "Speckle" menu > "Publish Rules" to get your TSV URL
5. Create an Automation in Speckle Automate using the Checker function
6. Paste your TSV URL into the function configuration
7. Run your automation
The Model Checker allows you to:
## Rule Types
- Define validation rules for your objects
- Configure severity levels for issues
- Check properties across different types of objects
- Generate reports of validation results
- Apply consistent standards across projects
- Property existence
- Value matching
- Numeric comparisons
- Range checks
- List membership
- Pattern matching
- Boolean checks
## Getting Started
## Severity Levels
### 1. Access the Model Checker Application
- WARNING: Issues that should be reviewed
- ERROR: Critical issues requiring attention
1. Go to the [Model Checker Application](https://model-checker.speckle.systems)
2. Sign in with your Speckle account
3. Create and manage your validation rules through the intuitive web interface
### 2. Create an Automation
1. Go to your workspace project in [Speckle](https://app.speckle.systems/)
2. Create a new Automation
3. Select the Model Checker function
4. Configure the function:
- Set minimum severity level to report
- Configure other options as needed
5. Save and run your automation
## Rule Definition Format
Rules are defined with the following components:
| Logic | Property Name | Predicate | Value | Message | Report Severity |
|-------|---------------|--------------|-----------|----------------------|-----------------|
| WHERE | category | matches | Walls | Wall thickness check | ERROR |
| CHECK | Width | greater than | 200 | | |
| WHERE | category | matches | Columns | Column height check | WARNING |
| AND | height | in range | 2500,4000 | | |
### Component Explanation
- **Logic**: Defines how conditions are combined (WHERE, AND, CHECK)
- **Property Name**: The object property or parameter to check
- **Predicate**: Comparison operation (equals, greater than, etc.)
- **Value**: Reference value for comparison
- **Message**: Description shown in validation results
- **Report Severity**: ERROR, WARNING, or INFO
### Supported Predicates
| Predicate | Description | Example |
|------------------|-----------------------------|---------------------------------------|
| exists | Checks if a property exists | `height` exists |
| equal to | Exact value match | `width` equal to `300` |
| not equal to | Value doesn't match | `material` not equal to `Concrete` |
| greater than | Value exceeds threshold | `height` greater than `3000` |
| less than | Value below threshold | `thickness` less than `50` |
| in range | Value within bounds | `elevation` in range `0,10000` |
| in list | Value in allowed set | `type` in list `W1,W2,W3` |
| contains | Property contains substring | `name` contains `Beam` |
| does not contain | Property doesn't contain | `name` does not contain `temp` |
| is true | Boolean property is true | `is_structural` is true |
| is false | Boolean property is false | `is_placeholder` is false |
| is like | Loose text matching | `name` is like `Wall` matches `Walls` |
## Rule Logic
- **WHERE**: Filters objects to check (like SELECT WHERE in SQL)
- **AND**: Additional filter conditions
- **CHECK**: Final check condition (optional, defaults to last AND)
Objects pass a rule when they match all conditions. Objects that match WHERE/AND filters but fail the CHECK condition
are reported as issues.
## Working with Object Properties
The Model Checker understands properties in Speckle objects regardless of schema:
- Direct properties: `category`, `name`, `id`
- Nested properties: `parameters.WIDTH.value`
- Revit parameters: Use parameter names like `Mark`, `Width`, `Assembly Code`
## Example Rules
[Screenshot or example table to be added]
### Wall Thickness Check
```
Rule: WHERE category equals "Walls" AND width less than "200"
Message: "Walls must have width of at least 200."
Severity: ERROR
```
### Door Naming Convention
```
Rule: WHERE category equals "Doors" AND name is not like "^D\d{3}$"
Message: "All doors must have a name that follows the format "D" followed by three digits."
Severity: WARNING
```
### Structural Column Height Range
```
Rule: WHERE category equals "Columns" AND is_structural is true AND height not in range "2400,4000"
Message: "Structural columns must have a height between 2400 and 4000."
Severity: ERROR
```
## Support
For issues or questions, please open a GitHub issue.
For issues or questions, please let us know on the [Speckle Community Forum](https://speckle.community/).
### Alternative: TSV File Format
While the Model Checker Application is the recommended way to create and manage rules, you can also create compatible
TSV (Tab-Separated Values) files manually. This can be useful for:
- Programmatically generating rules
- Version controlling rules in a text format
- Integrating with existing workflows
- Creating rules in bulk
The TSV file should follow the same structure as shown in the table above, with columns separated by tabs. The file will
then need to be hosted somewhere and served with MIME-type of `text/tab-separated-values` and the URL used in the
automation configuration.
-20
View File
@@ -1,20 +0,0 @@
{
"speckleToken": "YOUR SPEKCLE TOKEN",
"functionInputs": {
"whisperMessage": "you are doing something weird",
"forbiddenSpeckleType": "wall"
},
"automationRunData": {
"project_id": "project id",
"speckle_server_url": "https://latest.speckle.systems",
"automation_id": "automation id",
"automation_run_id": "automation run id",
"function_run_id": "function run id",
"triggers": [
{
"payload": { "modelId": "model id", "versionId": "version id" },
"triggerType": "versionCreation"
}
]
}
}
-27
View File
@@ -1,27 +0,0 @@
"""Helper module for a simple speckle object tree flattening."""
from collections.abc import Iterable
from specklepy.objects import Base
def flatten_base(base: Base) -> Iterable[Base]:
"""Flatten a base object into an iterable of bases.
This function recursively traverses the `elements` or `@elements` attribute of the
base object, yielding each nested base object.
Args:
base (Base): The base object to flatten.
Yields:
Base: Each nested base object in the hierarchy.
"""
# Attempt to get the elements attribute, fallback to @elements if necessary
elements = getattr(base, "elements", getattr(base, "@elements", None))
if elements is not None:
for element in elements:
yield from flatten_base(element)
yield base
+13
View File
@@ -0,0 +1,13 @@
#!/bin/bash
echo "Running pre-commit hook..."
# Ensure dependencies are installed with uv
uv pip install --requirement requirements.txt
# Export dependencies with uv (and overwrite requirements.txt)
uv pip freeze > requirements.txt
# Add generated requirements.txt to git
git add requirements.txt
echo "Pre-commit hook completed successfully!"
Generated
-1797
View File
File diff suppressed because it is too large Load Diff
+41 -23
View File
@@ -1,30 +1,28 @@
[tool.poetry]
name = "speckle-automate-py"
version = "0.1.0"
[project]
name = "speckle-automate-checker"
version = "3.0.0"
description = "Allows for QAQC property checking with Speckle"
authors = ["Jonathon Broughton <jonathon@speckle.systems>"]
authors = [{ name = "Jonathon Broughton", email = "jonathon@speckle.systems" }]
readme = "README.md"
package-mode = false
requires-python = ">=3.13"
dependencies = [
"more-itertools>=10.6.0",
"pandas>=2.2.3",
"pydantic==2.10.6",
"python-dotenv>=1.0.1",
"python-levenshtein>=0.26.1",
"specklepy>=3.0.0",
"pydantic-settings>=2.7.1",
]
[tool.poetry.dependencies]
more-itertools = "^10.6.0"
pandas = "^2.2.3"
python = "^3.11"
python-dotenv = "^1.0.1"
python-levenshtein = "^0.26.1"
specklepy = "^2.21.0"
[tool.poetry.group.dev.dependencies]
black = "^25.0.0"
mypy = "^1.3.0"
pydantic-settings = "^2.3.0"
pytest = "^8.0.0"
ruff = "^0.9.5"
# specklepy = { path = "../specklepy", develop = true }
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
[project.optional-dependencies]
dev = [
"mypy>=1.15.0",
"pytest>=8.3.4",
"pytest-assertcount>=1.0.0",
"ruff==0.11.12",
]
[tool.ruff]
select = [
@@ -34,6 +32,26 @@ select = [
"D", # pydocstyle
"I", # isort
]
ignore = ["F401", "F403", "E501"]
exclude = [".venv", "**/*.yml"]
line-length = 79
[tool.ruff.pydocstyle]
convention = "google"
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
line-ending = "auto"
docstring-code-format = true
docstring-code-line-length = 79
[tool.ruff.isort]
known-first-party = ["src"]
[tool.ruff.lint.pydocstyle]
convention = "google"
[tool.setuptools]
py-modules = []
+23
View File
@@ -0,0 +1,23 @@
argcomplete==3.6.2
click==8.1.8
colorama==0.4.6
coverage==7.8.2
flake8==7.2.0
iniconfig==2.1.0
isort==6.0.1
mccabe==0.7.0
mypy_extensions==1.1.0
packaging==24.2
pathspec==0.12.1
pipx==1.7.1
platformdirs==4.3.7
pluggy==1.6.0
pycodestyle==2.13.0
pyflakes==3.3.2
Pygments==2.19.1
pytest>=8.3.4
pytest-assertcount>=1.0.0
pytest-cov==6.1.1
ruff==0.11.12
userpath==1.9.2
mypy>=1.15.0
+7
View File
@@ -0,0 +1,7 @@
more-itertools>=10.6.0
pandas>=2.2.3
pydantic==2.10.6
python-dotenv>=1.0.1
python-levenshtein>=0.26.1
specklepy>=3.0.0
pydantic-settings>=2.7.1
+22
View File
@@ -0,0 +1,22 @@
#!/bin/sh
# Store the current Python environment
CURRENT_ENV=$(pip freeze)
# Remove dev dependencies
pip uninstall -y pytest pytest-cov isort flake8 ruff
# Generate production requirements
pip freeze > requirements.txt
# Reinstall dev dependencies
pip install pytest pytest-cov isort flake8 ruff
# Generate dev requirements
pip freeze > requirements-dev.txt
# Restore the original environment
pip uninstall -y pytest pytest-cov isort flake8 ruff
echo "$CURRENT_ENV" | pip install -r /dev/stdin
echo "Requirements files have been updated successfully!"
+9
View File
@@ -0,0 +1,9 @@
#!/bin/bash
# Copy the pre-commit hook to the .git/hooks/ directory
cp hooks/pre-commit .git/hooks/pre-commit
# Ensure the hook is executable
chmod +x .git/hooks/pre-commit
echo "Git hooks have been set up!"
+12
View File
@@ -0,0 +1,12 @@
{
"folders": [
{
"path": "."
}
],
"settings": {
"files.autoSave": "onFocusChange",
"editor.defaultFormatter": null,
"editor.formatOnSave": true
}
}
+31
View File
@@ -0,0 +1,31 @@
from specklepy.objects.base import Base
from src.rules import PropertyRules
def filter_objects_by_category(speckle_objects: list[Base], category_input: str) -> tuple[list[Base], list[Base]]:
"""Filters objects by category value and test.
This function takes a list of Speckle objects, filters out the objects
with a matching category value and satisfies the test, and returns
both the matching and non-matching objects.
Args:
speckle_objects (List[Base]): The list of Speckle objects to filter.
category_input (str): The category value to match against.
Returns:
Tuple[List[Base], List[Base]]: A tuple containing two lists:
- The first list contains objects with matching category and test.
- The second list contains objects without matching category or test.
"""
matching_objects = []
non_matching_objects = []
for obj in speckle_objects:
if PropertyRules.is_category(obj, category_input):
matching_objects.append(obj)
else:
non_matching_objects.append(obj)
return matching_objects, non_matching_objects
+103 -23
View File
@@ -1,46 +1,126 @@
"""This module contains the function's business logic.
"""This is the main entry point for the Speckle Automate function.
Use the automation_context module to wrap your function in an Automate context helper.
The Speckle Automate system works as follows:
1. When a model is committed to Speckle, it triggers automations associated with the project
2. For each automation, Speckle Automate prepares a runtime environment and context
3. The automation context includes the model data and function inputs
4. This function is executed to process the model and provide results
5. Results are attached to objects in the model, creating an annotated view
This function implements a configurable rule-based validation system that:
- Reads validation rules from an external spreadsheet
- Applies these rules to objects in the Speckle model
- Reports validation results back to the Speckle platform
- Provides an annotated view of the model showing validation issues
"""
from speckle_automate import AutomationContext, AutomateBase
from speckle_automate import AutomationContext
from specklepy.objects.base import Base
from src.rules import apply_rules_to_objects
from src.helpers import flatten_base, speckle_print
from src.inputs import FunctionInputs
from src.helpers import flatten_base
from src.rule_processor import apply_rules_to_objects
from src.spreadsheet import read_rules_from_spreadsheet
VERSION: int = 2
def automate_function(
automate_context: AutomationContext,
function_inputs: FunctionInputs,
automate_context: AutomationContext,
function_inputs: FunctionInputs,
) -> None:
"""This version of the function will add a check for the new provide inputs.
"""Main entry point for the Speckle Automate function.
This function is called by the Speckle Automate system when the automation is triggered.
It orchestrates the entire validation process:
1. Receiving and flattening the model data
2. Detecting the Speckle object schema version
3. Loading and grouping rules from the external spreadsheet
4. Applying rules to objects and collecting results
5. Reporting results back to the Speckle platform
Args:
automate_context: A context helper object, that carries relevant information
about the runtime context of this function.
It gives access to the Speckle project data, that triggered this run.
It also has convenience methods attach result data to the Speckle model.
function_inputs: An instance object matching the defined schema.
automate_context: A context helper provided by Speckle Automate that:
- Provides access to the Speckle model data
- Handles result reporting and view management
- Manages run status (success, failure, exception)
function_inputs: User-provided inputs defined in the FunctionInputs schema,
particularly the URL to the rules spreadsheet
"""
# -------------------------------------------------------------------------
# Step 1: Receive and process the model data
# -------------------------------------------------------------------------
# the context provides a convenient way, to receive the triggering version
version_root_object = automate_context.receive_version()
# The AutomationContext provides a convenient way to access the model data
# that triggered this automation run
version_root_object: Base = automate_context.receive_version()
# We can continue to work with a flattened list of objects.
# Flatten the object tree into a list of objects
# The Speckle object model is hierarchical, but for validation purposes,
# it's easier to work with a flat list of objects
flat_list_of_objects = list(flatten_base(version_root_object))
# read the rules from the spreadsheet
rules = read_rules_from_spreadsheet(function_inputs.spreadsheet_url)
# -------------------------------------------------------------------------
# Step 2: Detect Speckle object schema version
# -------------------------------------------------------------------------
# apply the rules to the objects
apply_rules_to_objects(flat_list_of_objects, rules, automate_context)
# The Speckle object schema has evolved over time
# In newer models, we can detect the version from the root object
# This version information helps our validation logic handle different schemas
global VERSION
VERSION = getattr(version_root_object, "version", 2) # noqa: F841SION = getattr(version_root_object,"version", 2) # noqa: F841 # noqa: F841
# set the automation context view, to the original model / version view
# In v2, parameters are stored in a 'parameters' dictionary on each object
# In v3, they are nested in 'properties.Parameters' with categorization
speckle_print(f"Detected Speckle object schema version: {VERSION}")
# -------------------------------------------------------------------------
# Step 3: Load and process rules from the spreadsheet
# -------------------------------------------------------------------------
# The rules are defined in an external spreadsheet (TSV format)
# This allows non-technical users to define and modify rules
# without changing the code
grouped_rules, messages = read_rules_from_spreadsheet(
function_inputs.spreadsheet_url
)
# Handle any validation messages from rule processing
for message in messages:
speckle_print(message) # or log them appropriately
# If rule processing failed, mark the run as failed and exit
if grouped_rules is None:
automate_context.mark_run_exception("Failed to process rules")
return
# -------------------------------------------------------------------------
# Step 4: Apply rules to objects and collect results
# -------------------------------------------------------------------------
# This is where the actual validation happens
# Each rule is applied to relevant objects, and results are collected
# Results are attached to objects in the model to create an annotated view
apply_rules_to_objects(
flat_list_of_objects,
grouped_rules,
automate_context,
minimum_severity=function_inputs.minimum_severity,
hide_skipped=function_inputs.hide_skipped,
)
# -------------------------------------------------------------------------
# Step 5: Finalize the automation run
# -------------------------------------------------------------------------
# Set the context view to the original model/version view
# This ensures that the results are displayed in the correct context
automate_context.set_context_view()
# report success
# Mark the run as successful and provide a summary message
# This message will be displayed to the user in the Speckle UI
automate_context.mark_run_success(
f"Successfully applied rules to {len(flat_list_of_objects)} objects."
f"Successfully applied {len(grouped_rules)} rules to "
f"{len(flat_list_of_objects)} version {VERSION} objects."
)
+69 -26
View File
@@ -1,14 +1,14 @@
"""Helper module for a speckle object tree flattening."""
from collections.abc import Iterable
from typing import Optional, Tuple, List
from collections.abc import Generator, Iterable
from typing import Any
from specklepy.objects import Base
from specklepy.objects.other import Instance, Transform
from specklepy.objects.proxies import InstanceProxy as Instance
def speckle_print(log_string: str = "banana") -> None:
"""Print a string to the console with a green color."""
print("\033[92m" + str(log_string) + "\033[0m")
@@ -21,7 +21,28 @@ def flatten_base(base: Base) -> Iterable[Base]:
yield base
def flatten_base_thorough(base: Base, parent_type: str = None) -> Iterable[Base]:
def get_item(obj: Base | dict[str, Any], key, default=None):
"""Get an item from a dictionary or an object with a default value."""
if isinstance(obj, dict): # If it's a dictionary
return obj.get(key, default)
elif hasattr(obj, key): # If it's an object with the attribute
return getattr(obj, key, default)
return default # Return default if it's neither a dict nor an object with
# the attribute
def has_item(obj: Base | dict[str, Any], key: str) -> bool:
"""Check if an object has a key or an attribute."""
if isinstance(obj, dict):
return key in obj
elif hasattr(obj, key):
return True
return False
def flatten_base_thorough(
base: Base, parent_type: str | None = None
) -> Iterable[Base]:
"""Take a base and flatten it to an iterable of bases.
Args:
@@ -63,31 +84,44 @@ def flatten_base_thorough(base: Base, parent_type: str = None) -> Iterable[Base]
def extract_base_and_transform(
base: Base,
inherited_instance_id: Optional[str] = None,
transform_list: Optional[List[Transform]] = None,
) -> Tuple[Base, str, Optional[List[Transform]]]:
"""
Traverses Speckle object hierarchies to yield `Base` objects and their transformations.
Tailored to Speckle's AEC data structures, it covers the newer hierarchical structures
with Collections and also with patterns found in older Revit specific data.
inherited_instance_id: str | None = None,
transform_list: list[list[float]] | None = None,
) -> Generator[
Base
| str
| list[list[float]]
| None
| tuple[Base, Any | None, list[list[float]] | None | list[Any]],
Any | None,
]:
"""Traverses Speckle object hierarchies to yield `Base`s and transformas.
Tailored to Speckle's AEC data structures, it covers the newer
hierarchical structures with Collections and also with patterns found in
older Revit specific data.
Parameters:
- base (Base): The starting point `Base` object for traversal.
- inherited_instance_id (str, optional): The inherited identifier for `Base` objects without a unique ID.
- transform_list (List[Transform], optional): Accumulated list of transformations from parent to child objects.
- inherited_instance_id (str, optional): The inherited identifier for
`Base` objects without a unique ID.
- transform_list (List[List[float]], optional): Accumulated list of
transformations from parent to child objects.
Yields:
- tuple: A `Base` object, its identifier, and a list of applicable `Transform` objects or None.
- tuple: A `Base` object, its identifier, and a list of applicable
transformations or None.
The id of the `Base` object is either the inherited identifier for a definition from an instance
or the one defined in the object.
The id of the `Base` object is either the inherited identifier for a
definition from an instance or the one defined in the object.
"""
# Derive the identifier for the current `Base` object, defaulting to an inherited one if needed.
# Derive the identifier for the current `Base` object, defaulting to an
# inherited one if needed.
current_id = getattr(base, "id", inherited_instance_id)
transform_list = transform_list or []
if isinstance(base, Instance):
# Append transformation data and dive into the definition of `Instance` objects.
# Append transformation data and dive into the definition of `Instance`
# objects.
if base.transform:
transform_list.append(base.transform)
if base.definition:
@@ -98,22 +132,31 @@ def extract_base_and_transform(
# Initial yield for the current `Base` object.
yield base, current_id, transform_list
# Process 'elements' and '@elements', typical containers for `Base` objects in AEC models.
elements_attr = getattr(base, "elements", []) or getattr(base, "@elements", [])
# Process 'elements' and '@elements', typical containers for `Base`
# objects in AEC models.
elements_attr = getattr(base, "elements", []) or getattr(
base, "@elements", []
)
for element in elements_attr:
if isinstance(element, Base):
# Recurse into each `Base` object within 'elements' or '@elements'.
# Recurse into each `Base` object within 'elements' or
# '@elements'.
yield from extract_base_and_transform(
element, current_id, transform_list.copy()
)
# Recursively process '@'-prefixed properties that are Base objects with 'elements'.
# This is a common pattern in older Speckle data models, such as those used for Revit commits.
# Recursively process '@'-prefixed properties that are Base objects
# with 'elements'.
# This is a common pattern in older Speckle data models, such as those
# used for Revit commits.
for attr_name in dir(base):
if attr_name.startswith("@"):
attr_value = getattr(base, attr_name)
# If the attribute is a Base object containing 'elements', recurse into it.
if isinstance(attr_value, Base) and hasattr(attr_value, "elements"):
# If the attribute is a Base object containing 'elements',
# recurse into it.
if isinstance(attr_value, Base) and hasattr(
attr_value, "elements"
):
yield from extract_base_and_transform(
attr_value, current_id, transform_list.copy()
)
+44
View File
@@ -1,7 +1,30 @@
"""This file contains the inputs for the function.
It is used to define the inputs for the function and to validate them.
"""
from enum import Enum
from pydantic import Field
from speckle_automate import AutomateBase
class PropertyMatchMode(Enum):
"""Controls how strictly parameter names must match."""
STRICT = "strict" # Exact parameter path match
FUZZY = "fuzzy" # Search all parameters ignoring hierarchy
MIXED = "mixed" # Exact match first, fuzzy fallback
class MinimumSeverity(str, Enum):
"""Enum for minimum severity level to report."""
INFO = "Info"
WARNING = "Warning"
ERROR = "Error"
class FunctionInputs(AutomateBase):
"""These are function author defined values.
@@ -15,3 +38,24 @@ class FunctionInputs(AutomateBase):
title="Spreadsheet URL",
description="This is the URL of the spreadsheet to check. It should be a TSV format data source.",
)
minimum_severity: MinimumSeverity = Field(
default=MinimumSeverity.INFO,
title="Minimum Severity Level",
description="Only report test results with this severity level or higher. Info will show all results, Warning will show warnings and errors, Error will show only errors.",
)
hide_skipped: bool = Field(
default=False,
title="Hide Skipped Tests",
description="If enabled, tests that were skipped (no matching objects found) will not be reported.",
)
# property_match_mode: PropertyMatchMode = Field(
# default=PropertyMatchMode.MIXED,
# title="Property Match Mode",
# description='Controls how strictly parameter names must match. ' +
# 'STRICT will only match exact parameter paths, ' +
# 'FUZZY will search all parameters ignoring hierarchy, ' +
# 'MIXED will exact match first, fuzzy fallback.'
# )
+22
View File
@@ -0,0 +1,22 @@
"""Defines mappings between spreadsheet predicates and rule methods."""
from src.rules import PropertyRules
# Mapping of input predicates to the corresponding methods in PropertyRules
PREDICATE_METHOD_MAP = {
"exists": PropertyRules.has_parameter.__name__,
"greater than": PropertyRules.is_parameter_value_greater_than.__name__,
"less than": PropertyRules.is_parameter_value_less_than.__name__,
"in range": PropertyRules.is_parameter_value_in_range.__name__,
"in list": PropertyRules.is_parameter_value_in_list.__name__,
"equal to": PropertyRules.is_equal_value.__name__,
"not equal to": PropertyRules.is_not_equal_value.__name__,
"is true": PropertyRules.is_parameter_value_true.__name__,
"is false": PropertyRules.is_parameter_value_false.__name__,
"is like": PropertyRules.is_parameter_value_like.__name__,
"identical to": PropertyRules.is_identical_value.__name__,
"contains": PropertyRules.is_parameter_value_containing.__name__,
"does not contain": (
PropertyRules.is_parameter_value_not_containing.__name__
),
}
+527
View File
@@ -0,0 +1,527 @@
"""Module for processing rules against Speckle objects and updating the automate context with the results.
This module implements the core rule processing logic that:
1. Validates rule structure and logic
2. Evaluates rule conditions against Speckle objects
3. Separates filtering conditions and final check conditions
4. Processes rule groups and tracks results
5. Reports results back to the Speckle Automate context
The rule processing follows a "filter then validate" approach:
- Filter conditions (WHERE, AND) narrow down which objects to check
- The final check condition (CHECK or last AND) determines pass/fail
"""
import json
from enum import Enum
from typing import Any
import pandas as pd
from pandas.core.groupby import DataFrameGroupBy
from speckle_automate import AutomationContext, ObjectResultLevel
from specklepy.objects.base import Base
from src.helpers import speckle_print
from src.inputs import MinimumSeverity
from src.predicates import PREDICATE_METHOD_MAP
from src.rules import PropertyRules
def validate_rule_structure(rule_group: pd.DataFrame) -> None:
"""Validates the structure and logic of a rule group.
This ensures the rule follows the proper format:
- First condition must be WHERE
- Following conditions can be AND
- Only one CHECK condition is allowed (and must be last)
Args:
rule_group: DataFrame containing the rule conditions
Raises:
ValueError: If rule structure is invalid
"""
if rule_group.empty:
return
# Validate Logic column exists
if "Logic" not in rule_group.columns:
raise ValueError("Rule must have a 'Logic' column")
# Get uppercase Logic values for case-insensitive comparison
logic_values = rule_group["Logic"].str.upper()
# Check if first condition is WHERE
if logic_values.iloc[0] != "WHERE":
raise ValueError(
f"Rule {rule_group.iloc[0]['Rule Number']} must start with WHERE"
)
# Count CHECK conditions
check_count = sum(1 for value in logic_values if value == "CHECK")
if check_count > 1:
raise ValueError(
f"Rule {rule_group.iloc[0]['Rule Number']} has multiple CHECK conditions"
)
# If CHECK exists, ensure it's the last condition
check_indices = logic_values[logic_values == "CHECK"].index
if check_count == 1 and check_indices[0] != rule_group.index[-1]:
raise ValueError(
f"CHECK must be the last condition in rule {rule_group.iloc[0]['Rule Number']}"
)
# Validate Logic values
valid_values = {"WHERE", "AND", "CHECK"}
invalid_values = set(logic_values.unique()) - valid_values
if invalid_values:
raise ValueError(f"Invalid Logic values found: {invalid_values}")
def evaluate_condition(
speckle_object: Base,
condition: pd.Series,
rule_number: str | None = None,
case_number: int | None = None,
) -> bool:
"""Evaluates a single condition against a Speckle object.
This function is the bridge between the rules defined in the spreadsheet
and the property checking methods in PropertyRules. It:
1. Extracts the property name, predicate, and value from the condition
2. Maps the predicate to the corresponding method in PropertyRules
3. Calls the method with the object, property name, and value
Args:
speckle_object: The Speckle object to evaluate against
condition: A pandas Series containing the condition details
- 'Property Name': The name of the property to check
- 'Predicate': The comparison operation (like 'equals',
'greater than')
- 'Value': The value to compare against
rule_number: For tracking, the rule number being evaluated
case_number: For tracking, the condition number within the rule
Returns:
True if the condition is met, False otherwise
"""
property_name = condition.get(
"Property Name", condition.get("Property Path")
)
predicate_key = condition["Predicate"]
value = condition["Value"]
# Debugging info
_ = rule_number
_ = case_number
# Look up the method name in the predicate map
# This map connects spreadsheet predicates to PropertyRules methods
if predicate_key in PREDICATE_METHOD_MAP:
method_name = PREDICATE_METHOD_MAP[predicate_key]
method = getattr(PropertyRules, method_name, None)
if method:
# Call the method with the object, property name, and value
return method(speckle_object, property_name, value)
return False
def get_filters_and_check(
rule_group: pd.DataFrame,
) -> tuple[pd.DataFrame, pd.Series]:
"""Separates rule conditions into filtering conditions and the final check condition.
This function handles two rule formats:
1. Explicit format: WHERE + AND... + CHECK
2. Legacy format: WHERE + AND... (last AND is implicitly the check)
This separation enables the "filter then validate" approach.
Args:
rule_group: DataFrame containing rule conditions
Returns:
Tuple containing (filter_conditions, final_check_condition)
"""
if rule_group.empty:
return pd.DataFrame(), pd.Series()
# Get uppercase Logic values for case-insensitive comparison
logic_values = rule_group["Logic"].str.upper()
# Look for explicit CHECK
check_conditions = rule_group[logic_values == "CHECK"]
has_explicit_check = not check_conditions.empty
if has_explicit_check:
# Use first CHECK condition as final check
final_check = check_conditions.iloc[0]
# All other conditions are filters
filters = rule_group[logic_values != "CHECK"]
else:
# Legacy behavior: use last AND as check if present
and_conditions = rule_group[logic_values == "AND"]
if not and_conditions.empty:
# Get the last AND as the check
final_check = and_conditions.iloc[-1]
# All conditions up to the last AND are filters
last_and_idx = and_conditions.index[-1]
filters = rule_group[rule_group.index < last_and_idx]
else:
# No AND conditions found, just use WHERE as filter
filters = rule_group
final_check = rule_group.iloc[
0
] # Default to first condition as check
return filters, final_check
def process_rule(
speckle_objects: list[Base], rule_group: pd.DataFrame
) -> tuple[list[Any], list[Any]] | tuple[list[Base], list[Base]]:
"""Processes a rule group against a list of Speckle objects.
This function implements the "filter then validate" approach:
1. Apply filter conditions sequentially to narrow down objects
2. Apply the final check condition to determine pass/fail
This approach is efficient for large models as it reduces the number
of objects that need full validation.
Args:
speckle_objects: List of Speckle objects to be processed
rule_group: DataFrame defining the filter and check conditions
Returns:
A tuple of lists (pass_objects, fail_objects)
"""
if not speckle_objects or rule_group.empty:
return [], []
try:
validate_rule_structure(rule_group)
except ValueError as e:
speckle_print(f"Rule validation error: {str(e)}")
return [], []
# Get filters and final check
filters, final_check = get_filters_and_check(rule_group)
# Start with all objects
filtered_objects = speckle_objects.copy()
rule_number = rule_group.iloc[0]["Rule Number"]
# Apply each filter condition sequentially
for index, (_, filter_condition) in enumerate(filters.iterrows()):
filtered_objects = [
obj
for obj in filtered_objects
if evaluate_condition(
speckle_object=obj,
condition=filter_condition,
rule_number=rule_number,
case_number=index,
)
]
# Early exit if no objects pass filters
if not filtered_objects:
return [], []
# For remaining objects, evaluate the final check
# This separates objects into pass/fail groups
pass_objects = []
fail_objects = []
for obj in filtered_objects:
if evaluate_condition(
speckle_object=obj,
condition=final_check,
rule_number=rule_number,
case_number=len(filters),
):
pass_objects.append(obj)
else:
fail_objects.append(obj)
return pass_objects, fail_objects
def apply_rules_to_objects(
speckle_objects: list[Base],
grouped_rules: DataFrameGroupBy,
automate_context: AutomationContext,
minimum_severity: MinimumSeverity = MinimumSeverity.INFO,
hide_skipped: bool = False,
) -> dict[str, tuple[list[Base], list[Base]]]:
"""Applies rules to objects and updates the automate context results.
This is the main orchestration function that:
1. Processes each rule group against all objects
2. Filters results based on severity levels
3. Attaches results to objects in the Speckle Automate context
4. Reports skipped rules (where no objects matched filters)
Args:
speckle_objects: The list of objects to which rules are applied
grouped_rules: The rules grouped by rule number
automate_context: Context manager for attaching results to objects
minimum_severity: Minimum severity level to report
hide_skipped: Whether to hide skipped rules in results
Returns:
Dictionary mapping rule IDs to (pass_objects, fail_objects) tuples
"""
grouped_results = {}
rules_processed = 0
severity_levels = {
MinimumSeverity.INFO: 0,
MinimumSeverity.WARNING: 1,
MinimumSeverity.ERROR: 2,
}
min_severity_level = severity_levels[minimum_severity]
for rule_id, rule_group in grouped_rules:
rule_id_str = str(rule_id) # Convert rule_id to string
rules_processed += 1
# Ensure rule_group has necessary columns
if "Message" not in rule_group.columns or (
"Report Severity" not in rule_group.columns
and "Severity" not in rule_group.columns
):
continue # Or raise an exception if these columns are mandatory
# Get the severity level for this rule
rule_severity = get_severity(rule_group.iloc[-1])
rule_severity_level = severity_levels[
MinimumSeverity(rule_severity.value)
]
# Check if the rule severity level meets the minimum severity level
# no point in processing lower severity rules
if rule_severity_level < min_severity_level:
continue
pass_objects, fail_objects = process_rule(speckle_objects, rule_group)
# For passing objects, only attach if we're showing all levels (INFO)
if minimum_severity == MinimumSeverity.INFO:
attach_results(
pass_objects,
rule_group.iloc[-1],
rule_id_str,
automate_context,
True,
)
# For failing objects, attach if they meet minimum severity threshold
if len(fail_objects) and rule_severity_level >= min_severity_level:
attach_results(
fail_objects,
rule_group.iloc[-1],
rule_id_str,
automate_context,
False,
)
if (
len(pass_objects) == 0
and len(fail_objects) == 0
and not hide_skipped
):
speckle_print(f"Rule {rule_id_str} Skipped")
newBase = Base()
newBase.id = "123"
automate_context.attach_info_to_objects(
category=f"Rule {rule_id_str} Skipped",
affected_objects=[newBase],
# This is a hack to get a rule to report with no valid objects
message=f"No objects found for rule {rule_id_str}",
metadata={},
)
grouped_results[rule_id_str] = (pass_objects, fail_objects)
# return pass_objects, fail_objects for each rule
return grouped_results
class SeverityLevel(Enum):
"""Enumeration for severity levels of rule results.
These severity levels determine how rule failures are displayed:
- INFO: Informational, no action required
- WARNING: Potential issue that should be reviewed
- ERROR: Critical issue requiring attention
"""
INFO = "Info"
WARNING = "Warning"
ERROR = "Error"
def get_severity(rule_info: pd.Series) -> SeverityLevel:
"""Convert a string severity to the corresponding SeverityLevel enum.
This function normalizes user input with robust handling for:
- Case insensitivity (e.g., "info", "WARNING""Info", "Warning")
- Shorthand mappings (e.g., "WARN""Warning")
- Whitespace handling
- Default fallback to ERROR for invalid input
Args:
rule_info: Series containing rule information with 'Report Severity'
key
Returns:
Appropriate SeverityLevel enum value
"""
severity = rule_info.get("Report Severity") or rule_info.get(
"Severity"
) # Extract severity from input data
# If severity is None or not a string (e.g., numeric input),
# default to ERROR
if not isinstance(severity, str):
return SeverityLevel.ERROR
severity = (
severity.strip().upper()
) # Remove leading/trailing spaces & normalize case
# Define a mapping for shorthand or alternate spellings
alias_map = {
"WARN": "WARNING", # Treat "WARN" as "WARNING"
}
# Replace shorthand values if applicable
severity = alias_map.get(severity, severity)
# Attempt to match with an existing SeverityLevel enum value
# (case-insensitive)
return next(
(level for level in SeverityLevel if level.value.upper() == severity),
SeverityLevel.ERROR, # Default to ERROR if no match is found
)
def get_metadata(
rule_id: str,
rule_info: pd.Series,
passed: bool,
speckle_objects: list[Base],
) -> dict[str, str | int | Any]:
"""Generates structured metadata for rule results.
This metadata is attached to objects in the Speckle platform and is:
1. Validated for JSON serializability
2. Structured for consistent representation
3. Includes key information about the rule and results
Args:
rule_id: Identifier for the rule
rule_info: Series containing rule information
passed: Boolean indicating if the rule passed
speckle_objects: List of Speckle objects affected
Returns:
Dictionary containing metadata if valid JSON serializable,
empty dict otherwise
"""
try:
metadata = {
"rule_id": rule_id,
"status": "PASS" if passed else "FAIL",
"severity": get_severity(rule_info).value,
"rule_message": format_message(rule_info),
"object_count": len(speckle_objects),
}
# Validate JSON serializability
json.dumps(metadata)
return metadata
except (TypeError, ValueError, json.JSONDecodeError) as e:
# Log the error for debugging purposes
print(f"Error creating metadata: {str(e)}")
return {}
def attach_results(
speckle_objects: list[Base],
rule_info: pd.Series,
rule_id: str,
context: AutomationContext,
passed: bool,
) -> None:
"""Attaches rule results to objects in the Speckle Automate context.
This function is the interface to the Speckle platform for reporting
results:
- For failing objects, attaches results with appropriate severity levels
- For passing objects, attaches informational results
- Includes structured metadata for consistent reporting
Args:
speckle_objects: The list of objects affected by the rule
rule_info: Information about the rule
rule_id: Identifier for the rule
context: The Speckle Automate context for result attachment
passed: Whether the objects passed the rule
"""
if not speckle_objects:
return
# Create structured metadata for onward data analysis uses
metadata = get_metadata(rule_id, rule_info, passed, speckle_objects)
message = format_message(rule_info)
if not passed:
speckle_print(rule_info["Report Severity"])
severity = (
ObjectResultLevel.WARNING
if rule_info["Report Severity"].capitalize() in ["Warning", "Warn"]
else ObjectResultLevel.ERROR
)
context.attach_result_to_objects(
category=f"Rule {rule_id}",
affected_objects=speckle_objects,
message=message,
level=severity,
metadata=metadata,
)
else:
context.attach_info_to_objects(
category=f"Rule {rule_id}",
affected_objects=speckle_objects,
message=message,
metadata=metadata,
)
def format_message(rule_info):
"""Format the message for the rule result.
Handles cases where the message might be None or NaN.
Args:
rule_info: Series containing rule information with 'Message' key
Returns:
Formatted message string
"""
message = (
str(rule_info["Message"])
if rule_info["Message"] is not None
and not pd.isna(rule_info["Message"])
else "No Message"
)
return message
+705 -558
View File
File diff suppressed because it is too large Load Diff
+193 -8
View File
@@ -1,18 +1,203 @@
"""Module for reading and processing rules from a cloud hosted TSV file.
This module handles the loading and processing of validation rules from external
spreadsheet data, enabling non-technical users to define and modify rules.
Key features:
1. Reading from hosted TSV files (e.g., from Google Sheets)
2. Processing rule numbers for consistent grouping
3. Handling mixed data types in spreadsheet columns
4. Validating rule structure and providing feedback
5. Grouping related rule conditions for execution
The spreadsheet format used follows a specific structure:
- Rule Number: Groups related conditions together
- Logic: WHERE/AND/CHECK to define condition relationships
- Property Name: The property path to check
- Predicate: The comparison operation (equals, greater than, etc.)
- Value: The value to compare against
- Message: The message to display for rule results
- Severity: INFO/WARNING/ERROR level for failures
"""
import traceback
import pandas as pd
from pandas import DataFrame
from pandas.core.groupby import DataFrameGroupBy
def read_rules_from_spreadsheet(url):
"""Reads a TSV file from a provided URL and returns a DataFrame.
def process_rule_numbers(df: DataFrame) -> DataFrame:
"""Process rule numbers in a DataFrame while preserving original rule identifiers.
This function handles various rule numbering scenarios:
1. Preserves existing rule numbers exactly as provided
2. Generates sequential numbers for missing rule numbers
3. Ensures all rows in a logical rule group have the same rule number
This is important because rule numbers determine how conditions are grouped
and executed together.
Args:
url (str): The URL to the TSV file.
df: DataFrame with columns including 'Rule Number' and 'Logic'
Returns:
DataFrame: Pandas DataFrame containing the TSV data.
DataFrame with processed rule numbers, where all related conditions
have the same rule number
"""
# Create a copy to avoid modifying original
df = df.copy()
# Initialize tracking variables
used_rule_nums = set()
processed_rule_nums = []
next_auto_num = 1 # For generating missing rule numbers only
# Find indices where Logic is 'WHERE' to identify rule group starts
where_indices = df[df["Logic"].str.upper() == "WHERE"].index
# Process each group
for i in range(len(where_indices)):
start_idx = where_indices[i]
end_idx = where_indices[i + 1] if i + 1 < len(where_indices) else len(df)
# Get slice of rows for this group
group_slice = df.iloc[start_idx:end_idx]
# Try to get rule number from first row, fall back to "Rule #"
group_rule_num = (
group_slice["Rule Number"].iloc[0] if not pd.isna(group_slice["Rule Number"].iloc[0]) else "Rule #"
)
if pd.isna(group_rule_num):
# If no rule number, generate next available number
while str(next_auto_num) in used_rule_nums:
next_auto_num += 1
group_rule_num = str(next_auto_num)
next_auto_num += 1
else:
# Keep the original rule number exactly as is
group_rule_num = str(group_rule_num)
# Update tracking
used_rule_nums.add(group_rule_num)
# Fill rule numbers for this group
processed_rule_nums.extend([group_rule_num] * len(group_slice))
# Update DataFrame with processed rule numbers
df["Rule Number"] = processed_rule_nums
return df
def validate_rule_numbers(df: DataFrame) -> list[str]:
"""Validate rule numbers and return any warnings or errors.
This checks for issues like:
1. Missing rule numbers
2. Non-integer rule numbers
3. Duplicate rule numbers
These validations help ensure rule integrity without being overly strict,
allowing for different user approaches to rule numbering.
Args:
df: DataFrame with processed rule numbers
Returns:
List of warning/error messages
"""
messages = []
# Check for missing rule numbers
if df["Rule Number"].isna().any():
messages.append("Warning: Some rules are missing rule numbers")
# # Check for non-integer rule numbers
# non_int_mask = df["Rule Number"].apply(lambda x: not pd.isna(x) and not float(x).is_integer())
# if non_int_mask.any():
# messages.append("Warning: Some rule numbers are not integers")
# Check for duplicate rule numbers in WHERE rows
where_rules = df[df["Logic"].str.upper() == "WHERE"]["Rule Number"]
duplicates = where_rules[where_rules.duplicated()]
if not duplicates.empty:
messages.append(f"Warning: Duplicate rule numbers found: {list(duplicates)}")
return messages
def read_rules_from_spreadsheet(url: str) -> tuple[DataFrameGroupBy, list[str]] | tuple[None, list[str]]:
"""Reads rules from a TSV file at the provided URL, processes them, and returns grouped rules.
This function is the main entry point for rule loading:
1. Reads the TSV file from the provided URL
2. Converts mixed type columns to appropriate types
3. Processes rule numbers for consistent grouping
4. Validates rule numbers and collects messages
5. Groups rules by rule number for execution
Args:
url: The URL to the TSV file containing rule definitions
Returns:
Tuple containing:
- DataFrameGroupBy object with rules grouped by rule number (or None if error)
- List of validation messages/warnings
"""
try:
# Since the output is a TSV, we use `pd.read_csv` with `sep='\t'` to specify tab-separated values.
return pd.read_csv(url, sep="\t")
# Read the TSV file
# The TSV format is chosen for compatibility with Google Sheets
# and other spreadsheet applications
df = pd.read_csv(url, sep="\t")
# Convert mixed type columns
# This handles inconsistencies in spreadsheet data
df = convert_mixed_columns(df)
# Process rule numbers
# This ensures all related conditions have the same rule number
df = process_rule_numbers(df)
# Get validation messages
# These are warnings about potential issues with the rules
messages = validate_rule_numbers(df)
# Group by rule number
# This creates a DataFrameGroupBy object that groups related conditions
grouped_rules = df.groupby("Rule Number")
return grouped_rules, messages
except Exception as e:
print(f"Failed to read the TSV from the URL: {e}")
return None
# Handle any errors in reading or processing the spreadsheet
traceback.print_exc()
return None, [f"Failed to read the TSV from the URL: {str(e)}:{e.with_traceback(None)}"]
def convert_mixed_columns(df: DataFrame) -> DataFrame:
"""Converts columns in a DataFrame to appropriate types based on their content.
This handles common issues with spreadsheet data:
1. Numeric columns that contain strings
2. Mixed type columns
3. Empty cells and NaN values
The approach is to convert each column appropriately:
- Numeric columns remain as numbers
- Other columns are converted to strings, with empty strings for missing values
Args:
df: The DataFrame whose columns are to be converted
Returns:
DataFrame with columns converted to appropriate types
"""
df = df.apply(
lambda c: c
if c.dropna().apply(lambda x: str(x).replace(".", "", 1).isdigit()).any()
else c.map(lambda x: "" if pd.isna(x) else str(x))
)
return df
File diff suppressed because it is too large Load Diff
+696
View File
@@ -0,0 +1,696 @@
[
{
"id": "46f06fef727d64a0bbcbd7ced51e0cd2",
"name": "Walls - W30(Fc24)",
"type": "W30(Fc24)",
"level": {
"name": "1FL",
"units": "mm",
"elevation": 0
},
"units": "mm",
"family": "Basic Wall",
"flipped": false,
"category": "Walls",
"elements": [],
"location": {
"id": "9c76b8de34382c9052965ee463f8374b",
"end": {
"x": 22400.000000000015,
"y": 20500,
"z": 0,
"id": "3455575bfd8939f264d295b61e74156f",
"units": "mm",
"speckle_type": "Objects.Geometry.Point",
"applicationId": null
},
"bbox": null,
"start": {
"x": 22400.000000000007,
"y": 15199.999999999998,
"z": 0,
"id": "d0c4fdb2e11cc825e7f05f9dc88a0be1",
"units": "mm",
"speckle_type": "Objects.Geometry.Point",
"applicationId": null
},
"units": "mm",
"domain": {
"id": "3b97feaad2dbcc2d894c9cec024a9bf2",
"end": 17.388451443569522,
"start": -3.552713668866051e-14,
"speckle_type": "Objects.Primitive.Interval",
"applicationId": null
},
"length": 5300.000000000002,
"speckle_type": "Objects.Geometry.Line",
"applicationId": null
},
"topLevel": {
"name": "1FL",
"units": "mm",
"elevation": 0
},
"__closure": {
"0ad4db1fe261a5b640ad9f315a46a6fc": 100,
"0d5518a7a3e63fe345e198ad5d6acc4e": 100,
"ec2040af4bd9c8619f9029a43df61a2e": 100
},
"elementId": "4479852",
"worksetId": "0",
"properties": {
"Parameters": {
"Type Parameters": {
"Text": {
"符号": {
"name": "符号",
"value": "W30",
"internalDefinitionName": "ee1f33e1-5506-4a64-b87b-7b98d30aea52"
}
},
"Other": {
"Family Name": {
"name": "Family Name",
"value": "Basic Wall",
"internalDefinitionName": "SYMBOL_FAMILY_NAME_PARAM"
},
"横筋ピッチ": {
"name": "横筋ピッチ",
"units": "General",
"value": 0,
"internalDefinitionName": "cd19c040-0788-4424-8a67-7127427e311f"
},
"縦筋ピッチ": {
"name": "縦筋ピッチ",
"units": "General",
"value": 0,
"internalDefinitionName": "6796d143-ef09-4441-9ecc-3411fe837a65"
}
},
"Rebar Set": {
"横筋 主筋径1": {
"name": "横筋 主筋径1",
"value": "D13",
"internalDefinitionName": "横筋 主筋径1"
},
"横筋 主筋径2": {
"name": "横筋 主筋径2",
"value": "D16",
"internalDefinitionName": "横筋 主筋径2"
},
"縦筋 主筋径1": {
"name": "縦筋 主筋径1",
"value": "D13",
"internalDefinitionName": "縦筋 主筋径1"
},
"縦筋 主筋径2": {
"name": "縦筋 主筋径2",
"value": "D16",
"internalDefinitionName": "縦筋 主筋径2"
},
"横筋 主筋ピッチ": {
"name": "横筋 主筋ピッチ",
"value": "200",
"internalDefinitionName": "横筋 主筋ピッチ"
},
"縦筋 主筋ピッチ": {
"name": "縦筋 主筋ピッチ",
"value": "200",
"internalDefinitionName": "縦筋 主筋ピッチ"
},
"開口補強筋 斜筋 本数": {
"name": "開口補強筋 斜筋 本数",
"value": "―",
"internalDefinitionName": "開口補強筋 斜筋 本数"
},
"開口補強筋 横筋 本数": {
"name": "開口補強筋 横筋 本数",
"value": "―",
"internalDefinitionName": "開口補強筋 横筋 本数"
},
"開口補強筋 縦筋 本数": {
"name": "開口補強筋 縦筋 本数",
"value": "―",
"internalDefinitionName": "開口補強筋 縦筋 本数"
}
},
"Structure": {
"Fc24 (0)": {
"units": "mm",
"function": "Structure",
"material": "Fc24",
"thickness": 300
}
},
"Construction": {
"Width": {
"name": "Width",
"units": "Millimeters",
"value": 300,
"internalDefinitionName": "WALL_ATTR_WIDTH_PARAM"
},
"Function": {
"name": "Function",
"value": "Interior",
"internalDefinitionName": "FUNCTION_PARAM"
},
"Wrapping at Ends": {
"name": "Wrapping at Ends",
"value": "None",
"internalDefinitionName": "WRAPPING_AT_ENDS_PARAM"
},
"Wrapping at Inserts": {
"name": "Wrapping at Inserts",
"value": "Do not wrap",
"internalDefinitionName": "WRAPPING_AT_INSERTS_PARAM"
}
},
"Identity Data": {
"Cost": {
"name": "Cost",
"units": "Currency",
"value": 0,
"internalDefinitionName": "ALL_MODEL_COST"
},
"Type Name": {
"name": "Type Name",
"value": "W30(Fc24)",
"internalDefinitionName": "SYMBOL_NAME_PARAM"
}
},
"IFC Parameters": {
"Type IfcGUID": {
"name": "Type IfcGUID",
"value": "0oqavz9$zDZxycDyfZdYh0",
"internalDefinitionName": "IFC_TYPE_GUID"
},
"Export Type to IFC": {
"name": "Export Type to IFC",
"value": "Default",
"internalDefinitionName": "IFC_EXPORT_ELEMENT_TYPE"
}
},
"Analytical Properties": {
"Roughness": {
"name": "Roughness",
"value": 1,
"internalDefinitionName": "ANALYTICAL_ROUGHNESS"
},
"Absorptance": {
"name": "Absorptance",
"units": "General",
"value": 0.1,
"internalDefinitionName": "ANALYTICAL_ABSORPTANCE"
},
"Thermal Mass": {
"name": "Thermal Mass",
"units": "Kilojoules per square meter Kelvin",
"value": 0,
"internalDefinitionName": "ANALYTICAL_THERMAL_MASS"
},
"Thermal Resistance (R)": {
"name": "Thermal Resistance (R)",
"units": "Square meter kelvins per watt",
"value": 0,
"internalDefinitionName": "ANALYTICAL_THERMAL_RESISTANCE"
},
"Heat Transfer Coefficient (U)": {
"name": "Heat Transfer Coefficient (U)",
"units": "Watts per square meter kelvin",
"value": 0,
"internalDefinitionName": "ANALYTICAL_HEAT_TRANSFER_COEFFICIENT"
}
},
"Materials and Finishes": {
"Structural Material": {
"name": "Structural Material",
"value": "Fc24",
"internalDefinitionName": "STRUCTURAL_MATERIAL_PARAM"
}
}
},
"Instance Parameters": {
"Other": {
"Type": {
"name": "Type",
"value": "W30(Fc24)",
"internalDefinitionName": "ELEM_TYPE_PARAM"
},
"Family": {
"name": "Family",
"value": "Basic Wall",
"internalDefinitionName": "ELEM_FAMILY_PARAM"
},
"Type Id": {
"name": "Type Id",
"value": "Basic Wall W30(Fc24)",
"internalDefinitionName": "SYMBOL_ID_PARAM"
},
"Family and Type": {
"name": "Family and Type",
"value": "Basic Wall W30(Fc24)",
"internalDefinitionName": "ELEM_FAMILY_AND_TYPE_PARAM"
}
},
"Phasing": {
"Phase Created": {
"name": "Phase Created",
"value": "フェーズ1",
"internalDefinitionName": "PHASE_CREATED"
}
},
"Dimensions": {
"Area": {
"name": "Area",
"units": "Square meters",
"value": 7.630000000000015,
"internalDefinitionName": "HOST_AREA_COMPUTED"
},
"Length": {
"name": "Length",
"units": "Millimeters",
"value": 5300.000000000001,
"internalDefinitionName": "CURVE_ELEM_LENGTH"
},
"Volume": {
"name": "Volume",
"units": "Cubic meters",
"value": 2.2890000000000135,
"internalDefinitionName": "HOST_VOLUME_COMPUTED"
}
},
"Structural": {
"Structural": {
"name": "Structural",
"value": "Yes",
"internalDefinitionName": "WALL_STRUCTURAL_SIGNIFICANT"
},
"Structural Usage": {
"name": "Structural Usage",
"value": "Bearing",
"internalDefinitionName": "WALL_STRUCTURAL_USAGE_PARAM"
},
"Rebar Cover - Other Faces": {
"name": "Rebar Cover - Other Faces",
"value": "Rebar Cover Settings 内壁(フレーム、柱、および耐力壁)",
"internalDefinitionName": "CLEAR_COVER_OTHER"
},
"Rebar Cover - Exterior Face": {
"name": "Rebar Cover - Exterior Face",
"value": "Rebar Cover Settings 内壁(フレーム、柱、および耐力壁)",
"internalDefinitionName": "CLEAR_COVER_EXTERIOR"
},
"Rebar Cover - Interior Face": {
"name": "Rebar Cover - Interior Face",
"value": "Rebar Cover Settings 内壁(フレーム、柱、および耐力壁)",
"internalDefinitionName": "CLEAR_COVER_INTERIOR"
},
"構造スリット 下端": {
"name": "構造スリット 下端",
"value": "No",
"internalDefinitionName": "0560d1bb-007b-45e4-b8c3-87f9c6a3d73c"
},
"構造スリット 始端": {
"name": "構造スリット 始端",
"value": "No",
"internalDefinitionName": "60795a90-8c91-4e11-90a0-b77b819c2403"
},
"構造スリット 終端": {
"name": "構造スリット 終端",
"value": "No",
"internalDefinitionName": "e3223f14-6158-4d4f-8697-0de2bcc7f737"
}
},
"Constraints": {
"Top Offset": {
"name": "Top Offset",
"units": "Millimeters",
"value": -600,
"internalDefinitionName": "WALL_TOP_OFFSET"
},
"Base Offset": {
"name": "Base Offset",
"units": "Millimeters",
"value": -2000,
"internalDefinitionName": "WALL_BASE_OFFSET"
},
"Location Line": {
"name": "Location Line",
"value": "Core Centerline",
"internalDefinitionName": "WALL_KEY_REF_PARAM"
},
"Room Bounding": {
"name": "Room Bounding",
"value": "Yes",
"internalDefinitionName": "WALL_ATTR_ROOM_BOUNDING"
},
"Top Constraint": {
"name": "Top Constraint",
"value": "1FL",
"internalDefinitionName": "WALL_HEIGHT_TYPE"
},
"Base Constraint": {
"name": "Base Constraint",
"value": "1FL",
"internalDefinitionName": "WALL_BASE_CONSTRAINT"
},
"Related to Mass": {
"name": "Related to Mass",
"value": "No",
"internalDefinitionName": "RELATED_TO_MASS"
},
"Top is Attached": {
"name": "Top is Attached",
"value": "No",
"internalDefinitionName": "WALL_TOP_IS_ATTACHED"
},
"Base is Attached": {
"name": "Base is Attached",
"value": "No",
"internalDefinitionName": "WALL_BOTTOM_IS_ATTACHED"
},
"Unconnected Height": {
"name": "Unconnected Height",
"units": "Millimeters",
"value": 1400,
"internalDefinitionName": "WALL_USER_HEIGHT_PARAM"
},
"Top Extension Distance": {
"name": "Top Extension Distance",
"units": "Millimeters",
"value": 0,
"internalDefinitionName": "WALL_TOP_EXTENSION_DIST_PARAM"
},
"Base Extension Distance": {
"name": "Base Extension Distance",
"units": "Millimeters",
"value": 0,
"internalDefinitionName": "WALL_BOTTOM_EXTENSION_DIST_PARAM"
}
},
"Identity Data": {
"Has Association": {
"name": "Has Association",
"value": "Yes",
"internalDefinitionName": "ANALYTICAL_ELEMENT_HAS_ASSOCIATION"
},
"SPECKLE_Classification": {
"name": "SPECKLE_Classification",
"value": "Wall",
"internalDefinitionName": "SPECKLE_Classification"
}
},
"IFC Parameters": {
"IfcGUID": {
"name": "IfcGUID",
"value": "0oqavz9$zDZxycDyfZdYsW",
"internalDefinitionName": "IFC_GUID"
},
"Export to IFC": {
"name": "Export to IFC",
"value": "By Type",
"internalDefinitionName": "IFC_EXPORT_ELEMENT"
}
},
"Cross-Section Definition": {
"Cross-Section": {
"name": "Cross-Section",
"value": "Vertical",
"internalDefinitionName": "WALL_CROSS_SECTION"
}
}
}
},
"Material Quantities": {
"Fc24": {
"area": 7630000.000000016,
"units": "mm",
"volume": 2289000000.0000134,
"materialName": "Fc24",
"materialClass": "コンクリート",
"materialCategory": "コンクリート"
}
}
},
"worksetName": "ワークセット1",
"displayValue": [
{
"__closure": null,
"referencedId": "ec2040af4bd9c8619f9029a43df61a2e",
"speckle_type": "reference"
}
],
"isStructural": true,
"speckle_type": "Objects.Data.DataObject:Objects.Data.RevitObject",
"applicationId": "32d24e7d-27ff-4d8f-bf26-37ca63da76cc-00445b6c",
"builtInCategory": "OST_Walls",
"totalChildrenCount": null
},
{
"id": "0ad4db1fe261a5b640ad9f315a46a6fc",
"data": [
3,
5,
0,
1,
3,
5,
1,
4,
3,
4,
2,
3,
3,
2,
4,
1,
3,
11,
12,
9,
3,
9,
10,
11,
3,
12,
13,
9,
3,
13,
6,
8,
3,
7,
8,
6,
3,
13,
8,
9,
3,
15,
16,
18,
3,
14,
15,
19,
3,
17,
18,
16,
3,
19,
20,
14,
3,
18,
19,
15,
3,
21,
22,
23,
3,
23,
24,
21,
3,
26,
27,
29,
3,
25,
26,
30,
3,
28,
29,
27,
3,
30,
31,
25,
3,
29,
30,
26,
3,
32,
33,
34,
3,
34,
35,
32
],
"speckle_type": "Speckle.Core.Models.DataChunk",
"applicationId": null
},
{
"id": "0d5518a7a3e63fe345e198ad5d6acc4e",
"data": [
22550.00031738281,
15049.999969482422,
-2000.000015258789,
22550.00031738281,
20274.999645996093,
-2000.000015258789,
22550.00031738281,
20499.999865722657,
-2000.000015258789,
22550.00031738281,
20499.999865722657,
-600.0000045776368,
22550.00031738281,
20274.999645996093,
-600.0000045776368,
22550.00031738281,
15049.999969482422,
-600.0000045776368,
22250.000024414065,
15049.999969482422,
-2000.000015258789,
22250.000024414065,
15049.999969482422,
-600.0000045776368,
22250.000024414065,
15350.000262451173,
-600.0000045776368,
22250.000024414065,
20274.999645996093,
-600.0000045776368,
22250.000024414065,
20499.999865722657,
-600.0000045776368,
22250.000024414065,
20499.999865722657,
-2000.000015258789,
22250.000024414065,
20274.999645996093,
-2000.000015258789,
22250.000024414065,
15350.000262451173,
-2000.000015258789,
22550.00031738281,
20499.999865722657,
-2000.000015258789,
22550.00031738281,
20274.999645996093,
-2000.000015258789,
22550.00031738281,
15049.999969482422,
-2000.000015258789,
22250.000024414065,
15049.999969482422,
-2000.000015258789,
22250.000024414065,
15350.000262451173,
-2000.000015258789,
22250.000024414065,
20274.999645996093,
-2000.000015258789,
22250.000024414065,
20499.999865722657,
-2000.000015258789,
22550.00031738281,
20499.999865722657,
-2000.000015258789,
22250.000024414065,
20499.999865722657,
-2000.000015258789,
22250.000024414065,
20499.999865722657,
-600.0000045776368,
22550.00031738281,
20499.999865722657,
-600.0000045776368,
22550.00031738281,
15049.999969482422,
-600.0000045776368,
22550.00031738281,
20274.999645996093,
-600.0000045776368,
22550.00031738281,
20499.999865722657,
-600.0000045776368,
22250.000024414065,
20499.999865722657,
-600.0000045776368,
22250.000024414065,
20274.999645996093,
-600.0000045776368,
22250.000024414065,
15350.000262451173,
-600.0000045776368,
22250.000024414065,
15049.999969482422,
-600.0000045776368,
22550.00031738281,
15049.999969482422,
-600.0000045776368,
22250.000024414065,
15049.999969482422,
-600.0000045776368,
22250.000024414065,
15049.999969482422,
-2000.000015258789,
22550.00031738281,
15049.999969482422,
-2000.000015258789
],
"speckle_type": "Speckle.Core.Models.DataChunk",
"applicationId": null
},
{
"id": "ec2040af4bd9c8619f9029a43df61a2e",
"area": 0,
"bbox": null,
"faces": [
{
"__closure": null,
"referencedId": "0ad4db1fe261a5b640ad9f315a46a6fc",
"speckle_type": "reference"
}
],
"units": "mm",
"colors": [],
"volume": 0,
"vertices": [
{
"__closure": null,
"referencedId": "0d5518a7a3e63fe345e198ad5d6acc4e",
"speckle_type": "reference"
}
],
"__closure": {
"0ad4db1fe261a5b640ad9f315a46a6fc": 100,
"0d5518a7a3e63fe345e198ad5d6acc4e": 100
},
"speckle_type": "Objects.Geometry.Mesh",
"applicationId": "d2c33253-d0ed-4682-9633-e083daaa87db",
"textureCoordinates": []
}
]
+89 -18
View File
@@ -1,24 +1,95 @@
import os
from dotenv import load_dotenv
import pytest
from specklepy.objects.base import Base
def pytest_configure(config):
load_dotenv(dotenv_path=".env")
@pytest.fixture
def v3_wall():
"""Creates a v3-style Speckle wall object."""
wall = Base()
wall.id = "46f06fef727d64a0bbcbd7ced51e0cd2"
wall.name = "Walls - W30(Fc24)"
wall.type = "W30(Fc24)"
wall.units = "mm"
wall.family = "Basic Wall"
wall.flipped = False
wall.category = "Walls"
wall.elementId = "4479852"
wall.worksetId = "0"
token_var = "SPECKLE_TOKEN"
server_var = "SPECKLE_SERVER_URL"
token = os.getenv(token_var)
server = os.getenv(server_var)
# Create location geometry
wall.location = Base()
wall.location.id = "9c76b8de34382c9052965ee463f8374b"
wall.location.start = Base()
wall.location.start.x = 22400.000000000007
wall.location.start.y = 15199.999999999998
wall.location.start.z = 0
wall.location.start.id = "d0c4fdb2e11cc825e7f05f9dc88a0be1"
wall.location.start.units = "mm"
wall.location.start.speckle_type = "Objects.Geometry.Point"
wall.location.end = Base()
wall.location.end.x = 22400.000000000015
wall.location.end.y = 20500
wall.location.end.z = 0
wall.location.end.id = "3455575bfd8939f264d295b61e74156f"
wall.location.end.units = "mm"
wall.location.end.speckle_type = "Objects.Geometry.Point"
wall.location.units = "mm"
wall.location.domain = Base()
wall.location.domain.id = "3b97feaad2dbcc2d894c9cec024a9bf2"
wall.location.domain.end = 17.388451443569522
wall.location.domain.start = -3.552713668866051e-14
wall.location.domain.speckle_type = "Objects.Primitive.Interval"
wall.location.length = 5300.000000000002
wall.location.speckle_type = "Objects.Geometry.Line"
if not token:
raise ValueError(f"Cannot run tests without a {token_var} environment variable")
# Create level references
wall.level = Base()
wall.level.name = "1FL"
wall.level.units = "mm"
wall.level.elevation = 0
if not server:
raise ValueError(
f"Cannot run tests without a {server_var} environment variable"
)
wall.topLevel = Base()
wall.topLevel.name = "1FL"
wall.topLevel.units = "mm"
wall.topLevel.elevation = 0
# Set the token as an attribute on the config object
config.SPECKLE_TOKEN = token
config.SPECKLE_SERVER_URL = server
# Create properties structure
wall.properties = Base()
wall.properties.Parameters = Base()
wall.properties.Parameters["Type Parameters"] = Base()
# Add Text section
wall.properties.Parameters["Type Parameters"].Text = Base()
wall.properties.Parameters["Type Parameters"].Text["符号"] = {
"name": "符号",
"value": "W30",
"internalDefinitionName": "ee1f33e1-5506-4a64-b87b-7b98d30aea52",
}
# Add Structure section
wall.properties.Parameters["Type Parameters"].Structure = Base()
wall.properties.Parameters["Type Parameters"].Structure["Fc24 (0)"] = {
"units": "mm",
"function": "Structure",
"material": "Fc24",
"thickness": 300,
}
# Add Construction section
wall.properties.Parameters["Type Parameters"].Construction = Base()
wall.properties.Parameters["Type Parameters"].Construction.Width = {
"name": "Width",
"units": "Millimeters",
"value": 300,
"internalDefinitionName": "WALL_ATTR_WIDTH_PARAM",
}
# Add Instance Parameters
wall.properties.Parameters["Instance Parameters"] = Base()
wall.properties.Parameters["Instance Parameters"].Structural = Base()
wall.properties.Parameters["Instance Parameters"].Structural.Structural = {
"name": "Structural",
"value": "Yes",
}
return wall
-32
View File
@@ -1,32 +0,0 @@
"""Run integration tests with a speckle server."""
from speckle_automate import (
AutomationContext,
AutomationRunData,
AutomationStatus,
run_function
)
from src.inputs import FunctionInputs
from src.function import automate_function
def test_function_run(test_automation_run_data: AutomationRunData, test_automation_token: str):
"""Run an integration test for the automate function."""
automation_context = AutomationContext.initialize(
test_automation_run_data, test_automation_token
)
default_url: str = (
"https://docs.google.com/spreadsheets/d/e/2PACX-1vSFmjLfqxPKXJHg-wEs1cp_nJEJJhESGVTLCvWLG_"
"IgIuRZ4CmMDCSceOYFvuo8IqcmT4sj9qPiLfCx/pub?gid=0&single=true&output=tsv"
)
automate_sdk = run_function(
automation_context,
automate_function,
FunctionInputs(spreadsheet_url=default_url),
)
assert automate_sdk.run_status == AutomationStatus.SUCCEEDED
# cli command to run just this test with pytest: pytest tests/local_test_exercise2.py::test_function_run
+86
View File
@@ -0,0 +1,86 @@
from typing import Any
import pytest
from src.rules import PropertyRules
class TestValueComparison:
"""Test suite for value comparison functionality."""
@pytest.mark.parametrize(
"value1, value2",
[
# Basic numeric strings
("1400", 1400.0),
("1400.0", 1400),
("1400.00", 1400),
# Whitespace handling
(" 1400 ", 1400.0),
(" 1400 ", 1400.0),
("\t1400\n", 1400.0),
# Negative numbers
("-1400", -1400.0),
(" -1400 ", -1400.0),
("-1400.0", -1400),
# Zero handling
("0", 0.0),
("-0", 0.0),
("0.0", 0),
# Simple integers
("1", 1),
("1.0", 1),
],
)
def test_numeric_string_comparison(self, value1: Any, value2: Any):
"""Test comparison of numeric strings with numbers."""
assert PropertyRules.compare_values(value1, value2)
# Test reverse comparison
assert PropertyRules.compare_values(value2, value1)
@pytest.mark.parametrize(
"value1, value2, expected",
[
("Yes", True, True),
("No", False, True),
("yes", True, True),
("no", False, True),
("YES", True, True),
("NO", False, True),
("true", True, True),
("false", False, True),
("True", True, True),
("False", False, True),
],
)
def test_boolean_string_comparison(self, value1: str, value2: bool, expected: bool):
"""Test comparison of boolean strings with booleans."""
assert PropertyRules.compare_values(value1, value2) == expected
# Test reverse comparison
assert PropertyRules.compare_values(value2, value1) == expected
@pytest.mark.parametrize(
"value1, value2, case_sensitive, expected",
[
("hello", "HELLO", False, True),
("hello", "HELLO", True, False),
("Hello", "hello", False, True),
("Hello", "Hello", True, True),
],
)
def test_string_comparison(self, value1: str, value2: str, case_sensitive: bool, expected: bool):
"""Test string comparison with case sensitivity options."""
assert PropertyRules.compare_values(value1, value2, case_sensitive=case_sensitive) == expected
@pytest.mark.parametrize(
"value1, value2, tolerance, expected",
[
(1.0001, 1.0, 1e-3, True),
(1.0001, 1.0, 1e-6, False),
(1.00000001, 1.0, 1e-6, True),
(-1.0001, -1.0, 1e-3, True),
],
)
def test_float_comparison_tolerance(self, value1: float, value2: float, tolerance: float, expected: bool):
"""Test float comparison with different tolerance levels."""
assert PropertyRules.compare_values(value1, value2, tolerance=tolerance) == expected
+40 -20
View File
@@ -1,31 +1,51 @@
"""Run integration tests with a speckle server."""
from pydantic import SecretStr
from speckle_automate import (
AutomationContext,
AutomationRunData,
AutomationStatus,
run_function
run_function,
)
from speckle_automate.fixtures import * # noqa: F401, F403
from main import FunctionInputs, automate_function
from speckle_automate.fixtures import *
from src.function import automate_function
from src.helpers import speckle_print
from src.inputs import FunctionInputs, MinimumSeverity
def test_function_run(test_automation_run_data: AutomationRunData, test_automation_token: str):
"""Run an integration test for the automate function."""
automation_context = AutomationContext.initialize(
test_automation_run_data, test_automation_token
)
automate_sdk = run_function(
automation_context,
automate_function,
FunctionInputs(
forbidden_speckle_type="None",
whisper_message=SecretStr("testing automatically"),
),
)
class TestFunction:
"""Test suite for the automate function."""
assert automate_sdk.run_status == AutomationStatus.SUCCEEDED
def test_function_run(
self,
test_automation_run_data: AutomationRunData,
test_automation_token: str,
):
"""Run an integration test for the automate function.
Args:
test_automation_run_data (AutomationRunData): The automation run
data provided by sdk.
test_automation_token (str): The automation token.
"""
speckle_print(str(test_automation_run_data))
speckle_print(str(test_automation_token))
"""Run an integration test for the automate function."""
automation_context = AutomationContext.initialize(
test_automation_run_data, test_automation_token
)
default_url: str = "https://model-checker.speckle.systems/r/7YhnQyQNP_Ydv97QCwHbj7BWHrNkG022bez_jVkxbYs/tsv"
automate_sdk = run_function(
automation_context,
automate_function,
FunctionInputs(
spreadsheet_url=default_url,
minimum_severity=MinimumSeverity.INFO,
hide_skipped=True,
),
)
assert automate_sdk.run_status == AutomationStatus.SUCCEEDED
+524
View File
@@ -0,0 +1,524 @@
"""Test suite for parameter handling functionality."""
import pytest
from specklepy.objects.base import Base
from src.rules import PropertyRules
class TestParameterHandling:
"""Test suite for parameter handling functionality."""
@pytest.fixture
def test_objects(self) -> Base:
"""Pytest fixture to provide test objects."""
# Create a mock Base object with the required structure
v3_obj = Base()
v3_obj.properties = {
"Parameters": {
"category": "Walls",
"Width": 300,
"Construction": {"Width": 300},
"Instance Parameters": {
"Dimensions": {"Length": 5300.000000000001},
"Structural": {"Structural": {"value": "Yes"}},
"Room Bounding": {"value": "Yes"},
"top is attached": {"value": "No"},
},
"Type Parameters": {
"Structure": {"Fc24 (0)": {"thickness": 300}},
"Text": {"符号": {"value": "W30"}},
},
"Type": "W30(Fc24)",
}
}
v3_obj.speckle_type = "Revit"
return v3_obj
def test_deserialization_structure(self, test_objects):
"""Test that objects are properly deserialized with correct structure."""
v3_obj = test_objects
# Check base class type
assert isinstance(v3_obj, Base), f"Expected {v3_obj} to be an instance of Base"
# Check v3 structure
assert hasattr(v3_obj, "properties"), (
"v3_obj should have 'properties' attribute"
)
assert v3_obj.properties is not None, "v3_obj.properties should not be None"
assert "Parameters" in v3_obj.properties, (
"'Parameters' key should exist in v3_obj.properties"
)
@pytest.mark.parametrize(
"param_name, expected_result",
[
("category", True), # Test parameters that should exist
("Width", True), # Test nested parameters
("non_existent_param", False), # Test non-existent parameters
],
)
def test_v3_parameter_exists(self, test_objects, param_name, expected_result):
"""Test parameter existence checking in v3 objects."""
v3_obj = test_objects
assert PropertyRules.has_parameter(v3_obj, param_name) == expected_result
@pytest.mark.parametrize(
"param_name_1, param_name_2",
[
# Test direct value access
(
"location.length",
"location.length",
),
# Test .value key access
(
"Type Parameters.Text.符号",
"Type Parameters.Text.符号.value",
),
],
)
def test_v3_parameter_search_equivalence(
self,
v3_wall,
param_name_1,
param_name_2,
):
"""Test parameter existence checking equivalence in v3 objects."""
assert PropertyRules.get_parameter_value(
v3_wall, param_name_1
) == PropertyRules.get_parameter_value(v3_wall, param_name_2)
@pytest.mark.parametrize(
"param_name, expected_value, default_value",
[
# Test direct parameters
("category", "Walls", None),
# Test nested parameters - using both internal and friendly names
("Construction.Width", 300, None),
# Test parameters with units
(
"Instance Parameters.Dimensions.Length",
5300.000000000001,
None,
),
# Test non-existent parameters with a default value
(
"properties.Parameters.non_existent",
"default",
"default",
),
],
)
def test_parameter_value_retrieval(
self,
test_objects,
param_name,
expected_value,
default_value,
):
"""Test parameter value retrieval from v3 objects."""
v3_obj = test_objects
result = PropertyRules.get_parameter_value(
v3_obj,
param_name,
default_value=default_value,
)
assert result == expected_value
@pytest.mark.parametrize(
"param_name, expected_value, expected_result",
[
("category", "Walls", True), # Test exact match
("Width", 300, True), # Test numeric match
("category", "Windows", False), # Test non-match
],
)
def test_v3_parameter_value_matching(
self,
test_objects,
param_name,
expected_value,
expected_result,
):
"""Test parameter value matching in v3 objects."""
v3_obj = test_objects
assert (
PropertyRules.is_parameter_value(
v3_obj,
param_name,
expected_value,
)
== expected_result
)
@pytest.mark.parametrize(
"comparison_func, param_name, value, expected_result",
[
(
PropertyRules.is_parameter_value_greater_than,
"Width",
"200",
True,
), # Test greater than
(
PropertyRules.is_parameter_value_less_than,
"Width",
"400",
True,
), # Test less than
(
PropertyRules.is_parameter_value_in_range,
"Width",
"200,400",
True,
), # Test in range
],
)
def test_v3_parameter_numeric_comparisons(
self, test_objects, comparison_func, param_name, value, expected_result
):
"""Test numeric parameter comparisons in v3 objects."""
v3_obj = test_objects
assert comparison_func(v3_obj, param_name, value) == expected_result
@pytest.mark.parametrize(
"param_name, pattern, fuzzy, expected_result",
[
("category", "^Walls$", False, True), # Test exact pattern matches
("category", "Walls", True, True), # Test fuzzy matches
("category", "Wall", False, True), # Test partial pattern matches
("category", "^Windows$", False, False), # Test non-matches
],
)
def test_v3_parameter_value_like(
self, test_objects, param_name, pattern, fuzzy, expected_result
):
"""Test pattern matching on parameter values in v3 objects."""
v3_obj = test_objects
assert (
PropertyRules.is_parameter_value_like(
v3_obj, param_name, pattern, fuzzy=fuzzy
)
== expected_result
)
@pytest.mark.parametrize(
"param_name, valid_list, expected_result",
[
(
"category",
["Walls", "Windows", "Doors"],
True,
), # Test value in list
(
"category",
"Walls,Windows,Doors",
True,
), # Test comma-separated string list
(
"category",
["Windows", "Doors"],
False,
), # Test value not in list
],
)
def test_v3_parameter_lists(
self,
test_objects,
param_name,
valid_list,
expected_result,
):
"""Test list-based parameter checks in v3 objects."""
v3_obj = test_objects
assert (
PropertyRules.is_parameter_value_in_list(
v3_obj,
param_name,
valid_list,
)
== expected_result
)
@pytest.mark.parametrize(
"param_name, expected_result",
[
("Room Bounding", True), # Test true values
("top is attached", False), # Test false values
("Top is Attached", False), # Case sensitivity test
],
)
def test_v3_boolean_parameters(
self,
test_objects,
param_name,
expected_result,
):
"""Test boolean parameter checks in v3 objects."""
v3_obj = test_objects
if expected_result:
assert PropertyRules.is_parameter_value_true(v3_obj, param_name)
else:
assert PropertyRules.is_parameter_value_false(v3_obj, param_name)
@pytest.mark.parametrize(
"attribute, value, expected",
[
# Test numeric value comparisons
(
"Type Parameters.Structure.Fc24 (0).thickness",
300,
True,
),
(
"Instance Parameters.Dimensions.Length",
5300.000000000002,
True,
),
(
"Instance Parameters.Dimensions.Length",
5300,
True,
),
# Test string value comparisons
(
"Type Parameters.Text.符号.value",
"W30",
True,
),
(
"Instance Parameters.Structural.Structural.value",
"Yes",
True,
),
# Test non-matches
(
"Type Parameters.Structure.Fc24 (0).thickness",
301,
False,
),
(
"nonexistent_param",
"any_value",
False,
),
],
)
def test_v3_parameter_value_comparisons(
self,
test_objects,
attribute,
value,
expected,
):
"""Test value comparisons using v3 wall parameters."""
assert PropertyRules.is_equal_value(test_objects, attribute, value) == expected
@pytest.mark.parametrize(
"wall, attribute, value, expected",
[
# V3 wall tests
(
"v3_wall",
"Type Parameters.Structure.Fc24 (0).thickness",
300,
True,
),
("v3_wall", "type", "W30(Fc24)", True),
(
"v3_wall",
"Type Parameters.Structure.Fc24 (0).thickness",
300.0001,
False,
),
(
"v3_wall",
"location.length",
5300.000000000002,
False,
),
(
"v3_wall",
"location.length",
5300,
False,
),
],
)
def test_identical_comparisons(
self,
test_objects,
wall,
attribute,
value,
expected,
):
"""Test identical value comparisons on v3 wall."""
if attribute == "type":
# Use case-insensitive comparison for type parameter
assert (
PropertyRules.is_equal_value(
test_objects,
attribute,
value,
)
== expected
)
else:
# Use strict comparison for other parameters
assert (
PropertyRules.is_identical_value(
test_objects,
attribute,
value,
)
== expected
)
@pytest.mark.parametrize(
"wall, attribute, value",
[
# V3 wall tests
(
"v3_wall",
"Type Parameters.Structure.Fc24 (0).thickness",
301,
),
(
"v3_wall",
"Type Parameters.Text.符号.value",
"W31",
),
(
"v3_wall",
"nonexistent_param",
"any_value",
),
],
)
def test_not_equal_comparisons(
self,
test_objects,
wall,
attribute,
value,
):
"""Test not equal comparisons on v3 wall."""
assert PropertyRules.is_not_equal_value(test_objects, attribute, value)
@pytest.mark.parametrize(
"attribute, value, expected_equal, expected_identical",
[
# Test Yes/No conversion in equals (should convert)
(
"Instance Parameters.Structural.Structural.value",
True,
True,
False,
), # Yes vs True
(
"Instance Parameters.Structural.Structural.value",
"Yes",
True,
True,
), # Yes vs "Yes"
(
"Instance Parameters.Structural.Structural.value",
"yes",
True,
False,
), # Yes vs "yes"
],
)
def test_boolean_conversions(
self,
test_objects,
attribute,
value,
expected_equal,
expected_identical,
):
"""Test conversion of Yes/No strings to boolean values."""
assert (
PropertyRules.is_equal_value(test_objects, attribute, value)
== expected_equal
)
assert (
PropertyRules.is_identical_value(test_objects, attribute, value)
== expected_identical
)
@pytest.mark.parametrize(
"wall, attribute, expected_value",
[
# V3 wall tests
(
"v3_wall",
"Type Parameters.Structure.Fc24 (0).thickness",
"300",
),
(
"v3_wall",
"Instance Parameters.Dimensions.Length",
"5300.000000000002",
),
],
)
def test_numeric_string_handling(
self,
test_objects,
wall,
attribute,
expected_value,
):
"""Test handling of numeric strings in v3 wall."""
assert PropertyRules.is_equal_value(
test_objects,
attribute,
expected_value,
)
@pytest.mark.parametrize(
"param_name, substring, expected_result",
[
(
"speckle_type",
"Revit",
True,
), # Should pass as it does not contain Revit
(
"speckle_type",
"NotPresent",
True,
), # Should pass as it doesn't contain
(
"speckle_type",
"",
False,
), # Should fail as empty string is contained in any string
(
"non_existent",
"anything",
True,
), # Should pass as non-existent can't contain
],
)
def test_parameter_value_not_contains(
self,
test_objects,
param_name,
substring,
expected_result,
):
"""Test negative substring matching on parameter values."""
v3_obj = test_objects
assert (
PropertyRules.is_parameter_value_not_containing(
v3_obj,
param_name,
substring,
)
== expected_result
)
+36
View File
@@ -0,0 +1,36 @@
"""Tests for rule processing functionality."""
import pandas as pd
import pytest
@pytest.fixture
def explicit_check_rule():
"""Create a rule using explicit CHECK format."""
return pd.DataFrame(
{
"Rule Number": [1, 1, 1],
"Logic": ["WHERE", "AND", "CHECK"],
"Property Name": ["category", "width", "material"],
"Predicate": ["matches", "greater than", "matches"],
"Value": ["Walls", "200", "Concrete"],
"Message": ["Test message", "", ""],
"Severity": ["Error", "", ""],
}
)
@pytest.fixture
def legacy_rule():
"""Create a rule using legacy format (last AND is implicit check)."""
return pd.DataFrame(
{
"Rule Number": [1, 1, 1],
"Logic": ["WHERE", "AND", "AND"],
"Property Name": ["category", "width", "material"],
"Predicate": ["matches", "greater than", "matches"],
"Value": ["Walls", "200", "Concrete"],
"Message": ["Test message", "", ""],
"Severity": ["Error", "", ""],
}
)
+33
View File
@@ -0,0 +1,33 @@
import pandas as pd
import pytest
from src.rule_processor import SeverityLevel, get_severity
@pytest.mark.parametrize(
"input_severity, expected_enum",
[
("INFO", SeverityLevel.INFO),
("info", SeverityLevel.INFO),
("Info", SeverityLevel.INFO),
("WARNING", SeverityLevel.WARNING),
("warning", SeverityLevel.WARNING),
("Warning", SeverityLevel.WARNING),
("ERROR", SeverityLevel.ERROR),
("error", SeverityLevel.ERROR),
("Error", SeverityLevel.ERROR),
("WARN", SeverityLevel.WARNING),
("warn", SeverityLevel.WARNING),
("Critical", SeverityLevel.ERROR), # Invalid → Defaults to ERROR
("Severe", SeverityLevel.ERROR), # Invalid → Defaults to ERROR
("", SeverityLevel.ERROR), # Empty string → Defaults to ERROR
(None, SeverityLevel.ERROR), # None → Defaults to ERROR
(1.0, SeverityLevel.ERROR), # None → Defaults to ERROR
],
)
def test_severity_conversion(input_severity, expected_enum):
"""Test various user inputs for severity and check expected outputs."""
rule_info = pd.Series({"Report Severity": input_severity})
severity = get_severity(rule_info)
assert severity == expected_enum, f"Failed for input: {input_severity}"