diff --git a/DEVELOPER_README.md b/DEVELOPER_README.md index 6ceae07..3eee042 100644 --- a/DEVELOPER_README.md +++ b/DEVELOPER_README.md @@ -1,56 +1,377 @@ -# Checker Function Development Guide +# Checker Function Developer Guide -## Setup +This document provides technical details for developers working on the Speckle Checker Automate function. -1. Install dependencies: +## Project Overview + +The Checker function enables validation of Speckle objects against user-defined rules in a spreadsheet. It's designed to +be flexible, supporting various object schemas including both v2 and v3 Speckle APIs. + +## Setup Development Environment + +### Prerequisites + +- Python 3.10+ +- Poetry for dependency management + +### Installation + +1. Clone the repository +2. Install dependencies: + ```bash + poetry install + ``` +3. Activate the virtual environment: + ```bash + poetry shell + ``` + +### Test Automation Environment + +The project uses Speckle's [Test Automation feature](https://speckle.guide/automate/function-testing.html) to run +integration tests against real Speckle data. This provides a sandboxed environment to validate the function's business +logic without triggering actual automations. + +#### Setting Up a Test Automation + +1. Navigate to your Speckle project +2. Go to the **Automations** tab +3. Click **New Automation** +4. Select **Create Test Automation** in the bottom left +5. Follow the configuration steps + +Note: To create a test automation, you must: + +- Be an owner of the Speckle project +- Have published this function to the Function Library +- Have at least one release for the function + +#### Environment Configuration + +For local integration testing, create a `.env` file in the project root with these variables: + +``` +# Your Personal Access Token from Speckle +SPECKLE_TOKEN=your_speckle_token + +# The Speckle server URL +SPECKLE_SERVER_URL=https://app.speckle.systems + +# From the test automation URL: /projects/[project-id]/automations/[automation-id] +SPECKLE_PROJECT_ID=your_project_id +SPECKLE_AUTOMATION_ID=your_automation_id +``` + +This configuration allows the test suite to: + +1. Connect to your test automation via the Speckle API +2. Run the function locally against real Speckle data +3. Submit results to the test automation for validation + +For detailed instructions, refer to +the [official documentation on function testing](https://speckle.guide/automate/function-testing.html#how-to-create-a-test-automation). + +#### Running Integration Tests + +With the `.env` file configured: ```bash -poetry shell && poetry install +# Run the integration tests +pytest test_function.py ``` -2. Configure `.env`: +The SDK utilities will automatically: -``` -SPECKLE_TOKEN=your_speckle_token -SPECKLE_SERVER_URL=app.speckle.systems +- Connect to your test automation +- Execute your function with the specified test data +- Submit results back to Speckle + +Test results will be visible on the automation page in the Speckle UI. + +#### Unit Tests + +For unit tests that don't require a full Speckle connection, you can run: + +```bash +# Run unit tests only +pytest test_comparisons.py test_rule_processing.py ``` -Get test automation details from app.speckle.systems +Note: The `.env` file should never be committed to version control (it's included in .gitignore) ## Project Structure -- `function.py`: Main business logic -- `rules.py`: Rule definitions and processing -- `inputs.py`: Function input schema -- `helpers.py`: Utility functions -- `spreadsheet.py`: TSV handling +``` +├── main.py # Entry point for Automate +├── src/ +│ ├── function.py # Main function logic +│ ├── inputs.py # Function input schema +│ ├── helpers.py # Utility functions +│ ├── filters.py # Object filtering functions +│ ├── rules.py # Rule definitions and property handling +│ ├── predicates.py # Predicate mapping for spreadsheet values +│ ├── rule_processor.py # Rule application and result handling +│ └── spreadsheet.py # TSV file parsing +├── tests/ +│ ├── conftest.py # Test fixtures +│ ├── test_function.py # Main function tests +│ ├── test_comparisons.py # Value comparison tests +│ ├── test_parameters.py # Parameter handling tests +│ └── test_rule_processing.py # Rule processing tests +├── pyproject.toml # Project dependencies +└── poetry.lock # Locked dependencies +``` + +## Core Components + +### 1. Function Execution Flow + +The main execution flow is defined in `function.py`: + +1. `automate_function()` receives context and inputs from Automate +2. Retrieves Speckle objects via `automate_context.receive_version()` +3. Flattens the object tree using `flatten_base()` +4. Loads rules from the spreadsheet URL via `read_rules_from_spreadsheet()` +5. Applies rules to objects using `apply_rules_to_objects()` +6. Reports results via the Automate context + +### 2. Rule Processing + +Rules are processed through several stages: + +1. **Spreadsheet Parsing** (`spreadsheet.py`): + - Reads TSV data + - Groups rules by rule number + - Validates rule structure + +2. **Rule Application** (`rule_processor.py`): + - Processes rule logic (WHERE, AND, CHECK) + - Evaluates conditions against objects + - Attaches results to objects in Automate context + +3. **Property Rules** (`rules.py`): + - Handles property lookups in objects + - Implements comparison logic + - Supports both v2 and v3 Speckle schemas + +### 3. Property Access System + +The system uses a flexible property access mechanism that works with different Speckle schemas: + +- **V2 Schema**: Properties in `parameters` dictionary with internal definition names +- **V3 Schema**: Properties in nested `properties.Parameters` structure + +The `PropertyRules` class provides methods to: + +- Find properties by path or name +- Extract values with appropriate type conversion +- Perform comparisons with tolerance and type handling + +## Test-Driven Development for Rules + +The test infrastructure is designed to support Test-Driven Development (TDD) when creating new rules or extending +functionality. This approach is especially powerful for rule development as it allows you to verify behavior against +known test objects. + +### Using Test Fixtures for Rule Development + +The `conftest.py` file contains test fixtures that provide sample Speckle objects for testing: + +```python +@pytest.fixture +def v2_wall(): + """Creates a v2-style Speckle wall object""" + wall = Base() + wall.id = "cdb18060dc48281909e94f0f1d8d3cc0" + wall.type = "W30(Fc24)" + wall.units = "mm" + wall.family = "Basic Wall" + wall.height = 1400 + wall.flipped = False + wall.category = "Walls" + # ... more properties + + return wall + + +@pytest.fixture +def v3_wall(): + """Creates a v3-style Speckle wall object""" + wall = Base() + wall.id = "46f06fef727d64a0bbcbd7ced51e0cd2" + wall.name = "Walls - W30(Fc24)" + wall.type = "W30(Fc24)" + wall.units = "mm" + wall.family = "Basic Wall" + # ... more properties + + return wall +``` + +These fixtures create standardized test objects that represent different Speckle schema versions, allowing you to test +rule behavior consistently. + +### TDD Workflow for New Rules + +When developing a new rule or predicate, follow this TDD approach: + +1. **Add test fixtures**: First, expand `conftest.py` with representative objects that your rule will process + +2. **Write tests first**: Create test cases in a test file (e.g., `test_my_rule.py`): + ```python + def test_new_wall_rule(v2_wall, v3_wall): + """Test a new rule that checks wall thickness requirements""" + # Test with v2 schema + assert PropertyRules.is_new_wall_check(v2_wall, "width", "300") + + # Test with v3 schema + assert PropertyRules.is_new_wall_check(v3_wall, "Width", "300") + + # Test failure case + v2_wall.parameters["WALL_ATTR_WIDTH_PARAM"].value = 200 + assert not PropertyRules.is_new_wall_check(v2_wall, "width", "300") + ``` + +3. **Implement the rule**: Add the new rule method to the `PropertyRules` class in `rules.py`: + ```python + @staticmethod + def is_new_wall_check(speckle_object: Base, parameter_name: str, expected_value: str) -> bool: + """Checks if a wall meets specific thickness requirements""" + parameter_value = PropertyRules.get_parameter_value(speckle_object, parameter_name) + # Implement rule logic + return result + ``` + +4. **Add to predicate mapping**: Register your new rule in `predicates.py`: + ```python + PREDICATE_METHOD_MAP = { + # Existing predicates... + "new_wall_check": PropertyRules.is_new_wall_check.__name__, + } + ``` + +5. **Run tests to verify**: + ```bash + pytest test_my_rule.py -v + ``` + +### Creating Comprehensive Test Objects + +For the most effective testing, your test objects in `conftest.py` should: + +1. **Include diverse objects**: Walls, columns, beams, etc. +2. **Cover edge cases**: Null values, missing properties, special characters +3. **Represent both schemas**: Include both v2 and v3 format objects +4. **Include real-world examples**: Extract sample objects from actual projects + +You can extract real objects for testing using: + +```python +# Example code to extract and save real objects for test fixtures +from specklepy.api import operations +from specklepy.api.client import SpeckleClient +from specklepy.transports.server import ServerTransport + +client = SpeckleClient(host="app.speckle.systems") +client.authenticate_with_token(token) + +transport = ServerTransport(client=client, stream_id="stream_id") +obj = operations.receive("object_id", transport) + +# Print structure to help with fixture creation +print(obj.get_member_names()) +print(obj.get_dynamic_member_names()) +``` + +By following this TDD approach and maintaining comprehensive test fixtures, you can develop robust rules that work +reliably across different object schemas and handle edge cases appropriately. ## Testing +### Running Tests + ```bash -poetry run pytest +# Run all tests +pytest + +# Run specific test file +pytest tests/test_parameters.py + +# Run with coverage +pytest --cov=src ``` -## Extending Rules +### Test Data -1. Add new predicate to `input_predicate_mapping` in `rules.py` -2. Create corresponding method in `PropertyRules` class -3. Update tests +Test fixtures in `conftest.py` provide sample objects: -## Building +- `v2_wall`: Wall object in v2 schema +- `v3_wall`: Wall object in v3 schema -The function is packaged as a Docker container: +### Manual Testing with Real Data -```bash -docker build -f ./Dockerfile -t checker . -``` +For testing with real Speckle data: -## Local Testing +```python +from specklepy.api import operations +from specklepy.api.client import SpeckleClient +from specklepy.api.credentials import get_account_from_token +from specklepy.transports.server import ServerTransport -```bash -docker run --rm checker python -u main.py run [automation_data] [parameters] [token] +client = SpeckleClient(host="app.speckle.systems") +account = get_account_from_token(token, "app.speckle.systems") +client.authenticate_with_account(account) + +transport = ServerTransport(client=client, stream_id="your_stream_id") +commit = client.commit.get(stream_id="your_stream_id", commit_id="your_commit_id") +obj = operations.receive(commit.referencedObject, transport) ``` ## Deployment -Create a GitHub release to trigger deployment to Speckle Automate. \ No newline at end of file +The function is deployed through GitHub Actions: + +1. Create a GitHub release to trigger the build workflow +2. The workflow builds the necessary artifacts and pushes them to the Speckle Automate registry +3. The function becomes available in the Speckle Automate UI + +## Performance Considerations + +- **Large Object Trees**: When processing large models, use aggressive filtering with WHERE clauses +- **Rule Complexity**: Minimize the number of nested property lookups +- **Memory Usage**: Be aware of object reference handling and avoid deep copies + +## Troubleshooting + +### Common Issues + +1. **Rule not matching expected objects**: + - Check property paths for the specific object type + - Verify data types (strings vs. numbers) + - Enable debug logging + +2. **Slow performance**: + - Check for inefficient property lookups + - Add more specific WHERE filters to reduce object set + +3. **Docker build failures**: + - Check dependency compatibility + - Verify Python version requirements + +## Contributing + +1. Create a branch for your feature or fix +2. Add tests for new functionality +3. Update documentation +4. Submit a pull request +5. Ensure CI tests pass + +## Future Development + +Potential improvements: + +- Support for more complex rule logic (OR conditions) +- UI-based rule editor +- Result visualization tools +- Performance optimizations for large models +- Support for referencing other objects in rules \ No newline at end of file diff --git a/README.md b/README.md index e052b82..f76f1a4 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,121 @@ -# Public Function: Checker +# Speckle Checker -Validate Speckle objects against configurable rules using spreadsheet definitions. +Speckle Checker is an Automate function that validates Speckle objects against configurable rules defined in a +spreadsheet. This approach provides a flexible way to implement quality checks without coding, making it accessible to +all team members. -## Usage +## Overview -1. Access the template Google Sheet [link needed] -2. Make a copy to your Google Drive using File > Make a copy -3. Define your rules in your sheet -4. Click "Speckle" menu > "Publish Rules" to get your TSV URL -5. Create an Automation in Speckle Automate using the Checker function -6. Paste your TSV URL into the function configuration -7. Run your automation +The Checker function allows you to: -## Rule Types +- Define validation rules in a spreadsheet +- Configure severity levels for issues +- Check properties across different types of objects +- Generate reports of validation results +- Apply consistent standards across projects -- Property existence -- Value matching -- Numeric comparisons -- Range checks -- List membership -- Pattern matching -- Boolean checks +## Getting Started -## Severity Levels +### 1. Prepare Your Rule Spreadsheet -- WARNING: Issues that should be reviewed -- ERROR: Critical issues requiring attention +1. Access the [template spreadsheet](https://docs.google.com/spreadsheets/d/1hiPSw23eOaqd27QD_YsXvZg9PWm7_XBx/edit) ( + make a copy to your drive) +2. Define your rules using the format explained below +3. Publish your rules by clicking "File > Download > Tab-separated values (.tsv)" +4. Upload the TSV file to a hosting service (Google Drive, Dropbox, etc.) and get a public URL + +### 2. Create an Automation + +1. Go to [Speckle Automate](https://automate.speckle.dev/) +2. Create a new Automation +3. Select the Checker function +4. Configure the function: + - Paste your TSV URL + - Set minimum severity level to report + - Configure other options as needed +5. Save and run your automation + +## Rule Definition Format + +Rules are defined in a spreadsheet with the following columns: + +| Rule Number | Logic | Property Name | Predicate | Value | Message | Report Severity | +|-------------|-------|---------------|--------------|-----------|----------------------|-----------------| +| 1 | WHERE | category | matches | Walls | Wall thickness check | ERROR | +| 1 | AND | Width | greater than | 200 | | | +| 2 | WHERE | category | matches | Columns | Column height check | WARNING | +| 2 | AND | height | in range | 2500,4000 | | | + +### Column Explanation + +- **Rule Number**: Groups conditions that belong to the same rule +- **Logic**: Defines how conditions are combined (WHERE, AND, CHECK) +- **Property Name**: The object property or parameter to check +- **Predicate**: Comparison operation (equals, greater than, etc.) +- **Value**: Reference value for comparison +- **Message**: Description shown in validation results +- **Report Severity**: ERROR, WARNING, or INFO + +### Supported Predicates + +| Predicate | Description | Example | +|------------------|-----------------------------|------------------------------------| +| exists | Checks if a property exists | `height` exists | +| equal to | Exact value match | `width` equal to `300` | +| not equal to | Value doesn't match | `material` not equal to `Concrete` | +| greater than | Value exceeds threshold | `height` greater than `3000` | +| less than | Value below threshold | `thickness` less than `50` | +| in range | Value within bounds | `elevation` in range `0,10000` | +| in list | Value in allowed set | `type` in list `W1,W2,W3` | +| contains | Property contains substring | `name` contains `Beam` | +| does not contain | Property doesn't contain | `name` does not contain `temp` | +| is true | Boolean property is true | `is_structural` is true | +| is false | Boolean property is false | `is_placeholder` is false | +| is like | Pattern matching | `name` is like `^BR\d+$` | + +## Rule Logic + +- **WHERE**: Filters objects to check (like SELECT WHERE in SQL) +- **AND**: Additional filter conditions +- **CHECK**: Final check condition (optional, defaults to last AND) + +Objects pass a rule when they match all conditions. Objects that match WHERE/AND filters but fail the CHECK condition +are reported as issues. + +## Working with Object Properties + +The Checker understands properties in Speckle objects regardless of schema: + +- Direct properties: `category`, `name`, `id` +- Nested properties: `parameters.WIDTH.value` +- Revit parameters: Use parameter names like `Mark`, `Width`, `Assembly Code` ## Example Rules -[Screenshot or example table to be added] +### Wall Thickness Check + +``` +Rule 1: WHERE category equals "Walls" AND width less than "200" +Message: "Wall too thin - minimum thickness is 200mm" +Severity: ERROR +``` + +### Door Naming Convention + +``` +Rule 2: WHERE category equals "Doors" AND name is not like "^D\d{3}$" +Message: "Door name must follow pattern D followed by 3 digits" +Severity: WARNING +``` + +### Structural Column Height Range + +``` +Rule 3: WHERE category equals "Columns" AND is_structural is true AND height not in range "2400,4000" +Message: "Structural column height outside acceptable range (2400-4000mm)" +Severity: ERROR +``` ## Support -For issues or questions, please open a GitHub issue. \ No newline at end of file +For issues or questions, please open a GitHub issue or contact your Speckle support representative. \ No newline at end of file