Updates Model Checker documentation

Updates the documentation to reflect the shift from a spreadsheet-based rule definition to the new Model Checker Application.

Explains how to access the application, create rules, and configure automations.

Also introduces the alternative TSV file format for programmatically generating rules or version controlling rules.

.github/workflows/main.yml

@@ -44,3 +44,4 @@ jobs:
           speckle_function_id: ${{ secrets.SPECKLE_FUNCTION_ID }}
           speckle_function_input_schema_file_path: ${{ env.FUNCTION_SCHEMA_FILE_NAME }}
           speckle_function_command: 'python -u main.py run'
+          speckle_function_recommended_memory_mi: 5000

.idea/misc.xml

@@ -4,4 +4,7 @@
     <option name="sdkName" value="WSL Checker" />
   </component>
   <component name="ProjectRootManager" version="2" project-jdk-name="uv (Checker)" project-jdk-type="Python SDK" />
+  <component name="PythonCompatibilityInspectionAdvertiser">
+    <option name="version" value="3" />
+  </component>
 </project>

DEVELOPER_README.md

@@ -1,56 +1,377 @@
-# Checker Function Development Guide
+# Checker Function Developer Guide
 
-## Setup
+This document provides technical details for developers working on the Speckle Checker Automate function.
 
-1. Install dependencies:
+## Project Overview
+
+The Checker function enables validation of Speckle objects against user-defined rules in a spreadsheet. It's designed to
+be flexible, supporting various object schemas including both v2 and v3 Speckle APIs.
+
+## Setup Development Environment
+
+### Prerequisites
+
+- Python 3.10+
+- Poetry for dependency management
+
+### Installation
+
+1. Clone the repository
+2. Install dependencies:
+   ```bash
+   poetry install
+   ```
+3. Activate the virtual environment:
+   ```bash
+   poetry shell
+   ```
+
+### Test Automation Environment
+
+The project uses Speckle's [Test Automation feature](https://speckle.guide/automate/function-testing.html) to run
+integration tests against real Speckle data. This provides a sandboxed environment to validate the function's business
+logic without triggering actual automations.
+
+#### Setting Up a Test Automation
+
+1. Navigate to your Speckle project
+2. Go to the **Automations** tab
+3. Click **New Automation**
+4. Select **Create Test Automation** in the bottom left
+5. Follow the configuration steps
+
+Note: To create a test automation, you must:
+
+- Be an owner of the Speckle project
+- Have published this function to the Function Library
+- Have at least one release for the function
+
+#### Environment Configuration
+
+For local integration testing, create a `.env` file in the project root with these variables:
+
+```
+# Your Personal Access Token from Speckle
+SPECKLE_TOKEN=your_speckle_token
+
+# The Speckle server URL
+SPECKLE_SERVER_URL=https://app.speckle.systems
+
+# From the test automation URL: /projects/[project-id]/automations/[automation-id]
+SPECKLE_PROJECT_ID=your_project_id
+SPECKLE_AUTOMATION_ID=your_automation_id
+```
+
+This configuration allows the test suite to:
+
+1. Connect to your test automation via the Speckle API
+2. Run the function locally against real Speckle data
+3. Submit results to the test automation for validation
+
+For detailed instructions, refer to
+the [official documentation on function testing](https://speckle.guide/automate/function-testing.html#how-to-create-a-test-automation).
+
+#### Running Integration Tests
+
+With the `.env` file configured:
 
 ```bash
-poetry shell && poetry install
+# Run the integration tests
+pytest test_function.py
 ```
 
-2. Configure `.env`:
+The SDK utilities will automatically:
 
-```
-SPECKLE_TOKEN=your_speckle_token
-SPECKLE_SERVER_URL=app.speckle.systems
+- Connect to your test automation
+- Execute your function with the specified test data
+- Submit results back to Speckle
+
+Test results will be visible on the automation page in the Speckle UI.
+
+#### Unit Tests
+
+For unit tests that don't require a full Speckle connection, you can run:
+
+```bash
+# Run unit tests only
+pytest test_comparisons.py test_rule_processing.py
 ```
 
-Get test automation details from app.speckle.systems
+Note: The `.env` file should never be committed to version control (it's included in .gitignore)
 
 ## Project Structure
 
-- `function.py`: Main business logic
-- `rules.py`: Rule definitions and processing
-- `inputs.py`: Function input schema
-- `helpers.py`: Utility functions
-- `spreadsheet.py`: TSV handling
+```
+├── main.py                  # Entry point for Automate
+├── src/
+│   ├── function.py          # Main function logic
+│   ├── inputs.py            # Function input schema
+│   ├── helpers.py           # Utility functions
+│   ├── filters.py           # Object filtering functions
+│   ├── rules.py             # Rule definitions and property handling
+│   ├── predicates.py        # Predicate mapping for spreadsheet values
+│   ├── rule_processor.py    # Rule application and result handling
+│   └── spreadsheet.py       # TSV file parsing
+├── tests/
+│   ├── conftest.py          # Test fixtures
+│   ├── test_function.py     # Main function tests
+│   ├── test_comparisons.py  # Value comparison tests
+│   ├── test_parameters.py   # Parameter handling tests
+│   └── test_rule_processing.py  # Rule processing tests
+├── pyproject.toml           # Project dependencies
+└── poetry.lock              # Locked dependencies
+```
+
+## Core Components
+
+### 1. Function Execution Flow
+
+The main execution flow is defined in `function.py`:
+
+1. `automate_function()` receives context and inputs from Automate
+2. Retrieves Speckle objects via `automate_context.receive_version()`
+3. Flattens the object tree using `flatten_base()`
+4. Loads rules from the spreadsheet URL via `read_rules_from_spreadsheet()`
+5. Applies rules to objects using `apply_rules_to_objects()`
+6. Reports results via the Automate context
+
+### 2. Rule Processing
+
+Rules are processed through several stages:
+
+1. **Spreadsheet Parsing** (`spreadsheet.py`):
+    - Reads TSV data
+    - Groups rules by rule number
+    - Validates rule structure
+
+2. **Rule Application** (`rule_processor.py`):
+    - Processes rule logic (WHERE, AND, CHECK)
+    - Evaluates conditions against objects
+    - Attaches results to objects in Automate context
+
+3. **Property Rules** (`rules.py`):
+    - Handles property lookups in objects
+    - Implements comparison logic
+    - Supports both v2 and v3 Speckle schemas
+
+### 3. Property Access System
+
+The system uses a flexible property access mechanism that works with different Speckle schemas:
+
+- **V2 Schema**: Properties in `parameters` dictionary with internal definition names
+- **V3 Schema**: Properties in nested `properties.Parameters` structure
+
+The `PropertyRules` class provides methods to:
+
+- Find properties by path or name
+- Extract values with appropriate type conversion
+- Perform comparisons with tolerance and type handling
+
+## Test-Driven Development for Rules
+
+The test infrastructure is designed to support Test-Driven Development (TDD) when creating new rules or extending
+functionality. This approach is especially powerful for rule development as it allows you to verify behavior against
+known test objects.
+
+### Using Test Fixtures for Rule Development
+
+The `conftest.py` file contains test fixtures that provide sample Speckle objects for testing:
+
+```python
+@pytest.fixture
+def v2_wall():
+    """Creates a v2-style Speckle wall object"""
+    wall = Base()
+    wall.id = "cdb18060dc48281909e94f0f1d8d3cc0"
+    wall.type = "W30(Fc24)"
+    wall.units = "mm"
+    wall.family = "Basic Wall"
+    wall.height = 1400
+    wall.flipped = False
+    wall.category = "Walls"
+    # ... more properties
+
+    return wall
+
+
+@pytest.fixture
+def v3_wall():
+    """Creates a v3-style Speckle wall object"""
+    wall = Base()
+    wall.id = "46f06fef727d64a0bbcbd7ced51e0cd2"
+    wall.name = "Walls - W30(Fc24)"
+    wall.type = "W30(Fc24)"
+    wall.units = "mm"
+    wall.family = "Basic Wall"
+    # ... more properties
+
+    return wall
+```
+
+These fixtures create standardized test objects that represent different Speckle schema versions, allowing you to test
+rule behavior consistently.
+
+### TDD Workflow for New Rules
+
+When developing a new rule or predicate, follow this TDD approach:
+
+1. **Add test fixtures**: First, expand `conftest.py` with representative objects that your rule will process
+
+2. **Write tests first**: Create test cases in a test file (e.g., `test_my_rule.py`):
+   ```python
+   def test_new_wall_rule(v2_wall, v3_wall):
+       """Test a new rule that checks wall thickness requirements"""
+       # Test with v2 schema
+       assert PropertyRules.is_new_wall_check(v2_wall, "width", "300")
+       
+       # Test with v3 schema
+       assert PropertyRules.is_new_wall_check(v3_wall, "Width", "300")
+       
+       # Test failure case
+       v2_wall.parameters["WALL_ATTR_WIDTH_PARAM"].value = 200
+       assert not PropertyRules.is_new_wall_check(v2_wall, "width", "300")
+   ```
+
+3. **Implement the rule**: Add the new rule method to the `PropertyRules` class in `rules.py`:
+   ```python
+   @staticmethod
+   def is_new_wall_check(speckle_object: Base, parameter_name: str, expected_value: str) -> bool:
+       """Checks if a wall meets specific thickness requirements"""
+       parameter_value = PropertyRules.get_parameter_value(speckle_object, parameter_name)
+       # Implement rule logic
+       return result
+   ```
+
+4. **Add to predicate mapping**: Register your new rule in `predicates.py`:
+   ```python
+   PREDICATE_METHOD_MAP = {
+       # Existing predicates...
+       "new_wall_check": PropertyRules.is_new_wall_check.__name__,
+   }
+   ```
+
+5. **Run tests to verify**:
+   ```bash
+   pytest test_my_rule.py -v
+   ```
+
+### Creating Comprehensive Test Objects
+
+For the most effective testing, your test objects in `conftest.py` should:
+
+1. **Include diverse objects**: Walls, columns, beams, etc.
+2. **Cover edge cases**: Null values, missing properties, special characters
+3. **Represent both schemas**: Include both v2 and v3 format objects
+4. **Include real-world examples**: Extract sample objects from actual projects
+
+You can extract real objects for testing using:
+
+```python
+# Example code to extract and save real objects for test fixtures
+from specklepy.api import operations
+from specklepy.api.client import SpeckleClient
+from specklepy.transports.server import ServerTransport
+
+client = SpeckleClient(host="app.speckle.systems")
+client.authenticate_with_token(token)
+
+transport = ServerTransport(client=client, stream_id="stream_id")
+obj = operations.receive("object_id", transport)
+
+# Print structure to help with fixture creation
+print(obj.get_member_names())
+print(obj.get_dynamic_member_names())
+```
+
+By following this TDD approach and maintaining comprehensive test fixtures, you can develop robust rules that work
+reliably across different object schemas and handle edge cases appropriately.
 
 ## Testing
 
+### Running Tests
+
 ```bash
-poetry run pytest
+# Run all tests
+pytest
+
+# Run specific test file
+pytest tests/test_parameters.py
+
+# Run with coverage
+pytest --cov=src
 ```
 
-## Extending Rules
+### Test Data
 
-1. Add new predicate to `input_predicate_mapping` in `rules.py`
-2. Create corresponding method in `PropertyRules` class
-3. Update tests
+Test fixtures in `conftest.py` provide sample objects:
 
-## Building
+- `v2_wall`: Wall object in v2 schema
+- `v3_wall`: Wall object in v3 schema
 
-The function is packaged as a Docker container:
+### Manual Testing with Real Data
 
-```bash
-docker build -f ./Dockerfile -t checker .
-```
+For testing with real Speckle data:
 
-## Local Testing
+```python
+from specklepy.api import operations
+from specklepy.api.client import SpeckleClient
+from specklepy.api.credentials import get_account_from_token
+from specklepy.transports.server import ServerTransport
 
-```bash
-docker run --rm checker python -u main.py run [automation_data] [parameters] [token]
+client = SpeckleClient(host="app.speckle.systems")
+account = get_account_from_token(token, "app.speckle.systems")
+client.authenticate_with_account(account)
+
+transport = ServerTransport(client=client, stream_id="your_stream_id")
+commit = client.commit.get(stream_id="your_stream_id", commit_id="your_commit_id")
+obj = operations.receive(commit.referencedObject, transport)
 ```
 
 ## Deployment
 
-Create a GitHub release to trigger deployment to Speckle Automate.
+The function is deployed through GitHub Actions:
+
+1. Create a GitHub release to trigger the build workflow
+2. The workflow builds the necessary artifacts and pushes them to the Speckle Automate registry
+3. The function becomes available in the Speckle Automate UI
+
+## Performance Considerations
+
+- **Large Object Trees**: When processing large models, use aggressive filtering with WHERE clauses
+- **Rule Complexity**: Minimize the number of nested property lookups
+- **Memory Usage**: Be aware of object reference handling and avoid deep copies
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Rule not matching expected objects**:
+    - Check property paths for the specific object type
+    - Verify data types (strings vs. numbers)
+    - Enable debug logging
+
+2. **Slow performance**:
+    - Check for inefficient property lookups
+    - Add more specific WHERE filters to reduce object set
+
+3. **Docker build failures**:
+    - Check dependency compatibility
+    - Verify Python version requirements
+
+## Contributing
+
+1. Create a branch for your feature or fix
+2. Add tests for new functionality
+3. Update documentation
+4. Submit a pull request
+5. Ensure CI tests pass
+
+## Future Development
+
+Potential improvements:
+
+- Support for more complex rule logic (OR conditions)
+- UI-based rule editor
+- Result visualization tools
+- Performance optimizations for large models
+- Support for referencing other objects in rules

README.md

@@ -1,36 +1,130 @@
-# Public Function: Checker
+# Model Checker
 
-Validate Speckle objects against configurable rules using spreadsheet definitions.
+Model Checker is an Automate function that validates Speckle objects against configurable rules. This approach provides
+a flexible way to implement quality checks and maintain consistent standards across projects.
 
-## Usage
+## Overview
 
-1. Access the template Google Sheet [link needed]
-2. Make a copy to your Google Drive using File > Make a copy
-3. Define your rules in your sheet
-4. Click "Speckle" menu > "Publish Rules" to get your TSV URL
-5. Create an Automation in Speckle Automate using the Checker function
-6. Paste your TSV URL into the function configuration
-7. Run your automation
+The Model Checker allows you to:
 
-## Rule Types
+- Define validation rules for your objects
+- Configure severity levels for issues
+- Check properties across different types of objects
+- Generate reports of validation results
+- Apply consistent standards across projects
 
-- Property existence
-- Value matching
-- Numeric comparisons
-- Range checks
-- List membership
-- Pattern matching
-- Boolean checks
+## Getting Started
 
-## Severity Levels
+### 1. Access the Model Checker Application
 
-- WARNING: Issues that should be reviewed
-- ERROR: Critical issues requiring attention
+1. Go to the [Model Checker Application](https://model-checker.speckle.systems)
+2. Sign in with your Speckle account
+3. Create and manage your validation rules through the intuitive web interface
+
+### 2. Create an Automation
+
+1. Go to your workspace project in [Speckle](https://app.speckle.systems/)
+2. Create a new Automation
+3. Select the Model Checker function
+4. Configure the function:
+    - Set minimum severity level to report
+    - Configure other options as needed
+5. Save and run your automation
+
+## Rule Definition Format
+
+Rules are defined with the following components:
+
+| Logic | Property Name | Predicate    | Value     | Message              | Report Severity |
+|-------|---------------|--------------|-----------|----------------------|-----------------|
+| WHERE | category      | matches      | Walls     | Wall thickness check | ERROR           |
+| CHECK | Width         | greater than | 200       |                      |                 |
+| WHERE | category      | matches      | Columns   | Column height check  | WARNING         |
+| AND   | height        | in range     | 2500,4000 |                      |                 |
+
+### Component Explanation
+
+- **Logic**: Defines how conditions are combined (WHERE, AND, CHECK)
+- **Property Name**: The object property or parameter to check
+- **Predicate**: Comparison operation (equals, greater than, etc.)
+- **Value**: Reference value for comparison
+- **Message**: Description shown in validation results
+- **Report Severity**: ERROR, WARNING, or INFO
+
+### Supported Predicates
+
+| Predicate        | Description                 | Example                               |
+|------------------|-----------------------------|---------------------------------------|
+| exists           | Checks if a property exists | `height` exists                       |
+| equal to         | Exact value match           | `width` equal to `300`                |
+| not equal to     | Value doesn't match         | `material` not equal to `Concrete`    |
+| greater than     | Value exceeds threshold     | `height` greater than `3000`          |
+| less than        | Value below threshold       | `thickness` less than `50`            |
+| in range         | Value within bounds         | `elevation` in range `0,10000`        |
+| in list          | Value in allowed set        | `type` in list `W1,W2,W3`             |
+| contains         | Property contains substring | `name` contains `Beam`                |
+| does not contain | Property doesn't contain    | `name` does not contain `temp`        |
+| is true          | Boolean property is true    | `is_structural` is true               |
+| is false         | Boolean property is false   | `is_placeholder` is false             |
+| is like          | Loose text matching         | `name` is like `Wall` matches `Walls` |
+
+## Rule Logic
+
+- **WHERE**: Filters objects to check (like SELECT WHERE in SQL)
+- **AND**: Additional filter conditions
+- **CHECK**: Final check condition (optional, defaults to last AND)
+
+Objects pass a rule when they match all conditions. Objects that match WHERE/AND filters but fail the CHECK condition
+are reported as issues.
+
+## Working with Object Properties
+
+The Model Checker understands properties in Speckle objects regardless of schema:
+
+- Direct properties: `category`, `name`, `id`
+- Nested properties: `parameters.WIDTH.value`
+- Revit parameters: Use parameter names like `Mark`, `Width`, `Assembly Code`
 
 ## Example Rules
 
-[Screenshot or example table to be added]
+### Wall Thickness Check
+
+```
+Rule: WHERE category equals "Walls" AND width less than "200"
+Message: "Walls must have width of at least 200."
+Severity: ERROR
+```
+
+### Door Naming Convention
+
+```
+Rule: WHERE category equals "Doors" AND name is not like "^D\d{3}$"
+Message: "All doors must have a name that follows the format "D" followed by three digits."
+Severity: WARNING
+```
+
+### Structural Column Height Range
+
+```
+Rule: WHERE category equals "Columns" AND is_structural is true AND height not in range "2400,4000"
+Message: "Structural columns must have a height between 2400 and 4000."
+Severity: ERROR
+```
 
 ## Support
 
-For issues or questions, please open a GitHub issue.
+For issues or questions, please let us know on the [Speckle Community Forum](https://speckle.community/).
+
+### Alternative: TSV File Format
+
+While the Model Checker Application is the recommended way to create and manage rules, you can also create compatible
+TSV (Tab-Separated Values) files manually. This can be useful for:
+
+- Programmatically generating rules
+- Version controlling rules in a text format
+- Integrating with existing workflows
+- Creating rules in bulk
+
+The TSV file should follow the same structure as shown in the table above, with columns separated by tabs. The file will
+then need to be hosted somewhere and served with MIME-type of `text/tab-separated-values` and the URL used in the
+automation configuration.

src/rule_processor.py

@@ -95,7 +95,7 @@ def evaluate_condition(
     Returns:
         True if the condition is met, False otherwise
     """
-    property_name = condition["Property Name"]
+    property_name = condition.get("Property Name", condition.get("Property Path"))
     predicate_key = condition["Predicate"]
     value = condition["Value"]
 
@@ -263,21 +263,27 @@ def apply_rules_to_objects(
         rules_processed += 1
 
         # Ensure rule_group has necessary columns
-        if "Message" not in rule_group.columns or "Report Severity" not in rule_group.columns:
+        if "Message" not in rule_group.columns or (
+            "Report Severity" not in rule_group.columns and "Severity" not in rule_group.columns
+        ):
             continue  # Or raise an exception if these columns are mandatory
 
-        pass_objects, fail_objects = process_rule(speckle_objects, rule_group)
-
         # Get the severity level for this rule
         rule_severity = get_severity(rule_group.iloc[-1])
         rule_severity_level = severity_levels[MinimumSeverity(rule_severity.value)]
 
+        # Check if the rule severity level meets the minimum severity level - no point in processing lower severity rules
+        if rule_severity_level < min_severity_level:
+            continue
+
+        pass_objects, fail_objects = process_rule(speckle_objects, rule_group)
+
         # For passing objects, only attach if we're showing all levels (INFO)
         if minimum_severity == MinimumSeverity.INFO:
             attach_results(pass_objects, rule_group.iloc[-1], rule_id_str, automate_context, True)
 
         # For failing objects, attach if they meet minimum severity threshold
-        if rule_severity_level >= min_severity_level:
+        if len(fail_objects) and rule_severity_level >= min_severity_level:
             attach_results(fail_objects, rule_group.iloc[-1], rule_id_str, automate_context, False)
 
         if len(pass_objects) == 0 and len(fail_objects) == 0 and not hide_skipped:
@@ -323,7 +329,7 @@ def get_severity(rule_info: pd.Series) -> SeverityLevel:
     Returns:
         Appropriate SeverityLevel enum value
     """
-    severity = rule_info.get("Report Severity")  # Extract severity from input data
+    severity = rule_info.get("Report Severity") or rule_info.get("Severity")  # Extract severity from input data
 
     # If severity is None or not a string (e.g., numeric input), default to ERROR
     if not isinstance(severity, str):

src/spreadsheet.py

@@ -64,8 +64,10 @@ def process_rule_numbers(df: DataFrame) -> DataFrame:
         # Get slice of rows for this group
         group_slice = df.iloc[start_idx:end_idx]
 
-        # Try to get rule number from first row
-        group_rule_num = group_slice["Rule Number"].iloc[0]
+        # Try to get rule number from first row, fall back to "Rule #"
+        group_rule_num = (
+            group_slice["Rule Number"].iloc[0] if not pd.isna(group_slice["Rule Number"].iloc[0]) else "Rule #"
+        )
 
         if pd.isna(group_rule_num):
             # If no rule number, generate next available number
@@ -90,8 +92,7 @@ def process_rule_numbers(df: DataFrame) -> DataFrame:
 
 
 def validate_rule_numbers(df: DataFrame) -> list[str]:
-    """ "
-    Validate rule numbers and return any warnings or errors.
+    """Validate rule numbers and return any warnings or errors.
 
     This checks for issues like:
     1. Missing rule numbers
@@ -128,8 +129,7 @@ def validate_rule_numbers(df: DataFrame) -> list[str]:
 
 
 def read_rules_from_spreadsheet(url: str) -> tuple[DataFrameGroupBy, list[str]] | tuple[None, list[str]]:
-    """ "
-    Reads rules from a TSV file at the provided URL, processes them, and returns grouped rules.
+    """Reads rules from a TSV file at the provided URL, processes them, and returns grouped rules.
 
     This function is the main entry point for rule loading:
     1. Reads the TSV file from the provided URL

tests/test_function.py

@@ -30,12 +30,14 @@ class TestFunction:
 
         """Run an integration test for the automate function."""
         automation_context = AutomationContext.initialize(test_automation_run_data, test_automation_token)
-        default_url: str = "https://drive.google.com/uc?export=download&id=1hiPSw23eOaqd27QD_YsXvZg9PWm7_XBx"
+        default_url: str = (
+            "https://speckle-model-checker-cedxvz7lzq-ew.a.run.app/r/6hdycwPELyTIT7Ueedh0UsWdJlTBefwSjDlcnd8LXGg/tsv"
+        )
 
         automate_sdk = run_function(
             automation_context,
             automate_function,
-            FunctionInputs(spreadsheet_url=default_url, minimum_severity=MinimumSeverity.WARNING, hide_skipped=True),
+            FunctionInputs(spreadsheet_url=default_url, minimum_severity=MinimumSeverity.INFO, hide_skipped=True),
         )
 
         assert automate_sdk.run_status == AutomationStatus.SUCCEEDED