Compare commits

..

5 Commits

Author SHA1 Message Date
Jonathon Broughton 089ecca34e Update README with clearer instructions
- Updated the template spreadsheet link.
- Changed steps for rule publishing and automation creation.
- Improved formatting of supported predicates table.
- Added a new support contact method via community forum.
2025-03-01 10:20:41 +00:00
Jonathon Broughton 1c09449f19 Add 'is set' predicate and not empty check
- Introduced a new predicate for checking if a parameter is set.
- Added a method to verify if a parameter exists and isn't empty.
- Updated tests to cover the new 'not empty' functionality.
2025-02-28 17:36:40 +00:00
Jonathon Broughton 0e81ac07f0 Merge remote-tracking branch 'origin/main' 2025-02-28 17:35:43 +00:00
Jonathon Broughton 7d3afe4f9f Merge remote-tracking branch 'origin/main' 2025-02-28 15:00:09 +00:00
Jonathon Broughton 997b1049aa Added over the top levels of documentation for future developers 2025-02-28 14:54:46 +00:00
9 changed files with 94 additions and 65 deletions
-1
View File
@@ -44,4 +44,3 @@ jobs:
speckle_function_id: ${{ secrets.SPECKLE_FUNCTION_ID }}
speckle_function_input_schema_file_path: ${{ env.FUNCTION_SCHEMA_FILE_NAME }}
speckle_function_command: 'python -u main.py run'
speckle_function_recommended_memory_mi: 5000
-3
View File
@@ -4,7 +4,4 @@
<option name="sdkName" value="WSL Checker" />
</component>
<component name="ProjectRootManager" version="2" project-jdk-name="uv (Checker)" project-jdk-type="Python SDK" />
<component name="PythonCompatibilityInspectionAdvertiser">
<option name="version" value="3" />
</component>
</project>
+30 -39
View File
@@ -1,13 +1,14 @@
# Model Checker
# Speckle Checker
Model Checker is an Automate function that validates Speckle objects against configurable rules. This approach provides
a flexible way to implement quality checks and maintain consistent standards across projects.
Speckle Checker is an Automate function that validates Speckle objects against configurable rules defined in a
spreadsheet. This approach provides a flexible way to implement quality checks without coding, making it accessible to
all team members.
## Overview
The Model Checker allows you to:
The Checker function allows you to:
- Define validation rules for your objects
- Define validation rules in a spreadsheet
- Configure severity levels for issues
- Check properties across different types of objects
- Generate reports of validation results
@@ -15,35 +16,39 @@ The Model Checker allows you to:
## Getting Started
### 1. Access the Model Checker Application
### 1. Prepare Your Rule Spreadsheet
1. Go to the [Model Checker Application](https://model-checker.speckle.systems)
2. Sign in with your Speckle account
3. Create and manage your validation rules through the intuitive web interface
1. Access
the [template spreadsheet](https://docs.google.com/spreadsheets/d/1eB0RVuOXdjLyn4_GAPSahV05p1lqfSGQbH8WWijnkkA/edit?gid=0#gid=0)
2. Use the Speckle menu to launch the Speckle sidebar and make a copy.
3. Define your rules using the format explained below
4. Publish your rules by clicking "Publish Rules". Copy the resultant URL.
### 2. Create an Automation
1. Go to your workspace project in [Speckle](https://app.speckle.systems/)
2. Create a new Automation
3. Select the Model Checker function
3. Select the Checker function
4. Configure the function:
- Paste your published rules URL
- Set minimum severity level to report
- Configure other options as needed
5. Save and run your automation
## Rule Definition Format
Rules are defined with the following components:
Rules are defined in a spreadsheet with the following columns:
| Logic | Property Name | Predicate | Value | Message | Report Severity |
|-------|---------------|--------------|-----------|----------------------|-----------------|
| WHERE | category | matches | Walls | Wall thickness check | ERROR |
| CHECK | Width | greater than | 200 | | |
| WHERE | category | matches | Columns | Column height check | WARNING |
| AND | height | in range | 2500,4000 | | |
| Rule Number | Logic | Property Name | Predicate | Value | Message | Report Severity |
|-------------|-------|---------------|--------------|-----------|----------------------|-----------------|
| 1 | WHERE | category | matches | Walls | Wall thickness check | ERROR |
| 1 | AND | Width | greater than | 200 | | |
| 2 | WHERE | category | matches | Columns | Column height check | WARNING |
| 2 | AND | height | in range | 2500,4000 | | |
### Component Explanation
### Column Explanation
- **Rule Number**: Groups conditions that belong to the same rule
- **Logic**: Defines how conditions are combined (WHERE, AND, CHECK)
- **Property Name**: The object property or parameter to check
- **Predicate**: Comparison operation (equals, greater than, etc.)
@@ -79,7 +84,7 @@ are reported as issues.
## Working with Object Properties
The Model Checker understands properties in Speckle objects regardless of schema:
The Checker understands properties in Speckle objects regardless of schema:
- Direct properties: `category`, `name`, `id`
- Nested properties: `parameters.WIDTH.value`
@@ -90,41 +95,27 @@ The Model Checker understands properties in Speckle objects regardless of schema
### Wall Thickness Check
```
Rule: WHERE category equals "Walls" AND width less than "200"
Message: "Walls must have width of at least 200."
Rule 1: WHERE category equals "Walls" AND width less than "200"
Message: "Wall too thin - minimum thickness is 200mm"
Severity: ERROR
```
### Door Naming Convention
```
Rule: WHERE category equals "Doors" AND name is not like "^D\d{3}$"
Message: "All doors must have a name that follows the format "D" followed by three digits."
Rule 2: WHERE category equals "Doors" AND name is not like "^D\d{3}$"
Message: "Door name must follow pattern D followed by 3 digits"
Severity: WARNING
```
### Structural Column Height Range
```
Rule: WHERE category equals "Columns" AND is_structural is true AND height not in range "2400,4000"
Message: "Structural columns must have a height between 2400 and 4000."
Rule 3: WHERE category equals "Columns" AND is_structural is true AND height not in range "2400,4000"
Message: "Structural column height outside acceptable range (2400-4000mm)"
Severity: ERROR
```
## Support
For issues or questions, please let us know on the [Speckle Community Forum](https://speckle.community/).
### Alternative: TSV File Format
While the Model Checker Application is the recommended way to create and manage rules, you can also create compatible
TSV (Tab-Separated Values) files manually. This can be useful for:
- Programmatically generating rules
- Version controlling rules in a text format
- Integrating with existing workflows
- Creating rules in bulk
The TSV file should follow the same structure as shown in the table above, with columns separated by tabs. The file will
then need to be hosted somewhere and served with MIME-type of `text/tab-separated-values` and the URL used in the
automation configuration.
+1
View File
@@ -17,4 +17,5 @@ PREDICATE_METHOD_MAP = {
"identical to": PropertyRules.is_identical_value.__name__,
"contains": PropertyRules.is_parameter_value_containing.__name__,
"does not contain": PropertyRules.is_parameter_value_not_containing.__name__,
"is set": PropertyRules.is_parameter_value_not_empty.__name__,
}
+6 -12
View File
@@ -95,7 +95,7 @@ def evaluate_condition(
Returns:
True if the condition is met, False otherwise
"""
property_name = condition.get("Property Name", condition.get("Property Path"))
property_name = condition["Property Name"]
predicate_key = condition["Predicate"]
value = condition["Value"]
@@ -263,27 +263,21 @@ def apply_rules_to_objects(
rules_processed += 1
# Ensure rule_group has necessary columns
if "Message" not in rule_group.columns or (
"Report Severity" not in rule_group.columns and "Severity" not in rule_group.columns
):
if "Message" not in rule_group.columns or "Report Severity" not in rule_group.columns:
continue # Or raise an exception if these columns are mandatory
pass_objects, fail_objects = process_rule(speckle_objects, rule_group)
# Get the severity level for this rule
rule_severity = get_severity(rule_group.iloc[-1])
rule_severity_level = severity_levels[MinimumSeverity(rule_severity.value)]
# Check if the rule severity level meets the minimum severity level - no point in processing lower severity rules
if rule_severity_level < min_severity_level:
continue
pass_objects, fail_objects = process_rule(speckle_objects, rule_group)
# For passing objects, only attach if we're showing all levels (INFO)
if minimum_severity == MinimumSeverity.INFO:
attach_results(pass_objects, rule_group.iloc[-1], rule_id_str, automate_context, True)
# For failing objects, attach if they meet minimum severity threshold
if len(fail_objects) and rule_severity_level >= min_severity_level:
if rule_severity_level >= min_severity_level:
attach_results(fail_objects, rule_group.iloc[-1], rule_id_str, automate_context, False)
if len(pass_objects) == 0 and len(fail_objects) == 0 and not hide_skipped:
@@ -329,7 +323,7 @@ def get_severity(rule_info: pd.Series) -> SeverityLevel:
Returns:
Appropriate SeverityLevel enum value
"""
severity = rule_info.get("Report Severity") or rule_info.get("Severity") # Extract severity from input data
severity = rule_info.get("Report Severity") # Extract severity from input data
# If severity is None or not a string (e.g., numeric input), default to ERROR
if not isinstance(severity, str):
+32
View File
@@ -890,3 +890,35 @@ class PropertyRules:
return PropertyRules.compare_values(
parameter_value, value_to_match, case_sensitive=True, tolerance=0, allow_yes_no_bools=False, use_exact=True
)
@staticmethod
def is_parameter_value_not_empty(speckle_object: Base, parameter_name: str) -> bool:
"""Checks if parameter exists and has a non-empty value.
Args:
speckle_object: The Speckle object to check
parameter_name: Name of the parameter to check
Returns:
bool: True if parameter exists and has a non-empty value
"""
# Get the parameter value
parameter_value = PropertyRules.get_parameter_value(speckle_object, parameter_name)
# Check if parameter exists
if parameter_value is None:
return False
# Check if value is an empty string
if isinstance(parameter_value, str) and parameter_value.strip() == "":
return False
# Check if value is an empty list or collection
if hasattr(parameter_value, "__len__") and len(parameter_value) == 0:
return False
# Additional check for "None" string which can sometimes appear in exports
if isinstance(parameter_value, str) and parameter_value.lower() == "none":
return False
return True
+6 -6
View File
@@ -64,10 +64,8 @@ def process_rule_numbers(df: DataFrame) -> DataFrame:
# Get slice of rows for this group
group_slice = df.iloc[start_idx:end_idx]
# Try to get rule number from first row, fall back to "Rule #"
group_rule_num = (
group_slice["Rule Number"].iloc[0] if not pd.isna(group_slice["Rule Number"].iloc[0]) else "Rule #"
)
# Try to get rule number from first row
group_rule_num = group_slice["Rule Number"].iloc[0]
if pd.isna(group_rule_num):
# If no rule number, generate next available number
@@ -92,7 +90,8 @@ def process_rule_numbers(df: DataFrame) -> DataFrame:
def validate_rule_numbers(df: DataFrame) -> list[str]:
"""Validate rule numbers and return any warnings or errors.
""" "
Validate rule numbers and return any warnings or errors.
This checks for issues like:
1. Missing rule numbers
@@ -129,7 +128,8 @@ def validate_rule_numbers(df: DataFrame) -> list[str]:
def read_rules_from_spreadsheet(url: str) -> tuple[DataFrameGroupBy, list[str]] | tuple[None, list[str]]:
"""Reads rules from a TSV file at the provided URL, processes them, and returns grouped rules.
""" "
Reads rules from a TSV file at the provided URL, processes them, and returns grouped rules.
This function is the main entry point for rule loading:
1. Reads the TSV file from the provided URL
+2 -4
View File
@@ -30,14 +30,12 @@ class TestFunction:
"""Run an integration test for the automate function."""
automation_context = AutomationContext.initialize(test_automation_run_data, test_automation_token)
default_url: str = (
"https://speckle-model-checker-cedxvz7lzq-ew.a.run.app/r/6hdycwPELyTIT7Ueedh0UsWdJlTBefwSjDlcnd8LXGg/tsv"
)
default_url: str = "https://drive.google.com/uc?export=download&id=1hiPSw23eOaqd27QD_YsXvZg9PWm7_XBx"
automate_sdk = run_function(
automation_context,
automate_function,
FunctionInputs(spreadsheet_url=default_url, minimum_severity=MinimumSeverity.INFO, hide_skipped=True),
FunctionInputs(spreadsheet_url=default_url, minimum_severity=MinimumSeverity.WARNING, hide_skipped=True),
)
assert automate_sdk.run_status == AutomationStatus.SUCCEEDED
+17
View File
@@ -415,3 +415,20 @@ class TestParameterHandling:
"""Test negative substring matching on parameter values."""
v2_obj, _ = test_objects
assert PropertyRules.is_parameter_value_not_containing(v2_obj, param_name, substring) == expected_result
@pytest.mark.parametrize(
"param_name, expected_result",
[
("category", True), # Parameter exists with non-empty value
("family", True), # Parameter exists with non-empty value
("non_existent_param", False), # Parameter doesn't exist
# The following would require setup with empty values
# ("empty_string_param", False), # Parameter exists but has empty string value
# ("none_string_param", False), # Parameter exists but has "None" string value
],
)
def test_parameter_not_empty(self, test_objects, param_name, expected_result):
"""Test 'not empty' check on parameter values."""
v2_obj, _ = test_objects
assert PropertyRules.is_parameter_value_not_empty(v2_obj, param_name) == expected_result