Release 2.0.0: TestBench 4.1 compatibility - #20
Merged
Merged
Conversation
Reworked result write-back (protocol.json merge, verdict aggregation up the test structure tree), structured and redesigned execution comments, blocked-element filtering, Robot Framework 6.1 floor, named-group subdivision patterns, and a restructured documentation set. See the 2.0.0 release notes for the full backwards-incompatible changes and feature list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UuyiqwrSFxgE2ESAcXArQu
There was a problem hiding this comment.
Pull request overview
This PR is the 2.0.0 release work to make testbench2robotframework compatible with TestBench 4.1, including major result write-back changes (protocol merge + verdict aggregation), redesigned execution comments, blocked-element filtering, and an updated docs/build/release setup.
Changes:
- Update compatibility floor/guards: TestBench 4.1 manifest version handling and Robot Framework >= 6.1.
- Rework result write-back: merge
protocol.json, recompute/propagate verdicts up the tree, and render structured HTML execution comments. - Improve generation ergonomics: blocked filtering, selective cleaning modes, metadata interpolation, and extensive documentation + release automation updates.
Reviewed changes
Copilot reviewed 34 out of 48 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| testbench2robotframework/utils.py | Version check updated for 4.1; add safe metadata interpolation helpers and typing refinements. |
| testbench2robotframework/testsuite_write.py | Add clean-mode behavior and selective cleanup of only generated .robot files. |
| testbench2robotframework/testbench2robotframework.py | Add blocked-element filtering to suite generation and path handling cleanup. |
| testbench2robotframework/testbench2rf.py | Regex/name-group handling improvements and suite metadata interpolation integration. |
| testbench2robotframework/robotframework2testbench.py | Accept Path inputs and improve CLI-facing error messages. |
| testbench2robotframework/result_writer.py | Major result write-back refactor: protocol merge, verdict aggregation, structured comments, timestamp handling changes. |
| testbench2robotframework/protocol_merge.py | New module for merging executions and consistent verdict aggregation/state mapping. |
| testbench2robotframework/model.py | Regenerated model for TestBench 4.1; add time field and bump spec version. |
| testbench2robotframework/model_utils.py | Harden dataclass conversion for non-type inputs. |
| testbench2robotframework/keyword_comment.py | New structured keyword execution comment renderer (single flat table). |
| testbench2robotframework/json_writer.py | Ensure tree serialization writes root first; messaging tweaks. |
| testbench2robotframework/json_reader.py | Add protocol.json loading and improve JSON error reporting. |
| testbench2robotframework/execution_comment.py | New structured rendering for test case / test case set execution comments. |
| testbench2robotframework/execution_artifacts.py | Typing improvements and clearer/confident error messages. |
| testbench2robotframework/config.py | Add new config options: clean mode, merge protocol, blocked filtering, keyword comment controls. |
| testbench2robotframework/cli.py | CLI option updates: --clean/--no-clean, --include-blocked, --no-merge-protocol, improved help text. |
| testbench2robotframework/blocked_filter.py | New module to filter blocked test elements during generation. |
| testbench2robotframework/init.py | Bump package version to 2.0.0. |
| requirements.txt | Add explicit click pin. |
| pyproject.toml | Set Robot Framework dependency floor to 6.1. |
| pyinstaller/run.py | New PyInstaller entry point for standalone binary packaging. |
| ExampleConfiguration/toml_config.toml | Update examples + document subdivision regex group rules; update library mapping example. |
| ExampleConfiguration/pyproject_example.toml | Same as above for pyproject-style example. |
| ExampleConfiguration/json_config.json | Update JSON example library mapping. |
| docs/usage/generate_tests.md | Expanded generation docs, blocked filtering, keyword mapping conventions, metadata placeholders. |
| docs/usage/fetch_results.md | Expanded write-back docs, protocol merge behavior, structured comments and controls. |
| docs/intro.md | Add release notes pointer + update requirements (RF 6.1, TestBench 4.1). |
| docs/getting_started/quick_start.md | Update CLI usage for clean flags; add complete round trip example. |
| docs/getting_started/installation.md | Update installation requirements and version output notes. |
| docs/configuration/pyproject_config.md | Restructure configuration documentation, lookup/merge behavior, pattern rules, JSON config. |
| docs/configuration/overview.md | Comprehensive configuration reference overhaul including precedence and new options. |
| docs/configuration/cli_options.md | Rewrite CLI usage docs and examples; explain config precedence and shapes. |
| DEVELOPMENT.md | Update release process to flit-based flow and reformat instructions. |
| .github/workflows/trigger-docs-release.yml | New workflow to trigger ecosystem docs versioning on releases. |
| .github/workflows/release-binaries.yml | New workflow to build wheel + platform binaries and upload to GitHub release. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
39
to
41
| def from_dict(cls: type[T], data: dict) -> T: | ||
| if not is_dataclass(cls): | ||
| if not isinstance(cls, type) or not is_dataclass(cls): | ||
| raise ValueError(ERROR_NOT_A_DATACLASS.format(dataclass=cls.__name__)) |
Comment on lines
+73
to
+76
| try: | ||
| head = robot_file.open(encoding="utf-8", errors="ignore").read(4096) | ||
| except OSError: | ||
| return False |
Comment on lines
+339
to
+343
| if itb_test_case.exec.key in ["", "-1"]: | ||
| logger.warning( | ||
| f"Test case {itb_test_case.uniqueID} was not exported with execution data " | ||
| f"and can therefore not be imported." | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reworked result write-back (protocol.json merge, verdict aggregation up the test structure tree), structured and redesigned execution comments, blocked-element filtering, Robot Framework 6.1 floor, named-group subdivision patterns, and a restructured documentation set.
See the 2.0.0 release notes for the full backwards-incompatible changes and feature list.