release: SkillSpector 2.12.0 - #550
mohgupta-ship-it wants to merge 39 commits into
Conversation
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
|
Codex on behalf of Mohit Gupta — independent qualification update, 16 September 2026. The freshly built 2.12.0 wheel from Fresh results after the AS3 update:
Both frozen corpora retain exactly the previous failure sets. Their behavioral expectations and fixture bytes were unchanged; only candidate identity/version provenance was refreshed. The consumer policy/limit assertions were retained. An initial provider-enabled consumer harness attempt was terminated and excluded; the reported counts come from the subsequent run with the established sanitized environment. Draft #563 separately fixes unsupported-primary and pure/mixed newline completeness gaps. At Current-head hosted CI/review and the remaining production/provider, Windows, published-artifact and adoption gates are still required. These results do not constitute release sign-off; no release or production pin was changed. |
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Reviewed current head e6bcf1044b781d9a28ae537cd8a56147edc9afa4, including the complete version/changelog/release-note diff, candidate history, existing review thread, version/lock consistency, component source identity, exact-head checks, and the stated release gates. The package and lock versions agree and all five hosted checks pass.
Changes are requested because two security guarantees in the release notes are not true of this exact candidate. Its letter-spacing implementation is byte-identical to the currently reviewed #470 implementation and still permits alternating-width spacing to evade P3/P4 and AE6. Its companion-classification analyzer blobs are identical to the unresolved #547 implementation, where four contextual fail-open paths remain. Either correct and requalify those implementations before publication or describe the precise limitations without claiming the risky cases remain covered. Deployment/provider, Windows, published-artifact, downstream-adoption, and independent release sign-off also remain merge gates.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head ff10467cd93832bc5a9a0ee8c07a2bcd3b91e01f after the automatic synchronization with main. This merge has the previously reviewed head e6bcf1044b781d9a28ae537cd8a56147edc9afa4 and current main e19ec01cc85b9891b2976a1157fce492504cfc13 as its exact parents. The only intervening tree changes are #558's three batch-scan files; none overlaps the four release files, whose reviewed contents are unchanged.
The two existing non-outdated release-note findings therefore remain current and unresolved. The candidate still claims irregular letter spacing fails closed although alternating-width short runs can evade P3/P4 and AE6, and it still makes an unconditional companion-context guarantee despite #547's unresolved PE3 and RA1 paths. Correct and requalify those implementations or accurately document the limitations. I have not duplicated the inline findings.
Exact-head CI run 35136927518 concluded action_required without starting jobs, leaving no check results. The security-release corrections, unresolved threads, active change request, missing exact-head checks, GitHub BLOCKED state, and previously identified release qualification/sign-off gates prevent merge.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
Re-reviewed exact head 2b39002 after reconciling the release catalog through main 4d52048. CHANGELOG.md, the 2.12.0 versioned notes, and the PR description now cover all 31 merged PRs since v2.11.2, including the 13 recent main merges. The letter-spacing and companion-context limitations remain explicit, and all review threads are resolved. Local lock/version checks, 10 release-helper/workflow tests, and git diff validation pass. All six exact-head CI jobs pass in run 35146009149. Approved; the deployment/provider, Windows, published-artifact, downstream-adoption, and remaining release-sign-off gates recorded in the PR still apply.
Brings the fork up to upstream main c13f70e; the version is still 2.11.2. The fork history was squashed into one commit on 2026-09-13, so it shares no merge-base with upstream and git merge refuses. The range diff was applied with git apply -3 instead. The fork content equals 69dcdfb plus the registered divergences, so conflicts landed only on those seven files; the other 116 applied cleanly. FORK.md now documents this procedure. Divergences, resolved by each row's rule: - static_runner.py takes upstream NVIDIA#522 (SKILLSPECTOR_MAX_STATIC_ANALYSIS_SECONDS_PER_ARTIFACT, default 300s). The fork's SKILLSPECTOR_MAX_STATIC_SECONDS override and its seven tests are removed. Downstream gates must use the upstream name when their pin moves. - test_static_yara.py, test_build_context.py and test_input_handler.py take upstream (NVIDIA#501-NVIDIA#505, NVIDIA#518 fix the same Windows issues); 301 passed on Windows, rows deleted. - test_security_end_to_end.py: upstream's version still fails nine_case on Windows (YARA load and SC8 budgets stay hard-coded), so the relaxation helper is re-applied on top; row kept and rewritten. - .gitignore keeps the fork block; README.md stays Traditional Chinese and the upstream README goes to README.en.md. Two new Windows divergences from new upstream tests: - tests/unit/test_cli.py: a file name containing a backslash is split into two path parts on Windows; skipped by a capability probe added to tests/platform_support.py. - test_json_container_ownership.py: oversized payloads became test ids, which pytest copies into PYTEST_CURRENT_TEST, over Windows' 32,767 character environment limit; short ids added, content unchanged. Triage: 13 of PRs NVIDIA#528-NVIDIA#580 merged into upstream main and arrive here; 29 stay open (including NVIDIA#550, release 2.12.0). Upstream closed this fork's PR NVIDIA#486 on 2026-09-15; NVIDIA#490 builds on it and is open. Verified on Windows in fresh-process batches against this tree: tests/unit 1563 passed, 29 skipped; tests/nodes 3822 passed, 11 skipped, 4 xfailed (plus test_json_container_ownership 71 passed after the id fix); remaining tests 182 passed, 16 skipped; test_security_end_to_end.py 98 passed. ruff check and format clean, check_divergence OK (10 diverging, 10 registered), check_pin_bounds OK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: SanHsien <34234698+SanHsien@users.noreply.github.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head 39d584c543028e77b9ed5bb35f92915d3dea50eb against the prior release findings, resolved threads, complete four-file release diff, current main ancestry, included PR states, version/lock consistency, and hosted checks.
The two prior security-claim findings are resolved: the letter-spacing and companion-classifier limitations are now stated explicitly without claiming fail-closed coverage.
A new exact-head release drift must be corrected. This head's second parent is current main 8028ce5d0db6ea2500a856252f82474f2103bf2b, which includes the P0 local-input security fix #562 (c3bb8132fdba) after the documented validation baseline 548e5e0afd25. However, #562 is absent from CHANGELOG.md and the release notes, and docs/release/skillspector-2.12.0.md still says the candidate was synchronized and validated only through 548e5e0. Add the included security fix to the changelog/release notes, update the exact candidate baseline, and rerun/record validation for the actual current head.
GitHub reports no checks for this exact head and mergeStateStatus=BLOCKED. The documentation/validation drift, missing exact-head checks, and remaining release qualification gates prevent merge.
Priority: P0 — release integrity and security-fix traceability require owner attention before publication.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head 09c3613e06eb2bfd6cdd6e4ce27a183330ee6b6d after the final automatic synchronization with settled main d162d9b343e559be13df8ebba093df3bc9d58c90. I verified the merge parentage, complete four-file release diff, prior findings/threads, version/lock consistency, and the post-baseline main history.
The earlier letter-spacing and companion-classifier claims remain correctly qualified. The release-integrity finding remains open and has widened: this exact candidate now incorporates the post-validation merges for #562, #526, #421, #491, #517, #571, and #575 (plus documentation cleanup), while the changelog/release notes still claim validation only through 548e5e0 and omit these included changes. Update the candidate inventory and security/behavior notes, record the true exact baseline, and rerun/record release validation for this head.
GitHub currently reports no checks for this exact head and mergeStateStatus=BLOCKED. The stale release inventory/validation, missing exact-head CI, and remaining release qualification gates prevent merge.
Priority: P0 — release traceability and validation must describe the code actually being published.
Prepare SkillSpector 2.12.0 from the published v2.11.2 release by updating the package version, lockfile, changelog, and versioned release notes.
This remains a minor release because it adds backward-compatible user-facing controls (
--fail-on-findingsand a configurable per-artifact static-analysis allowance), OpenCode integrations, Gemini model metadata, and interactive scan progress alongside security, completeness, reporting, reliability, performance, and false-positive fixes. Findings, scores, recommendations, completeness metadata, output routing, and opt-in exit behavior can change after upgrading.Included changes
New capabilities and CLI UX
gemini-3.5-flashbudgets and document its OpenAI-compatible configuration (Feature/gemini support and cli ux #7).opencode_clias a semantic-analysis provider for exactly OpenCode 1.18.30, with local-login checks, a verified deny-all policy, and fail-closed version/policy/output handling (feat(providers): add opencode_cli semantic-scan provider #536)./skillspectorcommand andskillspector_scantool. Static analysis is the default; semantic analysis is opt-in, and host-capability denial prevents launch (feat(analyzer): add OpenCode-native SkillSpector invocation skill and tool #537).--fail-on-findingsbehavior for single-skill, recursive, and MCP registry scans (feat(cli): add fail-on-findings option #469).SKILLSPECTOR_MAX_STATIC_ANALYSIS_SECONDS_PER_ARTIFACTto configure the per-artifact static-analysis allowance (feat: make static analysis runtime configurable #522).Security, completeness, and finding integrity
opaque_content, preventing a misleading complete orSAFEresult (fix(hooks): fail closed on unmodeled payload analysis #573).DO_NOT_INSTALL) when referenced or uninspected exclusions prevent complete analysis (fix(security): account for excluded executables #548).Accuracy, output, and performance
--outputis omitted and keep machine-readable JSON/SARIF stdout free of progress, advisories, and transitive warnings (fix(cli): print recursive SARIF to stdout and keep status text off it #565)./blob/file URLs to raw endpoints before download so scans inspect the requested file instead of forge HTML (fix(input): download the raw file for GitHub and GitLab /blob/ URLs #566).SKILLSPECTOR_MODEL_REGISTRYtoken-budget overrides for CLI providers, with safe fallback for missing or invalid entries (fix(providers): honor model registry overrides for CLI providers #463).Maintenance and documentation
Compatibility
--verboseretains the non-streamed diagnostic path./blob/inputs now scan raw bytes, so findings and recommendations can change from the prior incorrect forge-HTML result.SAFE; identifier-adjacent letter spacing can lose false P3/P4 findings while retaining AE6.opencode_cliand the OpenCode-native integration are opt-in. The provider requires an installed, authenticated OpenCode 1.18.30 executable; other versions fail closed. Install the native integration by copying.opencode/from a checkout (the wheel does not install it).SKILLSPECTOR_MODEL_REGISTRYbudgets; malformed values warn and retain fallback behavior.--no-llmfor an intended static-only scan. Requested but unavailable or incomplete semantic analysis remains incomplete and can affect strict CLI/MCP decisions.Candidate and validation
v2.11.2..548e5e0afd25595ef039c27cdeb283413c282c71.548e5e0afd25595ef039c27cdeb283413c282c71(fix(hooks): fail closed on unmodeled payload analysis #573); the release branch is zero commits behind.3dd2defa4beee58b9342fcdad2f0fe0e1a7add64.make test-ci: 5,744 passed, 14 skipped, 39 deselected, 4 expected failures; 90% coverage.skillspector --version, release-helper dry run, Ruff lint/format, package build, Twine checks, andgit diff --check: pass.The current PR head and its GitHub checks are the source of truth for final approval. Older candidate hashes, artifact identities, test counts, and hosted runs are historical and are intentionally not treated as certification of this candidate.
Known limitations and release status
TypeErrorwhen an optional end column is absent, although normal parsed files pass.--outputis used; recursive JSON and SARIF now emit their merged documents.opencode_clidepends on the installed CLI's authentication, exact 1.18.30 behavior, model availability, and external rate limits; token-usage accounting is not currently available, and unsupported output fails closed. Multilingual batch gap-fill was validated with mocked providers; live-provider qualification remains pending.Publication is pending required review and the outstanding release gates. No 2.12.0 release has been published.