[architecture] Extract testing: PHPUnit execution, extensions, coverage and PR feedback #380

Activity

1. coisa

   added

   enhancementNew feature or requestNew feature or request

   on Oct 21, 2026
2. github-project-automation

   added this to PHP Fast Forward Projecton Oct 21, 2026
3. github-project-automation

   moved this to Backlog in PHP Fast Forward Projecton Oct 21, 2026
4. coisa commented on Oct 25, 2026

   

   coisa

   on Oct 25, 2026

   ContributorAuthor

   More actions

   Testing extraction: initial custody, assessment and next gate — 2026-10-25

   The forwarded maintainer direction starts the implementation lane for this existing issue, while retaining separate repository-creation, release and consumer-adoption gates. This executor retains DevTools #376 and the Testing lane; Code Standards and consumer/framework executors keep their existing scopes. No additional implementer or card is introduced.

   Verified evidence

   • PR #376 remains open/unmerged at 4f3cf00517fb64adbaa26689b64431d27b40266d, base 84e154046e07cafa6ac09a9e12434b1ff5dcb5e0.
   • This issue is open and had no assignee/comments when read. Executor custody comes from the coordination direction; no GitHub assignment was inferred.
   • Read [architecture] Plan domain-owned tooling extraction and DevTools orchestration #240 and Define release-safe refs for consumer workflow wrappers and checked-out actions #238. DevTools retains composition; domain packages own behavior and compatible runtime/workflow/skill refs. Installation does not authorize publication.
   • Organization search found no Testing repository; direct php-fast-forward/testing API lookup returned 404. This means the proposed destination is not accessible through this connection, not proof that no differently named/private destination exists.
   • Inspected the real Changelog CheckCommand: readonly invocable POPO, Symfony command/input attributes, injected service interfaces, SymfonyStyle and explicit exits. Its manifest currently requires PHP ^8.5 and Symfony Console ^8.1; those requirements must not silently become the public Testing consumer floor.

   Bounded implementation plan

   1. Preserve existing staged optional-skills/profile work. Use an isolated Testing worktree from the verified current DevTools head; do not edit another executor's checkout.
   2. Inventory TestsCommand, src/PhpUnit/{Bootstrap,Coverage,Runner,Event}, their tests, phpunit.xml, testing-owned workflow/feedback sections and resources/skills/fast-forward-phpunit-tests. Keep aggregate reports and site publication with their current owners.
   3. Extract plain injected services for resolution, execution and result interpretation behind a thin invocable Symfony Console adapter. Preserve public PHPUnit/Prophecy/extension APIs, project config/bootstrap, cwd/cache/color, artifacts and exit behavior. CPX does not isolate classes sharing a PHP process.
   4. Define source-SHA/runtime/suite identity, pass/failure/valid-skip/no-tests, thresholds and artifact paths. Missing coverage cannot silently mean zero or success. Notifications and final bypass remain explicit opt-ins.
   5. Keep Add --parallel support to tests command for safe, zero-workaround parallel PHPUnit execution #8, Add Infection command integration to evaluate mutation testing coverage #262 and Resolve PHP test matrix from Composer constraints and setup-php stable availability #246 as separate capabilities. Bootstrap against verified compatible Code Standards and Changelog versions; initial Testing version line is 0.x. No production dependency back to DevTools.
   6. Integrate released usable Code Standards and its canonical skill into existing feat!: distribute DevTools through CPX and portable skills #376 only after version/archive/CLI/provenance verification, sequentially with its owner. No duplicate extraction PR.

   Scenarios and observed results

   Scenario	Status
   Current issue/PR state, contracts and reference implementation	Read and verified as above
   Passing/failing/skipped/empty/guide-only suites	Planned; not executed for an extracted Testing package
   Coverage threshold failure / unavailable driver	Planned; not executed
   Custom extensions / incompatible APIs / consumer overrides	Planned; not executed
   Local/global/CPX bootstrap, cwd/cache/color and artifact equivalence	Planned; not executed
   Fork PR execution and malformed/stale result artifacts	Planned; not executed
   Composer archive/resources/skill provenance, no-op and no-write preview	Planned; not executed

   Untrusted PR execution stays read-only. Any trusted feedback publisher must validate bounded artifacts against repository/PR/head SHA and never execute their contents. Serialized PHP coverage remains a trusted local API concern, not a trusted feedback artifact format.

   Exact pending actions

   • Destination resolution: identify the existing canonical Testing repository/path. If none exists, obtain explicit repository-creation authorization. This is a material prerequisite for the standalone extraction PR, not a reason to block local inventory/compatibility preparation.
   • Internal dependency evidence: verify usable released Code Standards version and payload with its owner. This is dependency coordination, not a new human architecture approval.
   • Publication gate: the suspended queue remains suspended. Local preparation/tests are allowed; no publication retry, alternate route, hook bypass or policy/credential change. PR publication requires the valid flow and exact-SHA review; report the tested patch and precise pending action if unavailable.

   No code changes, new repository/PR, merge, release or consumer rollout were performed in this assessment. Issue type, labels, assignee, milestone and project fields are unchanged; existing links provide coordination without another card.
5. coisa commented on Oct 25, 2026

   

   coisa

   on Oct 25, 2026

   ContributorAuthor

   More actions

   Local Testing preparation: imports/API inventory and executable bootstrap fixture — 2026-10-25

   Continuing the authorized local lane without waiting for a remote repository. This is a tested local preparation patch, not an extracted package or publication approval.

   Source: DevTools #376 at 4f3cf00517fb64adbaa26689b64431d27b40266d. Work is isolated; existing staged profiles/optional-skills and Code Standards/consumer checkouts were preserved.

   Inventory and concrete constraints

   The implementation boundary contains TestsCommand (645 lines) and eight PhpUnit files (775 lines), plus XML, DI, reporter/process environment, workflow and artifact contracts. They are not safe mechanical deletions.

   • Local PHPUnit is preferred independently of the tooling runtime. CoverageSummaryLoader executes coverage.php and expects its own CodeCoverage class: cross-major serialized objects are a concrete compatibility boundary. Keep this trusted-local API separate from normalized process results and trusted feedback artifacts.
   • Bootstrap runtime path, notification icon and XML extension FQCN still identify DevTools. Moving namespaces needs a bridge or deliberate configuration migration. Another autoloader cannot replace a class already loaded.
   • DevToolsExtension accepts generic Tracer but its default notification subscriber requires EventTracer. Custom tracer alone can therefore fail at construction; this existing source-level limitation is recorded, not fixed incidentally.
   • Coverage arguments only use PSR-4 while source capability discovery understands other mappings. Preserve/characterize files/classmap/PSR-0 and test-only skip policy explicitly.
   • Zero executable lines currently yields 100%; a missing coverage report is a failure. These are distinct contracts, not interchangeable defaults.
   • Read-only scan of 699 PHP files under local primary consumer src/tests found no direct FastForward\\DevTools\\PhpUnit imports. This bounded local observation is not remote or active-worktree inventory. The consumer owner's fresher matrix remains authoritative for that lane.

   Actual local patch and results

   Added only tests/PhpUnit/Bootstrap/BootstrapShimExecutionTest.php; final file SHA-256:
   07dcd8f58ed6563cc869cbb78766876eaaea1ae3cf6b57a662fe7bc3e0555d1e.

   Scenario	Observed result
   Execute real generated shim with runtime Composer loader already registered	Passed
   Consumer prepend loader wins a previously unresolved overlapping class; separate runtime class remains available	Passed
   Missing consumer bootstrap exits nonzero before continuation	Passed
   Consumer bootstrap and manifest contents unchanged	Passed
   New regression scope	2 tests / 13 assertions passed
   Selected existing PhpUnit/TestsCommand scope	45 tests / 156 assertions passed
   New file PHP lint / scoped ECS	Clean; ECS 0 errors, 0 diffs

   Used PHP 8.5.12, PHPUnit 12.5.40, php-code-coverage 12.5.8 and Symfony Console/Process 7.4.21 from an existing donor runtime, with test bootstrap remapping source/tests to the isolated worktree. No new package was installed. An initial Composer setup could not write a generated lock under filesystem restrictions, so the bounded tests reused the existing donor; there is no fresh-install claim. An existing selected test needed a writable worktree fixture directory; the same scope passed with permitted filesystem execution. No hook, policy or credentials changed.

   Not executed: full DevTools quality/report/coverage gate; PHP 8.3/8.4 matrix; cross-PHPUnit-major coverage compatibility; extracted Composer archive/install; fork feedback publisher; Code Standards adoption; consumer lifecycle or namespace isolation. Focused tests do not establish those claims.

   Supplier role/sync reconciliation

   Read the framework lifecycle audit. At the current supplier candidate, changelog-maintainer already declares fast-forward-changelog, and SyncCommand has no skills/agents synchronization reference. The installed legacy supplier is different. This does not prove upgrade/removal safety there; the existing consumer owner keeps the lifecycle patch and verification. No consumer edit or duplicate implementation was made.

   Next precise actions

   Continue separating bootstrap/runtime inputs and result interpretation from Console I/O, following the real Changelog invocable POPO pattern without copying its PHP 8.5/Symfony 8.1 floor blindly. Keep public extensions and local configs compatible; bypass/notifications remain opt-in in the extracted contract. #8/#262/#246 remain separate.

   Canonical Testing destination is still needed for a standalone PR, not for local inventory/fixtures. A valid exact-SHA review/publication flow is needed before publishing this patch; the suspended pipeline was not restarted or bypassed. No commit, push, PR, repository creation, merge, release or consumer rollout occurred. Existing issue metadata is unchanged.
6. coisa commented on Oct 25, 2026

   

   coisa

   on Oct 25, 2026

   ContributorAuthor

   More actions

   Local implementation cut: bootstrap runtime input and result interpretation — 2026-10-25

   Implemented the next bounded local cut from the previous fixture/inventory assessment, without creating a standalone repository or resuming publication.

   Baseline commit: 4f3cf00517fb64adbaa26689b64431d27b40266d (#376). This is an uncommitted local patch, not a new commit SHA or published package.

   Behavior and API

   • Added src/PhpUnit/Result/ProcessResultInterpreter.php: final readonly invocable POPO, accepting captured string + integer exit status and returning the existing payload. It has no Console/process/filesystem dependency. The parser was moved from TestsCommand without changing legacy semantics.
   • TestsCommand retains process execution, BufferedOutput consumption, coverage validation and presentation. A last optional constructor collaborator defaults to the interpreter, preserving existing nine-argument callers and command names/options.
   • BootstrapShimGenerator::generate() accepts optional third runtimeAutoload input. Existing two-argument callers keep the same DevTools runtime resolution and consumer-first content. An explicit caller-owned runtime can now be passed without changing the consumer configuration or public namespace.
   • No new interface family, generic runtime, dependency, PHP/Symfony floor, namespace relocation or capability/profile policy was introduced. Existing bypass/notification behavior was not changed in this compatibility cut; explicit opt-in profiles remain an extraction requirement.

   Final local identity

   Manifest SHA-256 across the seven files:
   c128b72864f1dee2ce71ef8e748957cb88155e8aced369cea2c8058fb0ada5d7.

   Algorithm: sort repository-relative paths; concatenate each path + NUL + lowercase file SHA-256 + LF; SHA-256 the concatenation. This identifies local file contents, not a Git commit/tree.

   File	SHA-256
   src/Console/Command/TestsCommand.php	df94ef64cb1839d5937d69ee2983ab2394e5ca6da92b2671566097a8287cca2d
   src/PhpUnit/Bootstrap/BootstrapShimGenerator.php	1d5e386a58f9a84d84e7f8f6d7fb231e4b9d1c4c78b7955f3aa46a23054ea2e2
   src/PhpUnit/Result/ProcessResultInterpreter.php	de067737bf5cfb3c31275488299fc4a1b8f346939751a32f0333c4abab85ec50
   tests/Console/Command/TestsCommandTest.php	14ca8247be4a881e1f45b098af34a25c3e7c49e26bb7ca01a55f24196dc05da7
   tests/PhpUnit/Bootstrap/BootstrapShimExecutionTest.php	10888ae82e103b23f7fc2c10f86f0ef807cb6a68ed58d893483237cf13e2d122
   tests/PhpUnit/Result/ProcessResultInterpreterTest.php	12882d6d358c49b41171b738113cc69b2834b921db3d71fc4235041f03a3e517
   docs/usage/testing-and-coverage.rst	665d648213bd0a232363f6d907135e5c9b54293bb93a24c04a5e32995e2630a9

   Actual validation, sequential and focused

   • Final selected PhpUnit + TestsCommand tests: 62 tests / 181 assertions passed, PHP 8.5.12 / PHPUnit 12.5.40, no coverage collection, no result cache, no configuration extensions. Existing donor dependencies were reused read-only with source/test mapping to the isolated worktree.
   • Interpreter cases: empty/whitespace; reporter summary; nonzero exit overriding reporter success and zero overriding reporter failure; supplemental text; malformed JSON/trailing output; legacy scalar JSON discard; raw_output precedence and existing first-key-result prelude constraint.
   • Bootstrap subprocess cases: default consumer precedence/runtime fallback; missing consumer bootstrap failure; explicit runtime path containing spaces/apostrophe; consumer→runtime order; escaped chosen path and unchanged consumer/bootstrap/runtime files.
   • Scoped ECS: 0 errors / 0 diffs across six PHP files. PHP lint and whitespace checks passed; RST underline check passed. Additional PHP 8.4 syntax-only check passed for the new interpreter, not a PHP 8.4 behavioral suite.
   • Independent read-only review found no actionable issue and confirmed the exact seven-file manifest above. This is local code review, not GitHub approval or publication authorization.

   Not executed: full DevTools gate/coverage reports, full PHP 8.3/8.4 runtime matrix, cross-major PHPUnit coverage compatibility, extracted archive/install, fork publisher, consumer lifecycle/adoption, Code Standards integration. No release-style result schema/source-SHA fields or namespace isolation are claimed.

   Remaining precise work and boundaries

   Continue resolving the existing public extension/bootstrap and coverage-object compatibility constraints before a standalone extraction. Canonical remote destination remains necessary for its PR, while this local implementation does not depend on creating one. Code Standards release/skill integration stays coordinated with its exclusive owner; consumer/framework checkouts were untouched.

   The publication gate remains suspended. No code commit/push/PR, repository creation, merge/release, hook alteration, policy/credential change or pipeline retry occurred. Existing profiles/skills staged work and issue metadata remain unchanged. #8/#262/#246 are still separate capabilities.