refactor(ci): scope toolchain validation

[imbajin]

Documentation-only PRs currently allocate several runners, routine edits also start both Server baselines and broad specialty workflows, and master merges repeat the same validation. A report-only job can keep finished tests waiting in the shared queue.

This change uses one Module validation entry and the existing selector:

• prose documentation runs only the entry; automatic labeling is removed, and cumulative source changes still select their consumers
• normal PR validation uses the current Server baseline; legacy compatibility, HDFS, Spark/Flink, image data checks and CodeQL follow their direct inputs
• master runtime pushes use one locked-baseline smoke runner: compile the current Java consumers, check Client/Loader read-write behavior, backend unit tests, Tools CLI and Go requests
• weekly full validation runs Monday at 06:00 Asia/Singapore; native entry selection offers mainline and full
• license and fixture contracts share the entry, image checks share one job per image, and standalone planners, result summaries and automatic retries are removed

Required CI contexts become empty while approving review protection remains. The real check-license-header entry satisfies the old rule until Gitbox reconciles it. Same-source SDK provenance, archive checksums, HTTPS fixtures, actual consumer tests and native job results remain enforced. Reusable matrices keep the complete workflow within ASF's recommended concurrency.

Validation: policy and runtime fixtures, Node checkout/install tests, the real Maven parent-metadata regression, image contract tests, RAT, actionlint and independent diff review passed locally. The org full workflow exercised the retained consumers and real images successfully before the final routing update; latest-head PR validation and native smoke validation are pending.

Loader production inputs now also validate the packaged Spark Loader and Flink CDC consumers; HDFS follows shared file/line readers and source contracts. Entry-workflow PR changes exercise every caller, including smoke, while master remains one smoke job and release pushes avoid duplicate smoke. Targeted selection regressions and independent review confirm these paths.

Conceptual overview

One entry selects validation from the cumulative diff: prose-only entry, affected current-baseline consumers, master smoke, or full validation. Optional checks follow relevant inputs; CI remains advisory with required human review.

Conceptual illustration; not test or release-acceptance evidence.

Observed master-push performance

Snapshot: 2026-10-15 (Asia/Taipei). The baseline is the three days before each CI change; the post-change sample includes all observed successful master pushes after Server #3337 and Toolchain #816.

Values are per-commit averages, before → after. All push workflows for the same commit are grouped together; failed or cancelled groups are excluded.

Repository	Successful commit samples	Runner jobs / commit	Runner execution (min)	First-start wait (min)	End-to-end completion (min)
Server	20 → 7	26.4 → 3.9	212.5 → 12.8	1.15 → 0.61	101.6 → 8.8
Toolchain	7 → 1	17.4 → 2.0	111.6 → 10.1	18.9 → 149.6	105.4 → 242.2

Runner execution sums the actual job durations across parallel runners. First-start wait runs from the earliest workflow creation to the first allocated runner starting. End-to-end completion ends when the last runner job finishes, including subsequent queue and dependency waits; it excludes external Codecov processing. The historical jobs API does not expose each job's enqueue time, so first-start wait is not average per-job queue time.

Observed runner execution decreased by approximately 94% for Server and 91% for Toolchain. Toolchain has only one post-change master sample: its entry and smoke run used 10.1 runner-minutes, but waited 149.6 minutes before the entry and another 82.5 minutes before smoke. Its longer completion time therefore does not indicate longer test execution.

These are observations across different commits and shared-queue conditions, not a controlled benchmark or evidence of a lasting queue-time improvement.

[bitflicker64]

Blocking: yes. Score: 6/10 (10 - 2 Important - 0.5 Minor = 7.5, capped at 6 for a blocking finding). Summary: The single entry, smoke-after-merge and scoped baselines are well covered by the policy tests, but loader core edits now skip the Spark Loader, Flink CDC and HDFS checks that exercise them until the weekly run, and ci.yml edits do not run the callers it now hosts. Evidence: ran python3 -m unittest discover -s .github/scripts -p 'test_*.py' at 11a355a (93 tests, OK, 4 skipped); called create_plan with single-path PR and push diffs to confirm the selections quoted inline; traced ci.yml callers and the reusable workflows statically. Most PR checks were still pending at this head; check-license-header passed.

[bitflicker64]

Blocking: no. Score: 10/10 (10 - 0). Summary: 4ca0605 fixes both points from my review at 11a355a. A Loader src/main/ change now selects Spark Loader and Flink CDC, and shared file/line reader and source contracts also select HDFS. A PR that edits ci.yml now runs every caller it hosts, while master pushes stay at entry plus smoke. Evidence: python3 -m unittest discover -s .github/scripts -p 'test_*.py' at 4ca0605: 95 tests OK, 4 skipped. Planner probes: ElementBuilder.java alone selects spark_loader and flink_cdc; reader/file/FileReader.java and reader/AbstractReader.java also select hdfs; ci.yml alone on a PR selects all optional checks, both images, security, dependency audit and mainline smoke. On a master push, ci.yml and ElementBuilder.java each give smoke only; a release push leaves smoke off. CI on this PR already shows the second fix live: the images, flink-cdc, launcher, upstream-master, CodeQL java, dependency audit and mainline smoke passed, as did the required check-license-header. Nothing has failed; the rest were still pending at review time.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.06%. Comparing base (8e6ed84) to head (4ca0605).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #816      +/-   ##
============================================
+ Coverage     47.04%   47.06%   +0.02%     
- Complexity     4517     4518       +1     
============================================
  Files           602      602              
  Lines         29448    29452       +4     
  Branches       3454     3456       +2     
============================================
+ Hits          13853    13862       +9     
+ Misses        14312    14308       -4     
+ Partials       1283     1282       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.