Skip to content

feat(ci): run affected actions in better order - #779

Merged
imbajin merged 13 commits into
apache:masterfrom
hugegraph:chore/ci-concurrency-20261005
Oct 6, 2026
Merged

imbajin merged 13 commits into
apache:masterfrom
hugegraph:chore/ci-concurrency-20261005

Conversation

@imbajin

@imbajin imbajin commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Select affected consumers and share Server preparation, while keeping each consumer's behavior tests independent.

HugeGraph Toolchain CI before and after

Before After
Broad module workflows prepare the same Server repeatedly. Select real consumers: Client → Loader/Tools/Spark/Hubble, Loader → Hubble. Build each pinned Server fixture once and start an independent service for each test.
Every Loader profile waits for Hadoop setup. Only HDFS starts NameNode/DataNode from the pinned ASF Hadoop 3.3.6 image; Java 11/17 tests remain on the runner.
Module tests do not validate final container packaging. Deployment and shared build inputs select current Loader/Hubble image checks, including the direct Loader CLI and Hubble UI assets.
Security competes with core tests; results are spread across workflows. Security follows a strict gate requiring all selected tests and image checks to pass. Newer PR heads cancel superseded runs; license checks remain independent.

- cancel older runs of each PR workflow
- verify retry source before and after the delay
- test retry freshness and document CI behavior
- explain PR concurrency alongside workflow policy
- match comments on active target branches
- avoid conflicts when syncing master into PRs
@codecov

codecov Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.75%. Comparing base (b066b80) to head (bc609f9).
⚠️ Report is 250 commits behind head on master.

Additional details and impacted files
@@              Coverage Diff              @@
##             master     #779       +/-   ##
=============================================
- Coverage     62.49%   45.75%   -16.75%     
- Complexity     1903     4315     +2412     
=============================================
  Files           262      602      +340     
  Lines          9541    29044    +19503     
  Branches        886     3369     +2483     
=============================================
+ Hits           5963    13288     +7325     
- Misses         3190    14505    +11315     
- Partials        388     1251      +863     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The concurrency groups look right and the checker's tests pass, but the new PR lookup never finds a fork PR, so automatic retries stop for every PR opened from a fork, including this one. Score 6/10. Evidence: ran test_check_rerun.py at the exact head (all pass), then ran decide() against real runs of this repo through gh api with the run forced to a failed state: fork PR #771 run 37298674115 returns skip ('no open PR in this repository with the failed head'), while same-repo PR #780 run 37263450044 returns rerun.

Comment thread .github/scripts/check-rerun.py Outdated
- select related modules and verify reusable test receipts
- share immutable server fixtures with separate runtime services
- isolate HDFS setup and bound downloads and readiness
- defer security checks and preserve protection migration
@imbajin imbajin changed the title chore(ci): cancel superseded PR runs chore(ci): run affected tests before optional jobs Oct 5, 2026
- retain upstream client and installer compatibility fixes
- reconcile shared artifacts with updated source provenance
- keep automatic fixture checks in the unified CI entry
@imbajin imbajin changed the title chore(ci): run affected tests before optional jobs feat(ci): run affected action with jobs Oct 5, 2026
- restore existing version-specific archives
- validate restored data with the pinned checksum
- retain bounded download and readiness deadlines
- remove the extra trailing blank line
- keep bounded service readiness behavior
- pass the full pull request whitespace check
@github-actions github-actions Bot added hubble hugegraph-hubble loader hugegraph-loader client hugegraph-client labels Oct 5, 2026
- merge upstream loader and tools preparation fixes
- preserve Spark Java 17 test and release fixture setup
- keep shared artifacts and affected-module orchestration
- retain affected suites and verified shared fixtures
- test Toolchain on Java 17 with isolated Server JVMs
- inherit existing Surefire and JaCoCo test-tool versions
- gate both current and released Hubble fixtures
@imbajin
imbajin force-pushed the chore/ci-concurrency-20261005 branch from 996ebbc to c5a2524 Compare October 5, 2026 18:39

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Score: 4/10. Summary: The plan/gate structure and receipt verification are careful, and the claim that business Java code, compiler targets and dependency sources are unchanged holds (only hubble-be test plugins move to the root Surefire/JaCoCo versions). One issue blocks: Analyze (java) fails on every docs-only push to master. Evidence: read the full diff against master (34 files); ran python3 -m unittest discover -s .github/scripts -p 'test_*.py' (40 tests OK), runtime_test.py (6 OK) and the three node --test suites (25 pass); parsed all workflow YAML and ran bash -n on every changed script; called select() and create_plan() with concrete paths and a push event. actionlint and shellcheck were not available. Required CI at c5a2524 is still queued and the four earlier runs on this branch were cancelled, so the new pipeline has not completed end to end yet. Not verified: Hadoop 3.3.6 daemons now inherit JDK 17 in the hdfs job (previously 11).

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/scripts/ci-policy.py
Comment thread .github/scripts/ci-policy.py
Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/spark-connector-ci.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
- include Go consumers of shared fixture scripts
- verify packaged docs and optional proof provenance
- restrict legacy security aliases and write permissions
- document JDK coverage and pending runtime validation
@imbajin imbajin changed the title feat(ci): run affected action with jobs chore(ci): run affected tests before security Oct 5, 2026
- retain necessary fixture and plan artifacts for seven days
- select image startup checks from packaging inputs
- bind image reuse to complete source and job evidence
- preserve module coverage and document retry limits
@github-advanced-security

Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: The two commits since c5a2524 fix all the earlier points: Analyze (java) is PR-only now, scan write permission sits on the security job, Go is selected for the shared Client server scripts, the Hubble baseline only enters the fingerprint when Hubble is selected, the packaged Hubble README forces fresh validation, and fork PRs get a qualified owner:branch lookup in check-rerun.py. The new image jobs and image proofs look correct. One gap is left: no required job compiles Toolchain code with JDK 11 any more, but both Dockerfiles still build with JDK 11, so a Java source change that uses a post-11 API passes every required check and breaks the image build. Score 8/10. Evidence: read the full diff against master (37 files) and the incremental diff c5a2524..0613a57. python3 -m unittest discover -s .github/scripts -p 'test_*.py' gives 61 OK with 2 Server-only skips, runtime_test.py gives 6 OK, and the three node --test suites give 25 pass. Probed select() and select_images() with concrete paths. Compiled a List.of(1).stream().toList() call with -source 1.8 -target 1.8: javac 17 accepts it and javac 11 rejects it. Every required check at 0613a57 is green, including both image jobs; only codecov/project fails, because of the stale base.

Comment thread .github/workflows/loader-ci.yml Outdated
- preserve upstream Boot 3 and Java 17 requirements
- retain shared fixtures and both Server baselines
- include upstream archive and launcher checks
- retain supported JVM validation after the Hubble merge
- check final image entry points and public UI assets
- remove untrusted result reuse and PR artifact retries
- resolve consumed baselines and reject stale PR inputs
Use pinned Hadoop containers with reachable block endpoints
Keep Java 11 and Java 17 consumer tests and bounded readiness
Collect HDFS logs and clean up containers after testing
Preserve historical CodeQL configuration identity
@imbajin imbajin changed the title chore(ci): run affected tests before security feat(ci): run affected actions in better order Oct 6, 2026
@imbajin
imbajin merged commit 595f7ed into apache:master Oct 6, 2026
29 of 30 checks passed
@imbajin
imbajin deleted the chore/ci-concurrency-20261005 branch October 6, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client hugegraph-client hubble hugegraph-hubble loader hugegraph-loader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants