Skip to content

[FLINK-40163] [ci] Add new baseline nightly build to Github Actions#28768

Open
dalelane wants to merge 1 commit into
apache:masterfrom
dalelane:flink-40163-add-baseline-nightly-gha-release
Open

[FLINK-40163] [ci] Add new baseline nightly build to Github Actions#28768
dalelane wants to merge 1 commit into
apache:masterfrom
dalelane:flink-40163-add-baseline-nightly-gha-release

Conversation

@dalelane

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Azure's nightly stage in tools/azure-pipelines/build-apache-repo.yml has cron_azure as a default baseline build, but there is no equivalent build in the Github Actions scheduled nightly pipeline.

The goal of this commit is to update the GHA nightly release pipeline towards parity with the AZP implementation, treating it as a reference.

Brief change log

Added new Java 17 job to the GHA nightly release pipeline

Verifying this change

Manually triggered the Github Actions pipeline in my own fork.

(see dalelane#3 )

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Was generative AI tooling used to co-author this PR?

no

Azure's nightly stage in tools/azure-pipelines/build-apache-repo.yml
has cron_azure as a default baseline build, but there is no
equivalent build in the Github Actions scheduled nightly pipeline.

The goal of this commit is to update the GHA nightly release
pipeline towards parity with the AZP implementation, treating it
as a reference.

Signed-off-by: Dale Lane <dale.lane@uk.ibm.com>
@flinkbot

flinkbot commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@snuyanzin

Copy link
Copy Markdown
Contributor

@dalelane can you please explain why do we need it?
We already have java17 in default ci for merged PRs

@snuyanzin

Copy link
Copy Markdown
Contributor

The reason I'm asking is that we have a limited resource for GHA
and Apache infra has already complained that we use too much, see for more details https://lists.apache.org/thread/jn9t7qjj4wlz98t1ddqxwypcn6g7wrds

now if we add this it will be added to master and release branches (supported)

@dalelane

Copy link
Copy Markdown
Contributor Author

@snuyanzin The (admittedly over-long!) background is in https://issues.apache.org/jira/browse/FLINK-27075?focusedCommentId=18095329&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-18095329

The TLDR is that (given the overall objective to get the GHA pipelines to a point where they could be a viable replacement for the Azure pipelines) I looked to see where there were things done in the AZP pipeline with no equivalent in the GHA pipeline. The goal was parity, rather than coming at this from first principles to decide if it's needed in it's own right.

It's obviously entirely valid to take a step back and decide on whether it's needed, rather than just blindly copying it across (which, to be fair, @XComp did highlight in dalelane#3 (review) ) but I lack the context to have a strong opinion on that. I'm open to a steer from you and @XComp on what would be sensible.

uses: ./.github/workflows/template.flink-ci.yml
with:
workflow-caller-id: java17
environment: 'PROFILE="-Dflink.hadoop.version=2.10.2 -Djdk17 -Pjava17-target"'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this -Dflink.hadoop.version=2.10.2 but java 11 and 21 are -Dinclude_hadoop_aws

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants