Skip to content

Speed up breeze database startup by shorten the healthcheck interval - #73167

Merged
potiuk merged 1 commit into
apache:mainfrom
Andrushika:breeze-db-healthcheck-start-interval
Sep 16, 2026
Merged

potiuk merged 1 commit into
apache:mainfrom
Andrushika:breeze-db-healthcheck-start-interval

Conversation

@Andrushika

@Andrushika Andrushika commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Why

breeze start-airflow starts the postgres or mysql container first and waits until Docker marks it healthy before starting the airflow container. Docker Engine 25 probes the healthcheck every start_interval during start_period, and the default is 5 seconds. A fresh postgres container is usually ready in 2 to 3 seconds, so the first probe at 5 seconds is the thing everyone waits for.

What

Only two line changes: Add start_interval: 1s to the postgres and mysql healthchecks in scripts/ci/docker-compose/. Failed probes inside start_period do not count toward retries, and after start_period the normal interval: 10s applies as before.

Measured on a fresh postgres container, from compose up to the airflow container being allowed to start:

before after
postgres ready 7.75s 3.4s

start_interval needs Docker Engine 25 and Compose 2.20.2, which are already the minimum versions breeze checks for in global_constants.py.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Fable 5.1)

Generated-by: Claude Code (Fable 5.1) following the guidelines

Docker Engine 25 probes a container every start_interval during its
start_period, defaulting to 5 seconds, so a freshly created postgres or
mysql container is reported healthy at least 5 seconds after it is
actually ready and every dependent container waits for it. Probing
every second cuts a fresh postgres start from 7.75s to 3.4s. Breeze
already requires Engine 25 and Compose 2.20.2, the versions that
introduced start_interval.
@Andrushika
Andrushika force-pushed the breeze-db-healthcheck-start-interval branch from d69c9bb to eebe664 Compare September 15, 2026 07:05
@Andrushika Andrushika changed the title Speed up breeze database startup by probing the healthcheck every second Speed up breeze database startup by shorten the healthcheck period Sep 15, 2026
@Andrushika
Andrushika marked this pull request as ready for review September 15, 2026 07:07
@Andrushika Andrushika changed the title Speed up breeze database startup by shorten the healthcheck period Speed up breeze database startup by shorten the healthcheck interval Sep 15, 2026

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Two lines, a measured benefit, and the compatibility question answered before it was asked — this is a nice change.

I checked the one claim that could have bitten us. start_interval needs Docker Engine 25 and Compose 2.20.2, and dev/breeze/src/airflow_breeze/global_constants.py already pins exactly those as the floor:

MIN_DOCKER_VERSION = "25.0.0"
MIN_DOCKER_COMPOSE_VERSION = "2.20.2"

So breeze refuses to run on anything that would silently ignore the new key. Had those floors been lower, an older Compose would have dropped start_interval without complaint and the change would have looked like it worked while doing nothing — worth saying out loud, since that is the failure mode this class of change usually has.

The semantics are right too: probes that fail inside start_period do not count against retries, a successful probe ends the start period immediately, and interval: 10s resumes afterwards, so the only thing this changes is how quickly a container that is already ready gets noticed. On MySQL it means up to sixty cheap probes instead of twelve during the start window, which is not a cost worth worrying about against four seconds saved on every single breeze start-airflow.

One observation rather than a request: about a dozen other healthchecks under scripts/ci/docker-compose/ (integration-cassandra, -mongo, -celery, -kerberos, -keycloak, -openlineage, -pinot, -trino, -elasticsearch, -localstack, -opensearch) have start_period and no start_interval, so the same one-line win is sitting there. Keeping this PR to the two backends looks like the right call — they are on the critical path of every start-up, while the integrations are opt-in — but if you were already planning a follow-up for the rest, that would be a welcome one.

No newsfragment needed and none added, which is right for dev tooling.


This review was drafted by an AI-assisted tool and
confirmed by an Airflow maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Airflow handles maintainer review:
contributing-docs/05_pull_requests.rst.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

@potiuk
potiuk merged commit 386132e into apache:main Sep 16, 2026
183 of 212 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test PR Link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants