Skip to content

fix(jobs): stale_timeout=0 means no grace period - #1623

Open
dimitri-yatsenko wants to merge 1 commit into
fix/jobs-refresh-restrictionfrom
fix/stale-timeout-zero
Open

dimitri-yatsenko wants to merge 1 commit into
fix/jobs-refresh-restrictionfrom
fix/stale-timeout-zero

Conversation

@dimitri-yatsenko

Copy link
Copy Markdown
Member

refresh() removes a job whose key has left key_source once the job is older than stale_timeout, a grace period for a key_source that is still being filled. stale_timeout=0 skipped that check instead, so there was no way to ask for the cleanup now: refresh(stale_timeout=0) left every such job in place, and so did populate() with config.jobs.stale_timeout = 0.

0 now means no grace period: on that refresh, every job whose key is no longer in key_source is removed, whatever its age, unless it is marked ignore. That includes reserved jobs, and success records of rows that remain after their key left key_source, which a nonzero stale_timeout keeps for that many seconds. stale_timeout=None still means the configured value (config.jobs.stale_timeout, 3600 seconds by default), so a refresh that does not pass it keeps the grace period. The setting's description, "Seconds before pending job is checked for staleness", now describes the grace period and what 0 does.

Skipping the check is no longer expressible as 0; a long stale_timeout keeps such jobs for as long as it says.

The tests delete upstream rows and check, on MySQL and PostgreSQL, that a default refresh keeps the pending jobs of the deleted keys, that refresh(stale_timeout=0) removes them, and that populate() with config.jobs.stale_timeout = 0 removes them instead of trying to compute them. Each fails without the change. They replace the test that held 0 to its old meaning.

Closes #1577.

Stacked on #1595: its base is fix/jobs-refresh-restriction until that merges.

refresh() removes a job whose key has left key_source once the job is
older than stale_timeout, a grace period for a key_source that is still
being filled. stale_timeout=0 skipped that check instead, so there was no
way to ask for the cleanup now: refresh(stale_timeout=0) left every such
job in place, and so did populate() with config.jobs.stale_timeout = 0.

0 now means no grace period: on that refresh, every job whose key is no
longer in key_source is removed, whatever its age, unless it is marked
ignore. That includes reserved jobs, and success records of rows that
remain after their key left key_source, which a nonzero stale_timeout
keeps for that many seconds. stale_timeout=None still means the
configured value (config.jobs.stale_timeout, 3600 seconds by default), so
a refresh that does not pass it keeps the grace period. The setting's
description, "Seconds before pending job is checked for staleness", now
describes the grace period and what 0 does.

Skipping the check is no longer expressible as 0; a long stale_timeout
keeps such jobs for as long as it says.

The tests delete upstream rows and check, on MySQL and PostgreSQL, that a
default refresh keeps the pending jobs of the deleted keys, that
refresh(stale_timeout=0) removes them, and that populate() with
config.jobs.stale_timeout = 0 removes them instead of trying to compute
them. Each fails without the change. They replace the test that held 0 to
its old meaning.

Closes #1577.
@dimitri-yatsenko dimitri-yatsenko added this to the v2.4 milestone Oct 17, 2026
@dimitri-yatsenko dimitri-yatsenko added the bug Indicates an unexpected problem or unintended behavior label Oct 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Indicates an unexpected problem or unintended behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant