Skip to content

fix(jobs): record _job_start_time on the server's clock - #1622

Open
dimitri-yatsenko wants to merge 1 commit into
masterfrom
fix/job-start-server-clock
Open

dimitri-yatsenko wants to merge 1 commit into
masterfrom
fix/job-start-server-clock

Conversation

@dimitri-yatsenko

Copy link
Copy Markdown
Member

populate() wrote _job_start_time as the populating host's local time, with no zone: time.time() at the start of the job, converted by datetime.fromtimestamp(). Two hosts in different time zones recorded different values for the same instant, and nothing in the row said which zone applied. The value could not be placed beside the job table's reserved_time and completed_time, which come from the server.

The start time is now computed in the UPDATE that already writes the hidden columns, as the server's current time less _job_duration, so no query is added. The expression is the start of the statement, not of the transaction: NOW(3) on MySQL, and statement_timestamp() on PostgreSQL, where CURRENT_TIMESTAMP is the start of the transaction. It comes from a new adapter method, statement_timestamp_expr().

_job_duration is still measured on the populating host, now with time.monotonic(), which neither the host's zone nor an adjustment to its clock affects. The UPDATE runs just after the commit that the duration includes, so the derived start is late by the time that one statement takes to reach the server.

Rows written before this change keep the populating host's local time. They cannot be corrected, because the zone was never recorded.

The test populates two keys as if from hosts in UTC-12 and UTC+14 and checks, on MySQL and PostgreSQL, that each _job_start_time falls between two readings of the server's clock taken around the call. Without the change it fails on both, 12 hours off.

Closes #1584.

populate() wrote _job_start_time as the populating host's local time, with
no zone: time.time() at the start of the job, converted by
datetime.fromtimestamp(). Two hosts in different time zones recorded
different values for the same instant, and nothing in the row said which
zone applied. The value could not be placed beside the job table's
reserved_time and completed_time, which come from the server.

The start time is now computed in the UPDATE that already writes the hidden
columns, as the server's current time less _job_duration, so no query is
added. The expression is the start of the statement, not of the
transaction: NOW(3) on MySQL, and statement_timestamp() on PostgreSQL,
where CURRENT_TIMESTAMP is the start of the transaction. It comes from a
new adapter method, statement_timestamp_expr().

_job_duration is still measured on the populating host, now with
time.monotonic(), which neither the host's zone nor an adjustment to its
clock affects. The UPDATE runs just after the commit that the duration
includes, so the derived start is late by the time that one statement
takes to reach the server.

Rows written before this change keep the populating host's local time.
They cannot be corrected, because the zone was never recorded.

The test populates two keys as if from hosts in UTC-12 and UTC+14 and
checks, on MySQL and PostgreSQL, that each _job_start_time falls between
two readings of the server's clock taken around the call. Without the
change it fails on both, 12 hours off.

Closes #1584.
@dimitri-yatsenko dimitri-yatsenko added this to the v2.4 milestone Oct 17, 2026
@dimitri-yatsenko dimitri-yatsenko added the bug Indicates an unexpected problem or unintended behavior label Oct 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Indicates an unexpected problem or unintended behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_job_start_time records the populating host's local time, with no zone

1 participant