CLOS-4518: Reenable systemd timers left disabled by cron-to-timer migrations - #69
Open
prilr wants to merge 3 commits into
Open
CLOS-4518: Reenable systemd timers left disabled by cron-to-timer migrations#69prilr wants to merge 3 commits into
prilr wants to merge 3 commits into
Conversation
The systemd service-state transition only re-applied source states to units that existed on the source system; units new on the target were dropped by _filter_irrelevant_services. Such units then relied solely on the package %systemd_post scriptlet to apply their vendor preset, which only runs on a fresh install (not when an existing package gains a new unit on upgrade), so preset-enabled units could end up disabled after the upgrade. Keep target-only units in the filter and, when a unit is absent on the source, honor the target vendor preset (enable if preset is "enable"), replicating what a fresh install would do. Guard _get_newly_enabled against the now-possible missing source entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
get_system_service_preset_files() filters preset entries down to '.service'
units and returns SystemdServicePreset models. Vendor presets matter for other
unit types too - timers in particular - and there is no model for those.
Add a small companion that takes a unit-file suffix and returns a plain
{unit: 'enable'|'disable'} mapping. It reuses the same preset discovery and
parsing, so preset-file override and first-match-wins semantics stay identical.
Pure addition: no existing caller changes behavior.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
When a package that already exists on the source system gains a systemd timer
on the target, the timer ends up disabled after an in-place upgrade even though
its vendor preset says 'enable': the %systemd_post scriptlet applies presets
only on a fresh install ($1 -eq 1), and leapp's systemd state transition scans
'--type=service' only, so non-service units are never transitioned.
Two packages migrate a cron job to a preset-enabled timer across EL8->EL9, and
both fail silently:
- logrotate: /etc/cron.daily/logrotate -> logrotate.timer. Nothing rotates
logs; they grow until the disk fills (ZD 284537, ~9.4 GB reclaimed).
- mdadm: /etc/cron.d/raid-check -> raid-check.timer. The weekly software RAID
consistency check never runs, so latent sector errors accumulate undetected
and surface as an unrecoverable read error during an array rebuild.
Rather than curate a list of affected timers, record the source system's timer
inventory during the Facts phase and, on first boot, enable only timers that are
absent from it, disabled, and preset-enabled. A timer that did not exist on the
source cannot have been disabled by the administrator, so applying the target
preset is safe by construction - the same rule the service-state transition uses
for units new on the target. Timers present on the source keep their state, so a
deliberate 'systemctl disable' always survives the upgrade. Without the source
inventory the actor does nothing rather than guess.
Validated CL8 -> CL9.8 on a no-panel VM with mdadm installed: logrotate.timer
and raid-check.timer both enabled and active afterwards (next runs scheduled),
while mdcheck_start/mdcheck_continue/mdmonitor-oneshot - present but disabled on
the source - were correctly left untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After a CL8-CL9 upgrade,
logrotate.timeris left disabled despite a vendor preset ofenable, so logrotate never runs and logs grow unbounded.Reported by a customer:
/var/log/messagesandmodsec_audit.logreached substantial sizes.Opening investigation suspected
.rpmnew/.rpmsavecron residue, but that was not relevant - the cron file is simply gone from the EL9 package, and the timer that replaced it was never enabled.Root cause
Three facts compound:
logrotateships/etc/cron.daily/logrotateand no timer. EL9 drops the cron file and shipslogrotate.timer, vendor presetenable.logrotateis an RPM upgrade, so its%systemd_postscriptlet — guarded by[ $1 -eq 1 ]— does not apply thetimer's preset. Presets are only applied on a fresh install.
common/libraries/systemd.pyscans with_SYSTEMCTL_CMD_OPTIONS = ['--type=service', ...], andget_system_service_preset_files()emits presets only for units ending in.service. Timers, sockets and paths are structurally invisible to the whole mechanism.Approach
The initial approach idea used a hardcoded list of affected timers.
However, it would (obviously) miss components that we didn't write into it, so was deemed unsuitable to handling various customer machine configurations. This implementation uses a rule instead of an enumeration:
A timer that did not exist on the source cannot have been disabled by the administrator, so applying the target preset is assumed safe - it reproduces what a fresh install would have done.
This is the same reasoning the service-state transition already applies to units new on the target.
Timers that did exist on the source keep whatever state the normal transition gave them, so a deliberate
systemctl disablealways survives the upgrade. If the source inventory message is missing, the actor does nothing and logs a warning, rather than guessing.Test run
Full CL8-CL9 upgrade on a no-panel VM with
mdadminstalled:logrotate.timerraid-check.timermdcheck_start/mdcheck_continue/mdmonitor-oneshotmlocate-updatedb.timerThe third row is of interest: a naive "enable every disabled preset-enabled timer" implementation would have wrongly flipped those three.
The upgrade itself was clean - CL9.8 final state, 3000+ packages, no errors.