Use literal seconds for lifecycle alert threshold defaults - #2778
Conversation
The threshold x-inputs used arithmetic defaults (10*60, 24*60*60, ...).
Arithmetic is valid inside the PromQL expressions, but the same values
are interpolated into the Go-template description annotations of the
LifecycleLateScan and LifecycleStalledScan alerts, and Go templates
have no '*' operator: when the consumer leaves an input unset, the
rendered annotation ('{{ 10*60 | humanizeDuration }}') fails to parse
with 'unexpected "*" in operand', prometheus-operator rejects the
rule group and aborts the whole Prometheus config sync, silently
disabling all alerting and any scrape config added afterwards.
The alerts CI never renders the defaults (the workflow supplies its
own threshold values), so this only surfaces in consumers that leave
inputs unset, e.g. Zenko with zenko-operator v1.8.13 which binds only
the latency thresholds.
Issue: BB-824
Hello delthas,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
@@ Coverage Diff @@
## development/9.5 #2778 +/- ##
===================================================
- Coverage 75.47% 75.40% -0.08%
===================================================
Files 201 201
Lines 13917 13917
===================================================
- Hits 10504 10494 -10
- Misses 3403 3413 +10
Partials 10 10
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
|
|
/approve |
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
The following options are set: approve |
|
I have successfully merged the changeset of this pull request
The following branches have NOT changed:
This pull request did not target the following hotfix branch(es) so they
Please check the status of the associated issue BB-824. Goodbye delthas. The following options are set: approve |
The lifecycle alert threshold
x-inputsused arithmetic defaults (10*60,24*60*60, ...). Arithmetic is valid inside the PromQL expressions, but the same values are interpolated into the Go-templatedescriptionannotations of theLifecycleLateScanandLifecycleStalledScanalerts, and Go templates have no*operator: when the consumer leaves an input unset, the rendered annotation ({{ 10*60 | humanizeDuration }}) fails to parse withunexpected "*" in operand. prometheus-operator then rejects the rule group and aborts the entire Prometheus config sync — all alerting silently dies, and any ServiceMonitor reconciled afterwards is never scraped.This is the root cause of the deterministic
end2end-pra(PRA nominal case) and Kafka Cleaner CTST failures on every Zenko branch bumping backbeat >= 9.5.0-preview.2 (scality/Zenko#2461, scality/Zenko#2447, scality/Zenko#2451): zenko-operator v1.8.13 only binds the latency thresholds, so the conductor-scan inputs introduced by BB-740 fall back to their raw defaults.The alerts CI workflow does not catch this because it always supplies its own threshold values instead of rendering the file defaults.
Using literal seconds keeps the PromQL semantics identical and renders valid annotations (
{{ 600 | humanizeDuration }}→ "10 minutes").Issue: BB-824