feat: alert before a token the jobs use expires

README.md

@@ -16,6 +16,7 @@ share for translations. One package, `session_ops`, deployed to one self-hosted
 | `snode-list` | Weekday: the fallback service node list into session-ios | [snode-list](docs/jobs/snode-list.md) |
 | `release-stats` | On demand: download counts of the latest releases | [release-stats](docs/jobs/release-stats.md) |
 | `session-ops-silence` | Discord alerts for a job that failed, or stopped running | [session-ops-silence](docs/jobs/session-ops-silence.md) |
+| `token-expiry` | Discord alerts 14 days, 7 days and 24 hours before a token expires | [token-expiry](docs/jobs/token-expiry.md) |
 
 Run by hand, not scheduled: [`sogs-ban`](docs/tools/sogs-ban.md) for community bans, and
 [`sogs-perms`](src/session_ops/sogs/perms.py) for a room's per-account permissions.

deploy/README.md

@@ -56,6 +56,8 @@ files and state over. Remove `/opt/zendesk`, `/etc/zendesk` and `/var/lib/zendes
 
 Each `/etc/session-ops/<name>.env` has a commented `<name>.env.example` beside it,
 installed from [`env/`](env/), saying what goes in it.
+`/etc/session-ops/expiry.toml` is not a secret: the expiry dates
+[token-expiry](../docs/jobs/token-expiry.md) cannot read from an API, from `expiry.toml.example`.
 
 ## Checking
 

deploy/env/expiry.toml.example

@@ -0,0 +1,14 @@
+# /etc/session-ops/expiry.toml: what token-expiry cannot learn by itself. Not a
+# secret: mode 644.
+
+[expires]
+# Crowdin → Account Settings → API → Personal Access Tokens, the Expires column.
+# A date, or "never" when it does not expire or this host does not use it.
+#CROWDIN_API_TOKEN = "never"
+
+# Only for a Claude Code token installed before token-expiry first ran: it otherwise
+# dates a token from the day it first saw it. Ignored once that token is replaced.
+# The fingerprint is printed by `session-ops run token-expiry --dry-run`.
+#[issued.CLAUDE_CODE_OAUTH_TOKEN]
+#fingerprint = "3f2a9c01b4de"
+#date = 2025-11-20

deploy/install.sh

@@ -73,7 +73,7 @@ for name in zendesk github-prs crowdin publish alerts; do
 done
 # What each file takes, commented; the env files stay empty until filled, since a job
 # is enabled once its env files have content.
-install -m 644 "$ROOT"/deploy/env/*.env.example "$ETC/"
+install -m 644 "$ROOT"/deploy/env/*.example "$ETC/"
 # The publishing units load this as a credential, and a missing file would stop them
 # starting; left empty, their runs exit naming the key.
 [ -e "$ETC/github-app.pem" ] || install -m 600 /dev/null "$ETC/github-app.pem"

docs/jobs/token-expiry.md

@@ -0,0 +1,30 @@
+# Token Expiry Alerts
+
+Posts to the alerts channel 14 days, 7 days and 24 hours before a token the jobs use
+expires, and once when it has. Quiet otherwise. Times are UTC; a date without one, such
+as Crowdin's, counts from 00:00 UTC that day.
+
+| | |
+| --- | --- |
+| Runs | `session-ops@token-expiry.timer`, daily at 09:00 Melbourne |
+| Secrets | `/etc/session-ops/alerts.env`: `ALERT_DISCORD_WEBHOOK_URL`; `github-prs.env` and `zendesk.env` for the tokens it checks |
+| Dates | `/etc/session-ops/expiry.toml`, from [`expiry.toml.example`](../../deploy/env/expiry.toml.example) |
+| Dry run | `session-ops run token-expiry --dry-run` prints each token's expiry and the alert it would post |
+| Logs | `journalctl -u session-ops@token-expiry -n 50 --no-pager` |
+
+| Token | Expiry from |
+| --- | --- |
+| `GITHUB_PRS_TOKEN` | GitHub's `github-authentication-token-expiration` header; nothing to update on rotation |
+| `CROWDIN_API_TOKEN` | `expiry.toml`: the Expires column at https://crowdin.com/settings#api-key |
+| `CLAUDE_CODE_OAUTH_TOKEN` | A year after the job first saw it, by fingerprint; nothing to update on rotation |
+
+`CROWDIN_API_TOKEN` is reported on every daily run until `expiry.toml` has a date or
+`"never"` for it.
+
+The Claude Code token's first sighting lives in `/var/lib/session-ops/token-expiry/state.json`.
+For a token installed before the job first ran, or if that file is lost, add its real
+issue date under `[issued]` with the fingerprint the dry run prints; the entry is
+ignored once the token changes. A date that is wrong is not caught: the Zendesk
+digest's failure alert, which names a dead Claude login, is then the backstop.
+
+The Zendesk API token, the Discord webhooks and the GitHub App key do not expire.

pyproject.toml

@@ -33,6 +33,7 @@ sogs-perms = "session_ops.sogs.perms:cli"
 session-ops = "session_ops.ops.runner:main"
 session-ops-alert = "session_ops.monitor.alert:main"
 session-ops-silence = "session_ops.monitor.silence:main"
+session-ops-token-expiry = "session_ops.monitor.token_expiry:main"
 
 [dependency-groups]
 dev = [

src/session_ops/jobs.toml

@@ -84,6 +84,22 @@ schedule = "hourly"
 max_age_hours = 3
 timeout = "5min"
 
+[[job]]
+name = "token-expiry"
+description = "Alert before a token the jobs use expires"
+entry = "session_ops.monitor.token_expiry:main"
+args = ["--state", "{state}/state.json"]
+user = "sessionops"
+# For the GitHub token it probes and the Claude Code token it fingerprints, which brings
+# the rest of zendesk.env's secrets into this unit. expiry.toml holds none.
+env_files = ["/etc/session-ops/alerts.env", "/etc/session-ops/github-prs.env",
+             "/etc/session-ops/zendesk.env"]
+env = ["ALERT_DISCORD_WEBHOOK_URL"]
+schedule = "*-*-* 09:00 Australia/Melbourne"
+# A day, plus the hour a DST change adds and slack for the run.
+max_age_hours = 27
+timeout = "5min"
+
 [[job]]
 name = "crowdin-sync"
 description = "Crowdin translations into the platform repos"