Skip to content

feat: run scheduled maintenance endpoints in self-hosted deployments #2299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fancyHeat wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: run scheduled maintenance endpoints in self-hosted deployments #2299

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/test-self-hosting.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,27 @@ jobs:
exit 1
fi

- name: Test scheduled maintenance endpoints
run: |
echo "Waiting for cap-cron..."
timeout 150 bash -c 'until docker inspect cap-cron --format="{{.State.Health.Status}}" 2>/dev/null | grep -q "healthy"; do sleep 2; done'
for route in recover-failed-video-processing finalize-stale-desktop-segments cleanup-agent-api; do
code=$(docker exec cap-cron sh -c 'curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer $CRON_SECRET" "$CAP_WEB_INTERNAL_URL/api/cron/'"$route"'"')
Comment on lines +102 to +103

@greptile-apps greptile-apps Bot Sep 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Scheduler path remains untested

The workflow invokes each endpoint directly with docker exec, bypassing the new scheduler’s timing, route interpolation, and locking logic. A regression that leaves cap-cron healthy but prevents scheduled requests from launching would therefore still pass CI. Exercise at least one request through the scheduler path or make that scheduling logic independently testable.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: .github/workflows/test-self-hosting.yml
Line: 102-103

Comment:
**Scheduler path remains untested**

The workflow invokes each endpoint directly with `docker exec`, bypassing the new scheduler’s timing, route interpolation, and locking logic. A regression that leaves `cap-cron` healthy but prevents scheduled requests from launching would therefore still pass CI. Exercise at least one request through the scheduler path or make that scheduling logic independently testable.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

if [ "$code" = "200" ]; then
echo "✓ /api/cron/$route (HTTP $code)"
else
echo "✗ /api/cron/$route failed (HTTP $code)"
exit 1
fi
done
code=$(docker exec cap-cron sh -c 'curl -s -o /dev/null -w "%{http_code}" -H "Authorization: Bearer wrong" "$CAP_WEB_INTERNAL_URL/api/cron/cleanup-agent-api"')
if [ "$code" = "401" ]; then
echo "✓ Wrong CRON_SECRET is rejected"
else
echo "✗ Wrong CRON_SECRET returned HTTP $code"
exit 1
fi

- name: Test database has tables
run: |
echo "Testing database..."
Expand Down
24 changes: 22 additions & 2 deletions apps/web/content/docs/self-hosting.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Best for VPS, home servers, or any Docker-capable host.
- Media server (FFmpeg processing)
- MySQL database
- MinIO (S3-compatible storage)
- Scheduled maintenance (`cap-cron`)

**Steps:**
1. Clone the repository
Expand Down Expand Up @@ -89,6 +90,23 @@ For Coolify users, use `docker-compose.coolify.yml` which includes environment v

> **Note:** The Coolify compose file uses slightly different environment variable names: `WEB_URL` instead of `CAP_URL`, and `S3_PUBLIC_ENDPOINT` instead of `S3_PUBLIC_URL`.

## Scheduled Maintenance

Cap Web relies on a few scheduled endpoints to recover work that stalls. On Cap.so these run as Vercel Cron Jobs; in the Docker Compose and Coolify deployments the `cap-cron` service calls them on the same UTC schedule:

| Endpoint | Schedule | Purpose |
|---|---|---|
| `/api/cron/recover-failed-video-processing` | :07, :22, :37, :52 | Retries uploads that failed with transient media server errors and restarts processing, transcription and AI generation stalled for over an hour |
| `/api/cron/finalize-stale-desktop-segments` | Every 15 minutes | Finalizes segmented desktop recordings whose upload never completed |
| `/api/cron/cleanup-agent-api` | 03:17 daily | Deletes expired Agent API records |

The endpoints require `Authorization: Bearer $CRON_SECRET`, and the same `CRON_SECRET` must be set on `cap-web` and `cap-cron`. Without the scheduler, videos that hit a transient processing error stay stuck. If you run Cap Web without these compose files, call the endpoints on the schedule above from your own scheduler.

Check it with:
```bash
docker compose logs cap-cron
```

## Connecting Cap Desktop

1. Open Cap Desktop settings
Expand Down Expand Up @@ -194,6 +212,7 @@ The default `docker-compose.yml` contains **hardcoded placeholder secrets** that
- **Forge authentication sessions** (via `NEXTAUTH_SECRET`)
- **Decrypt sensitive database fields** (via `DATABASE_ENCRYPTION_KEY`)
- **Spoof webhook requests** (via `MEDIA_SERVER_WEBHOOK_SECRET`)
- **Trigger maintenance endpoints** (via `CRON_SECRET`)

This is fine for local development or testing on a private network, but **you must generate unique secrets before exposing Cap to the internet**.
</Warning>
Expand All @@ -203,15 +222,16 @@ This is fine for local development or testing on a private network, but **you mu
openssl rand -hex 32
```

Run this command three times to generate values for:
Run this command four times to generate values for:
- `NEXTAUTH_SECRET`
- `DATABASE_ENCRYPTION_KEY`
- `MEDIA_SERVER_WEBHOOK_SECRET`
- `CRON_SECRET`

**Full production checklist:**

- [ ] Set secure passwords: `MYSQL_PASSWORD`, `MINIO_ROOT_PASSWORD`
- [ ] Set secure secrets: `DATABASE_ENCRYPTION_KEY`, `NEXTAUTH_SECRET`, `MEDIA_SERVER_WEBHOOK_SECRET`
- [ ] Set secure secrets: `DATABASE_ENCRYPTION_KEY`, `NEXTAUTH_SECRET`, `MEDIA_SERVER_WEBHOOK_SECRET`, `CRON_SECRET`
- [ ] Set `CAP_URL` to your public URL
- [ ] Set `S3_PUBLIC_URL` to your MinIO/S3 public URL
- [ ] Configure a reverse proxy (nginx, Caddy, Traefik) with SSL
Expand Down
4 changes: 4 additions & 0 deletions docker-compose.coolify.env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ NEXTAUTH_SECRET=
# Generate with: openssl rand -hex 32
MEDIA_SERVER_WEBHOOK_SECRET=

# Secret the cap-cron service uses to call Cap Web's scheduled maintenance endpoints
# Generate with: openssl rand -hex 32
Comment on lines +23 to +24

@greptile-apps greptile-apps Bot Sep 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Routine comments violate policy

These comments only restate the variable’s purpose and its standard generation command. This violates the repository directive to avoid code comments unless they preserve non-obvious context from a bug or complex investigation, so this requirement must be satisfied before merging.

Suggested change
# Secret the cap-cron service uses to call Cap Web's scheduled maintenance endpoints
# Generate with: openssl rand -hex 32

Context Used: AGENTS.md (source)

Prompt To Fix With AI 
This is a comment left during a code review.
Path: docker-compose.coolify.env.example
Line: 23-24

Comment:
**Routine comments violate policy**

These comments only restate the variable’s purpose and its standard generation command. This violates the repository directive to avoid code comments unless they preserve non-obvious context from a bug or complex investigation, so this requirement must be satisfied before merging.

```suggestion

```

**Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md))

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

CRON_SECRET=

# ===================
# DATABASE
# ===================
Expand Down
43 changes: 43 additions & 0 deletions docker-compose.coolify.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ services:
MEDIA_SERVER_URL: 'http://media-server:3456'
MEDIA_SERVER_WEBHOOK_SECRET: '${SERVICE_HEX_32_MEDIASERVER}'
MEDIA_SERVER_WEBHOOK_URL: 'http://cap-web:3000'
CRON_SECRET: '${SERVICE_HEX_32_CRONSECRET}'
healthcheck:
test:
- CMD
Expand Down Expand Up @@ -59,6 +60,48 @@ services:
timeout: 10s
retries: 3
start_period: 10s
cap-cron:
image: 'curlimages/curl:8.22.0'
restart: unless-stopped
depends_on:
cap-web:
condition: service_healthy
environment:
CRON_SECRET: '${SERVICE_HEX_32_CRONSECRET}'
CAP_WEB_INTERNAL_URL: 'http://cap-web:3000'
entrypoint:
- /bin/sh
- '-c'
- |
trap 'exit 0' TERM INT
run() {
mkdir "/tmp/lock.$$1" 2>/dev/null || return 0
code=$$(curl -s -o "/tmp/out.$$1" -w '%{http_code}' --max-time 600 -H "Authorization: Bearer $$CRON_SECRET" "$$CAP_WEB_INTERNAL_URL/api/cron/$$1")
echo "$$(date -u +%FT%TZ) $$1 $$code $$(head -c 500 "/tmp/out.$$1")"
rmdir "/tmp/lock.$$1"
}
rmdir /tmp/lock.* 2>/dev/null
while :; do
jobs >/dev/null
touch /tmp/heartbeat
sleep $$((60 - $$(date +%s) % 60)) &
wait $$!
m=$$(date -u +%M); m=$${m#0}
h=$$(date -u +%H); h=$${h#0}
case $$m in 7|22|37|52) run recover-failed-video-processing & ;; esac
if [ $$((m % 15)) -eq 0 ]; then run finalize-stale-desktop-segments & fi
if [ "$$h" = 3 ] && [ "$$m" = 17 ]; then run cleanup-agent-api & fi
done
healthcheck:
test:
- CMD
- sh
- '-c'
- 'find /tmp/heartbeat -mmin -3 | grep -q .'
interval: 60s
timeout: 5s
retries: 3
start_period: 90s
mysql:
image: 'mysql:8.0'
restart: unless-stopped
Expand Down
43 changes: 43 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ services:
DATABASE_ENCRYPTION_KEY: ${DATABASE_ENCRYPTION_KEY:-0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef}
NEXTAUTH_SECRET: ${NEXTAUTH_SECRET:-abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789}
MEDIA_SERVER_WEBHOOK_SECRET: ${MEDIA_SERVER_WEBHOOK_SECRET:-fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210}
CRON_SECRET: ${CRON_SECRET:-0f1e2d3c4b5a69780f1e2d3c4b5a69780f1e2d3c4b5a69780f1e2d3c4b5a6978}

@superagent-security superagent-security Bot Sep 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Do not ship a predictable fallback secret for externally reachable maintenance endpoints

A predictable compose fallback authenticates the new recovery and deletion endpoints if operators omit CRON_SECRET.

Require a unique CRON_SECRET; do not provide a production compose fallback for this endpoint-authentication secret.

AI prompt 
Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior.

<file name="docker-compose.yml">
<violation number="1" location="docker-compose.yml:20">
<priority>P2</priority>
<title>Do not ship a predictable fallback secret for externally reachable maintenance endpoints</title>
<evidence>The newly added CRON_SECRET configuration supplies a deterministic built-in fallback instead of requiring an operator-provided secret. The same fallback is passed to cap-cron, so anyone who knows the published compose file can authenticate to the maintenance routes when this default is left unchanged. These routes perform recovery and deletion operations, expanding the impact beyond a normal local-development placeholder.</evidence>
<recommendation>Remove the CRON_SECRET fallback and fail the deployment or disable cap-cron when CRON_SECRET is unset. If a local-development default is required, make it opt-in through a clearly named development override and ensure production compose validation rejects it; keep the production and Coolify paths dependent on a unique generated secret.</recommendation>
</violation>
</file>

CAP_AWS_ACCESS_KEY: ${MINIO_ROOT_USER:-cap-admin}
CAP_AWS_SECRET_KEY: ${MINIO_ROOT_PASSWORD:-cap-minio-pwd-456}
CAP_AWS_BUCKET: cap
Expand Down Expand Up @@ -61,6 +62,48 @@ services:
networks:
- cap-network

cap-cron:
container_name: cap-cron
image: curlimages/curl:8.22.0
restart: unless-stopped
depends_on:
cap-web:
condition: service_healthy
environment:
CRON_SECRET: ${CRON_SECRET:-0f1e2d3c4b5a69780f1e2d3c4b5a69780f1e2d3c4b5a69780f1e2d3c4b5a6978}
CAP_WEB_INTERNAL_URL: http://cap-web:3000
entrypoint:
- /bin/sh
- -c
- |
trap 'exit 0' TERM INT
run() {
mkdir "/tmp/lock.$$1" 2>/dev/null || return 0
code=$$(curl -s -o "/tmp/out.$$1" -w '%{http_code}' --max-time 600 -H "Authorization: Bearer $$CRON_SECRET" "$$CAP_WEB_INTERNAL_URL/api/cron/$$1")
echo "$$(date -u +%FT%TZ) $$1 $$code $$(head -c 500 "/tmp/out.$$1")"
rmdir "/tmp/lock.$$1"
}
rmdir /tmp/lock.* 2>/dev/null
while :; do
jobs >/dev/null
touch /tmp/heartbeat
sleep $$((60 - $$(date +%s) % 60)) &
wait $$!
m=$$(date -u +%M); m=$${m#0}
h=$$(date -u +%H); h=$${h#0}
case $$m in 7|22|37|52) run recover-failed-video-processing & ;; esac
if [ $$((m % 15)) -eq 0 ]; then run finalize-stale-desktop-segments & fi
if [ "$$h" = 3 ] && [ "$$m" = 17 ]; then run cleanup-agent-api & fi
done
healthcheck:
test: ["CMD", "sh", "-c", "find /tmp/heartbeat -mmin -3 | grep -q ."]
interval: 60s
timeout: 5s
retries: 3
start_period: 90s
networks:
- cap-network

mysql:
container_name: cap-mysql
image: mysql:8.0
Expand Down