Docs: update AGENTS, CONTRIBUTING, and the templates

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,5 +1,5 @@
 # Servicename: context
-<!-- Examples: Dockhand: new service | Changedetection: fix port mapping -->
+<!-- Title format: "Service: what changes", as in CONTRIBUTING.md. Examples: Dockhand: new service | Changedetection: fix port mapping -->
 
 ## Description
 <!-- Briefly describe what changed. -->
@@ -17,7 +17,9 @@
 - [ ] I have performed a self-review of my code and followed the [templates](https://github.com/tailscale-dev/ScaleTail/tree/main/templates/service-template) structure.
 - [ ] I have added verification that the stack works as expected.
 - [ ] I have updated necessary documentation (e.g. frontpage [README.md](https://github.com/tailscale-dev/ScaleTail/blob/main/README.md) ).
-- [ ] I have selected the correct label(s) for this PR.
+- [ ] I ran Compose validation with dummy values for required variables.
+- [ ] I ran rumdl on the whole repository.
+- [ ] I committed no working credential.
 
 ## Additional Context
 <!-- Any extra info for reviewers, such as gotchas, special requirements, devices, or dependencies. Use "None" if not applicable. -->

AGENTS.md

@@ -1,26 +1,58 @@
 # Agent guidance
 
-Follow [CONTRIBUTING.md](CONTRIBUTING.md) and the current service template.
+ScaleTail is a collection of Docker Compose stacks. Each directory in `services/` is one stack, which usually runs a self-hosted application behind a Tailscale sidecar container.
+
+## Where the rules live
+
+- [CONTRIBUTING.md](CONTRIBUTING.md): how to add, update, verify, and submit a service. Follow it.
+- [templates/service-template/](templates/service-template/): the starting point for every service. Its comments explain each setting.
+- [documentation/standard-setup.md](documentation/standard-setup.md): the shared setup. A service README lists only what differs, under "Deviations from the standard setup".
+- [.github/workflows/](.github/workflows/): the Compose validation and Markdown lint that CI runs.
 
 ## Before you edit
 
-- Inspect the target service and its README.
-- Confirm ports, commands, volumes, and variables in the image's upstream docs.
-- Preserve service-specific exceptions that still apply.
+- Read the service's `compose.yaml`, `.env`, and README, including "Deviations from the standard setup".
+- Confirm ports, commands, volumes, variables, and health checks in the image's upstream docs.
+- Preserve documented exceptions. Some stacks use other Compose service keys or container names, and other containers or `.env` values refer to those names.
 - Keep changes focused on the requested service or documentation.
-- Never commit working auth keys, passwords, tokens, or other credentials.
+
+## Files that change together
+
+- Add, rename, or remove a service: the `services/<name>/` directory and its row in the root `README.md`. Insert a new row in alphabetical order by the bold name, and leave the other rows alone.
+- Make existing users act, such as with a new required secret, a moved volume, or a changed `SERVICE`: add an "Upgrading" section to the service README. `SERVICE` also names the Tailnet device, the containers, and most data folders.
+- Change the template: also update `documentation/standard-setup.md` and `CONTRIBUTING.md` where they describe the same setting. Validate it from `templates/service-template/` with `SERVICE=dummy IMAGE_URL=dummy docker compose config --quiet`, because the template leaves both values empty.
+
+## Secrets and local data
+
+- Every `.env` file is tracked and published. Never commit an auth key, password, token, or other credential.
+- Leave each secret empty in `.env`. When the stack cannot start without it, read it in `compose.yaml` as `${VAR:?Set VAR in .env}`, and add a `# Required:` comment above it in `.env` that says what to set, such as how to generate it. Keep `TS_AUTHKEY` optional, because only the first start needs it.
+- Start test stacks from a copy outside the repository, such as `cp -R services/<name> /tmp/<name>`, and put real values only in the copy. A running stack writes `./config`, `./ts/state` (with the device key), and its data folders next to `compose.yaml`.
+- Before you commit, run `git status` and stage only the files you meant to change.
 
 ## Verification
 
-Run `docker compose config --quiet` from each changed service directory. When
-possible, start the stack and test its main function through the Tailnet.
+From each changed service directory, validate Compose. The command gives each required variable a dummy value, as CI does, so you do not edit `.env`:
+
+```sh
+env $(grep -ohE '\$\{[A-Za-z0-9_]+:\?' compose.yaml .env | sed -E 's/^\$\{//; s/:\?$/=dummy/' | sort -u) docker compose config --quiet
+```
 
-From the repository root, lint the changed Markdown files. Replace the example paths below as needed:
+It must exit with status 0. CI fails on "variable is not set" warnings.
+
+From the repository root, lint all Markdown with the rumdl version that `.github/workflows/linting.yml` pins, currently 0.2.78, and check your changes for whitespace errors:
 
 ```sh
-rumdl check --config .markdownlint.yml AGENTS.md CONTRIBUTING.md
-git diff --check
+uvx rumdl@0.2.78 check --config .markdownlint.yml .
+git diff --check origin/main
 ```
 
-Report the checks you ran. State any checks that credentials, hardware,
-permissions, or unavailable tools prevented.
+Lint the whole tree, because renaming a service breaks a link in the root `README.md`. `git diff --check origin/main` also covers staged and committed changes, but not untracked files, so stage new files first.
+
+When you have an auth key, start the copied stack and test its main function through the Tailnet. Run each health check command in its container with `docker exec`.
+
+Report the checks you ran. State any checks that credentials, hardware, permissions, or unavailable tools prevented.
+
+## Pull requests
+
+- Title the pull request `Service: what changes`, such as `Umami: new service`. For several services, list them or use `All services:`. For other areas, use `CI:`, `Docs:`, or `Template:`.
+- Fill in the [pull request template](.github/PULL_REQUEST_TEMPLATE.md). Maintainers apply the labels.

CONTRIBUTING.md

@@ -12,33 +12,63 @@ Thanks for helping improve these Tailscale sidecar examples.
 
    This command includes the hidden `.env` file. Use a lowercase directory name.
 
-2. Update `.env` with safe example values.
+2. Update `.env`.
 
-   Set `SERVICE`, `IMAGE_URL`, `SERVICEPORT`, and the application variables.
-   Never commit a working auth key, password, token, or other credential.
+   Set `SERVICE`, usually to the directory name, and set `IMAGE_URL`,
+   `SERVICEPORT`, and the application variables. Define every variable that
+   `compose.yaml` uses, unless it has a default such as `${VAR:-value}`. Put
+   comments on their own line: `VAR= # note` sets `VAR` to `# note`.
+
+   `.env` is published with the repository. Never commit a working auth key,
+   password, token, or other credential. Leave each secret empty. When the stack
+   cannot start without it, require it in `compose.yaml`:
+
+   ```yaml
+   - DB_PASSWORD=${DB_PASSWORD:?Set DB_PASSWORD in .env}
+   ```
+
+   Compose then stops with an error until the user sets a value. Above the
+   empty variable in `.env`, add a comment that starts with `# Required:` and
+   says what to set, such as
+   `# Required: signs the sessions. Generate it with: openssl rand -hex 32`.
+   Keep `TS_AUTHKEY` and optional secrets without `:?`.
 
 3. Adapt `compose.yaml`.
 
    - Keep the Compose service keys `tailscale` and `application`.
-   - Name the containers `tailscale-${SERVICE}` and `app-${SERVICE}`.
+   - Name the containers `tailscale-${SERVICE}` and `app-${SERVICE}`, unless the
+     stack documents a deviation under "Deviations from the standard setup".
    - Keep `network_mode: service:tailscale` on the application.
    - Keep the application's health-based dependency on `tailscale`.
    - Add all required persistent volumes.
    - Add required devices and capabilities explicitly.
+   - When the stack needs another Compose service key or container name, more
+     than one application container, or none, list the difference and the
+     reason under "Deviations from the standard setup". Prefer
+     `app-${SERVICE}-<role>` for extra containers, such as `app-${SERVICE}-db`.
+   - Pass `PUID` and `PGID` only when the image reads them, such as LinuxServer.io
+     images. Keep `TZ`, as the template does.
+   - Write the image tag explicitly and use the tag upstream recommends. Pin
+     databases, caches, and search engines to a major or major.minor tag. When
+     you pin an exact version, say why in a comment.
+   - Keep application data under `./${SERVICE}-data/`, so that `.gitignore`
+     covers it.
 
 4. Set the Serve proxy to the application's internal port.
 
-   The Serve JSON does not read `SERVICEPORT` from `.env`. Keep runtime variables
-   escaped, such as `$${TS_CERT_DOMAIN}`.
+   Hard-code the port in the Serve JSON. Do not use `${SERVICEPORT}`: it is
+   meant for the optional LAN `ports` block and can differ from the port that
+   Serve must proxy to. Keep runtime variables escaped, such as
+   `$${TS_CERT_DOMAIN}`.
 
    Keep the `ports` block commented for Tailnet-only access. Document any LAN
-   port you expose. Remove the Serve configuration when the service does not use
-   Tailscale Serve.
+   port you expose. When nothing in the stack uses Tailscale Serve, remove the
+   Serve configuration: the `TS_SERVE_CONFIG` line, the `configs` entry of the
+   `tailscale` service, and the top-level `configs` block.
 
 5. Configure the application's health check.
 
-   Use the first option that the image supports. The template lists the same
-   options in the same order.
+   Use the first option that the image supports.
 
    1. When the image defines its own `HEALTHCHECK`, omit the block and add a
       `# Healthcheck: defined by the image (...)` comment that names the
@@ -54,42 +84,37 @@ Thanks for helping improve these Tailscale sidecar examples.
       is possible. Omit the block and add a `# Healthcheck: none possible ...`
       comment that names the reason.
 
-   Run the command inside the running container before you commit it. Avoid
-   `pgrep -f ${SERVICE}`, which breaks when `SERVICE` is renamed.
-
-   A container that runs once and exits needs no check. Omit the block and
-   add a `# Healthcheck: none needed ...` comment that names the reason.
-
-   For a database, connect over TCP, for example with
-   `pg_isready -h 127.0.0.1` or `mariadb-admin ping -h 127.0.0.1`. During the
-   first start, the image runs a temporary server that accepts only socket
-   connections, so a socket check reports ready too early. When another
-   service depends on the database, wait with `condition: service_healthy`.
-   A database image's own `HEALTHCHECK` still comes first. When that check
-   connects over the socket, say so in the comment.
-
-   Keep the Tailscale health check.
+   Use the comment text from the template's `compose.yaml`, which covers the
+   same options, the database case, and the no-check cases. Run the command
+   inside the running container before you commit it. Keep the Tailscale health
+   check.
 
 6. Complete the service README.
 
    Every service README uses the headings of the template, in the same order.
-   Do not rename them and do not add others, so that every service reads the
-   same way. Replace the placeholders in capitals.
+   Do not rename them, and add no headings other than the optional sections
+   below, so that every service reads the same way. Replace the placeholders in
+   capitals.
 
-   - **Introduction.** Say what the service does in one or two sentences and
-     link to the upstream project. Leave feature lists to upstream.
+   - **Introduction.** The paragraph under the title. Say what the service does
+     in one or two sentences and link to the upstream project. Leave feature
+     lists to upstream.
    - **At a glance.** Give the Tailnet address, the port that Tailscale Serve
      forwards to, the image, and the data paths on the host. Add a row for
-     each further port that users connect to, such as DNS or SMTP.
+     each further port that users connect to, such as DNS or SMTP. Write `None`
+     for the web interface when there is none, and add an `API` row when Serve
+     publishes an API.
    - **Before you start.** List only what the Quick Start does not cover:
      values that must change in `.env`, secrets to generate, folders to
-     create, and required host groups or devices.
+     create, and required host groups or devices. List every required secret
+     and say that Compose stops with an error until it is set.
    - **Deviations from the standard setup.** List every difference from
      [the standard setup](documentation/standard-setup.md) and give the
      reason. Examples are extra containers, published host ports, a changed
      or removed Serve configuration, DNS settings, and added capabilities.
    - **First run.** Describe what the user does after the first start, such
-     as creating the first account or finding a generated password.
+     as creating the first account or finding a generated password. When there
+     is nothing to do, write `Nothing to set up. Open the web interface.`
    - **Links.** Link to the upstream documentation and source code.
 
    When a section has nothing to report, keep the heading and the sentence
@@ -101,15 +126,17 @@ Thanks for helping improve these Tailscale sidecar examples.
 
    - **Configuration.** Optional settings that users commonly change.
    - **Troubleshooting.** Known errors and their solutions.
-   - **Upgrading.** Steps for users of an older version of the stack.
+   - **Upgrading.** Required when a change makes existing users act, such as a
+     new required secret, a moved data folder, or a changed `SERVICE`. Say what
+     to do, as in [FossFLOW](services/fossflow/README.md#upgrading).
 
    State only what you confirmed in the Compose file, the upstream
    documentation, or a running stack. Put guidance that applies to more than
    one service in `documentation/` and link to it.
 
 7. Add the service to the correct category in the root `README.md`.
 
-   Keep the entries in that category alphabetized.
+   Insert the row in alphabetical order of the bold name, ignoring case.
 
 ## Update a service
 
@@ -119,20 +146,48 @@ Thanks for helping improve these Tailscale sidecar examples.
 - Use `${VARIABLE}` for Compose interpolation, not `$(VARIABLE)`.
 - Update the service README when ports, paths, setup, or behavior change.
 - Update the root service list when you add, remove, or rename a service.
+- Changing `SERVICE` renames the Tailnet device and its address, the
+  containers, and the `./${SERVICE}-data` folder. Treat it like a volume change
+  and add an "Upgrading" section.
+- Do not rename existing containers or service keys. Other containers and
+  `.env` values may refer to them.
 
 Preserve valid service-specific exceptions.
 
 ## Verify your change
 
-Run Compose validation from each changed service directory:
+Run Compose validation from each changed service directory. Stacks that
+require your own secrets stop with "required variable ... is missing a value".
+Give those variables a dummy value on the command line, as CI does, instead of
+editing `.env`:
 
 ```sh
-docker compose config --quiet
+env $(grep -ohE '\$\{[A-Za-z0-9_]+:\?' compose.yaml .env | sed -E 's/^\$\{//; s/:\?$/=dummy/' | sort -u) docker compose config --quiet
 ```
 
-This command does not prove that the application works.
+The command must exit with status 0 and add no "variable is not set"
+warnings. It does not prove that the application works.
+
+The template leaves `SERVICE` and `IMAGE_URL` empty. To validate it, run
+`SERVICE=dummy IMAGE_URL=dummy docker compose config --quiet` from
+`templates/service-template/`.
+
+From the repository root, lint all Markdown with the rumdl version that
+`.github/workflows/linting.yml` pins, currently 0.2.78:
+
+```sh
+uvx rumdl@0.2.78 check --config .markdownlint.yml .
+```
+
+After you stage new files, check whitespace with `git diff --check origin/main`.
+
+When possible, start the stack from a copy outside the repository, such as
+`cp -R services/my-service /tmp/my-service`, and put real values only in the
+copy. A running stack writes `./config`, `./ts/state` (with the Tailscale
+device key), and its data folders next to `compose.yaml`. Before you commit,
+run `git status` and stage only the files you changed on purpose.
 
-When possible, start the stack and confirm:
+Then confirm:
 
 - Tailscale becomes healthy and joins the Tailnet.
 - The application starts and is reachable through the Tailnet.
@@ -141,5 +196,8 @@ When possible, start the stack and confirm:
 
 ## Submit a pull request
 
-Follow the pull request template. Report the checks you ran and any checks you
-could not run.
+Title the pull request `Service: what changes`, such as `Umami: new service`.
+For several services, list them or use `All services:`. For other areas, use
+`CI:`, `Docs:`, or `Template:`. Fill in the pull request template, and report
+the checks you ran and any checks you could not run. Maintainers apply the
+labels.

templates/service-template/.env

@@ -1,6 +1,5 @@
-#version=1.1
 #URL=https://github.com/tailscale-dev/ScaleTail
-#COMPOSE_PROJECT_NAME= # Optional: only use when running multiple deployments on the same infrastructure.
+# To run a second copy, copy the folder under another name and change SERVICE.
 
 # Service Configuration
 # Service name (e.g., adguard). Used as hostname in Tailscale and for container naming (app-${SERVICE}).
@@ -11,17 +10,18 @@ IMAGE_URL=
 # Network Configuration
 # Port to expose to local network. Uncomment the "ports:" section in compose.yaml to enable.
 SERVICEPORT=
-DNS_SERVER=9.9.9.9 # Preferred DNS server for Tailscale. Uncomment the "dns:" section in compose.yaml to enable.
+# Preferred DNS server for Tailscale. Uncomment the "dns:" section in compose.yaml to enable.
+DNS_SERVER=9.9.9.9
 
 # Tailscale Configuration
-# Auth key from https://tailscale.com/admin/authkeys. See: https://tailscale.com/kb/1085/auth-keys#generate-an-auth-key for instructions.
+# Auth key from https://console.tailscale.com/admin/settings/keys. See: https://tailscale.com/kb/1085/auth-keys#generate-an-auth-key for instructions.
 TS_AUTHKEY=
 
 # Optional Service variables
 # PUID=1000
 
-# Time Zone setting for containers
-TZ=Europe/Amsterdam # See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
+# Time Zone setting for containers. See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
+TZ=Europe/Amsterdam
 
 # Any Container environment variables are declared below. See https://docs.docker.com/compose/how-tos/environment-variables/