feat(adapters): Superpowers skill evaluation adapter

.gitignore

@@ -64,3 +64,6 @@ docs/让*
 tests/run_*.sh
 tests/launch_*.py
 *.launch.log
+uv.lock
+# Superpowers smoke runs: raw agent output + local paths, share sanitized excerpts instead
+smoke_results/

docs/superpowers/SECURITY.md

@@ -0,0 +1,82 @@
+# Security Considerations for Superpowers Adapter
+
+## Execution Model
+
+The Superpowers adapter runs Claude Code with candidate skills that control agent behavior. A candidate skill is **untrusted input**: it can
+
+- Execute arbitrary shell commands
+- Read/write files reachable by the process
+- Read environment variables passed to the process
+- Make network requests
+
+`--allowedTools` scopes which tools the agent may call. It is **not** an
+isolation boundary — `Bash` and `Read` are granted, so anything the process can
+reach, the candidate can reach.
+
+## Current Mitigations
+
+1. **No host credential reuse by default.** The scenario `HOME` is empty; host
+   `~/.claude/credentials.json` and `settings.json` are never copied or
+   symlinked. Reuse is opt-in via `SKILLOPT_HOST_AUTH=1`, which warns.
+2. **Fail closed.** With neither `ANTHROPIC_API_KEY` nor `SKILLOPT_HOST_AUTH=1`,
+   the scenario errors (`NO_AUTH`) instead of running unauthenticated.
+3. **Scrubbed environment.** Only `HOME`, `PATH`, `TERM`, `LANG` and (if set)
+   `ANTHROPIC_API_KEY` are passed; the host environment is not inherited. `PATH`
+   is minimal by default (shim dir + `/usr/bin:/bin`); opt in to the host `PATH`
+   with `SKILLOPT_INHERIT_PATH=1`. Note this is convenience/hygiene, not a
+   boundary — a `Bash`-holding agent can still call absolute paths.
+4. **Isolated project and HOME** per scenario, inside a temp workspace.
+5. **OS-level sandbox** (⚠️ **experimental, not yet validated end-to-end**),
+   opt-in via `SKILLOPT_SANDBOX=bwrap|docker`. The `bwrap` path is the intended
+   Linux boundary; both modes are provided as scaffolding and are not exercised
+   in CI. Treat them as a starting point, not a hardened guarantee — details
+   like the in-container interpreter/`claude` binary (`SKILLOPT_CLAUDE_BIN`,
+   `SKILLOPT_SHIM_PYTHON`) may need tuning for your image.
+6. **Execution evidence.** `harness_test_passes` re-runs the tests after the
+   agent exits — this is unforgeable and is the authoritative gate. `pytest_runs`
+   (nonce-tagged invocation log) is tamper-**evident**, not tamper-proof: an
+   unsandboxed agent runs as the same OS user with `Bash` and can reach the log,
+   so treat the count as corroborating unless running under `SKILLOPT_SANDBOX`.
+
+   ⚠️ The verification re-run **executes the (agent-modified) project code**. When
+   `SKILLOPT_SANDBOX` is set, the re-run goes through the same sandbox as the
+   agent, so untrusted code does not run on the host. In the default (no-sandbox)
+   mode it runs on the host — acceptable only for trusted candidates. Never
+   evaluate an untrusted candidate without `SKILLOPT_SANDBOX`.
+
+## Known Limitations
+
+- **API key exposure**: `ANTHROPIC_API_KEY`, if set, is visible to the agent
+  process. Use a scoped/disposable key, or run under `SKILLOPT_SANDBOX=docker`
+  with a key injected per run.
+- **`SKILLOPT_HOST_AUTH=1` exposes host credentials** to the candidate. Trusted
+  candidates only. Combining it with `SKILLOPT_SANDBOX` is refused (the host
+  `~/.claude` is not mounted, so the symlinks would dangle) — use
+  `ANTHROPIC_API_KEY` inside the sandbox instead.
+- **`SKILLOPT_UNSAFE=1`** disables permission checks entirely. Trusted
+  candidates only.
+- **No network isolation** in the default (unsandboxed) path.
+
+## Recommendations
+
+### Trusted candidates (your own skill, local machine)
+```bash
+ANTHROPIC_API_KEY=... python -m skillopt_sleep.adapters.superpowers --candidate my_skill.md
+```
+
+### Untrusted or model-generated candidates
+```bash
+# Linux
+SKILLOPT_SANDBOX=bwrap ANTHROPIC_API_KEY=... \
+  python -m skillopt_sleep.adapters.superpowers --candidate untrusted.md
+
+# Container
+SKILLOPT_SANDBOX=docker SKILLOPT_SANDBOX_IMAGE=skillopt-sandbox ANTHROPIC_API_KEY=... \
+  python -m skillopt_sleep.adapters.superpowers --candidate untrusted.md
+```
+
+## Follow-up Work
+
+- [ ] Published sandbox image with Claude Code + pytest preinstalled
+- [ ] Network egress allowlist (api.anthropic.com only)
+- [ ] Per-run scoped API keys

scripts/smoke_superpowers.sh

@@ -0,0 +1,90 @@
+#!/bin/bash
+# Smoke test for Superpowers adapter integration.
+# Run this manually (not in CI) to verify the adapter works with the real harness.
+#
+# Prerequisites:
+# - Claude Code installed, ANTHROPIC_API_KEY set (or SKILLOPT_HOST_AUTH=1 for a
+#   trusted candidate on your own machine)
+# - Same model/settings/pinned SHA for baseline and candidate runs
+#
+# Usage:
+#   ./scripts/smoke_superpowers.sh [candidate_skill_path]
+#
+# Output goes to smoke_results/ (gitignored). Results embed raw agent output and
+# local paths: do NOT commit them. Sanitized excerpts are written alongside each
+# run for pasting into a PR description or attaching as an artifact.
+
+set -euo pipefail
+
+SKILL="${1:-}"
+SCENARIO="${SKILLOPT_SCENARIO:-test-passes-verify}"
+SHA="${SKILLOPT_SHA:-d884ae04edebef577e82ff7c4e143debd0bbec99}"
+OUTDIR="smoke_results/$(date +%Y%m%d_%H%M%S)"
+mkdir -p "$OUTDIR"
+
+echo "Smoke test: Superpowers adapter"
+echo "Output:   $OUTDIR (gitignored - do not commit)"
+echo "Scenario: $SCENARIO"
+echo "SHA:      $SHA"
+echo ""
+
+run_scenario() {
+    local name="$1"
+    local candidate="${2:-}"
+    local outfile="$OUTDIR/${name}.json"
+
+    echo "=== $name ==="
+
+    local args=(
+        --skill verification-before-completion
+        --scenario "$SCENARIO"
+        --sha "$SHA"
+        --json
+    )
+    if [[ -n "$candidate" ]]; then
+        args+=(--candidate "$candidate")
+    fi
+
+    # No || true - fail if runner errors
+    python -m skillopt_sleep.adapters.superpowers "${args[@]}" > "$outfile"
+
+    # Best-effort sanitized summary for sharing: home dir, temp workspace paths
+    # and api-key-shaped tokens are redacted. Skim before sharing - it is a
+    # heuristic scrub, not a guarantee.
+    python - "$outfile" "$OUTDIR/${name}.summary.txt" <<'PY'
+import json, os, re, sys
+data = json.load(open(sys.argv[1]))
+home = os.path.expanduser("~")
+def clean(t):
+    t = t.replace(home, "~")
+    t = re.sub(r"/tmp/\S*skillopt\S*", "<workspace>", t)
+    t = re.sub(r"/(?:tmp|var)/\S*", "<path>", t)
+    return re.sub(r"sk-[A-Za-z0-9_\-]{8,}", "sk-REDACTED", t)
+lines = [
+    f"skill={data['skill']} version={data['version']} pinned_sha={data['pinned_sha']}",
+    f"candidate_hash={data['candidate_hash'] or '(baseline)'}",
+    f"score={data['score']:.2f} passed={data['passed']} failed={data['failed']}",
+]
+for s in data["scenarios"]:
+    lines.append(f"\n[{s['id']}] passed={s['passed']} error={s.get('error') or 'none'}")
+    lines.append(f"  evidence: {json.dumps(s.get('evidence', {}))}")
+    for c in s["checks"]:
+        lines.append(f"  {'PASS' if c['passed'] else 'FAIL'} {c['description']}")
+    lines.append("  output excerpt:")
+    for ln in clean(s.get("output", ""))[:800].splitlines():
+        lines.append(f"    {ln}")
+text = "\n".join(lines)
+open(sys.argv[2], "w").write(text + "\n")
+print(text)
+PY
+}
+
+run_scenario "baseline"
+
+if [[ -n "$SKILL" ]]; then
+    run_scenario "candidate" "$SKILL"
+fi
+
+echo ""
+echo "Results in $OUTDIR (gitignored)."
+echo "Share the *.summary.txt excerpts in the PR; do not commit the raw JSON."

skillopt_sleep/adapters/__init__.py

@@ -0,0 +1 @@
+"""SkillOpt adapters for external skill frameworks."""