Skip to content

Standardize agent research: auto-research.md, auto-research/ layout, and a mirror of research/ - #928

Closed
rohanarun wants to merge 6 commits into
CrocSwap:mainfrom
rohanarun:auto-research-standard
Closed

rohanarun wants to merge 6 commits into
CrocSwap:mainfrom
rohanarun:auto-research-standard

Conversation

@rohanarun

@rohanarun rohanarun commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

This PR proposes a standard auto-research/ folder that can be reused across auto-research repos, plus a root-level instruction file for agents. No mathematical claim or κ change.

  1. auto-research.md (root): the full community agent prompt, followed by short instructions for submitting all research to auto-research/. Work goes into an existing research track as auto-research/<track>/<YYYY-MM-DD>-<handle>-<slug>/, or into a new track if none fits.
  2. auto-research/README.md: the package standard. One directory per submission, auto-research/<YYYY-MM-DD>-<handle>-<slug>/, containing README.md, RESULT.json, verify.py/verify.sh, MANIFEST.json and NOTICE.md, with a RESULT.json schema.
  3. A byte-identical mirror of research/ under auto-research/: all 209 entries (20,119 files). Git stores the copies as the same blobs, so the repository barely grows. The diff is large only because every path appears twice.

What it deliberately does not change

research/ itself is untouched; its tree hash 66af7031… equals current main. More than 1,600 files (verifiers, manifests, hash pins and 52 Makefile lines) reference research/... paths, so moving the folder would break existing certificates. Existing verifiers keep running from research/. New packages go directly into auto-research/. Retiring the old path can be a separate, reviewed change once pins are updated.

Checks

  • git rev-parse HEAD:research equals main's, so research/ is unchanged.
  • auto-research/ contains every tracked research/ file, including force-tracked ones, plus README.md: 20,120 files.
  • No verifier, certificate, Makefile or upstream/ file is modified.

Prepared by Rohan Arun with Claude Opus 5.5 (Anthropic).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TGoMCHAqctZe4wTb4Qr7VV

rohanarun and others added 6 commits October 13, 2026 12:59
…andard

Proposes auto-research/ as the standard home for new agent research packages
across auto-research repos, keeps research/ frozen (1,600+ files pin its
paths), and adds root agent instructions pointing to the community prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TGoMCHAqctZe4wTb4Qr7VV
…unchanged)

All 209 research/ entries (20,119 files) are mirrored under auto-research/ as the
same git blobs. research/ is untouched because 1,600+ files pin its paths;
existing verifiers keep running from research/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TGoMCHAqctZe4wTb4Qr7VV
…r create a new one)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TGoMCHAqctZe4wTb4Qr7VV
…nstructions only

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TGoMCHAqctZe4wTb4Qr7VV
@antoine-olivier

Copy link
Copy Markdown

@rohanarun, it's Antoine, from the call on October 12. I'd like to contribute to the cross-repository part of this PR, and connect it with autoresearch.md, the guide @Mister-Meeseeks maintains on main. I can prepare a first patch on verification attestations. If we develop this format together, I would like each person's contributions to be explicitly credited in the document.

What I would add:

1. Verification attestations, separate from the result. status says what the result is, and verification is one of its values; the schema has no place to record who checked what. I propose separate attestations for execution, independent reproduction and review of the statement, each with its author, the revision checked, the exact command, its outcome (passed, failed, refuted) and its logs. A review never says "valid", only what was read. An attestation is a new contribution that points to the package, without editing its author's files. A verify.py that prints PASS can check the wrong property, which is why the statement review is separate. The replays in #214 could serve as a first concrete case.

2. Other people's work as input, not as permissions. The donation prompt lets the agent post and open PRs without asking again, and Rohan's app loads the guide verbatim. Instructions found in packages and discussions do not change the agent's permissions. Reproduction commands are read before they run, in an isolated environment enforced by the launcher, with no network and no credentials, and with dependencies prepared in advance and pinned by hash. The package also records the repository, path, commit and hash of the guide that was actually loaded.

3. Structured coverage. Search coverage is already required, as free text. Describing it in fields (domain, assumptions, part covered, linked evidence) could later feed find_overlap.py; a declaration alone does not prove that a domain is exhausted.

4. A responsible contact. In addition to the attribution already required, a responsible field, the account of the person who answers for the package, and the model version when it is known.

Would you rather have the first patch target auto-research/README.md in this PR, or a separate document linked from both guides?

@Mister-Meeseeks

Copy link
Copy Markdown
Contributor

100-PR batch review at d2daf3e927a845419e6dcc525e6f47bc7ce6682e.

This remains open after scoped triage. Next review step: Separate workflow proposal review: current AGENTS.md/autoresearch.md/package guidance now cover the contribution workflow. Avoid duplicating 20,119 historical files or introducing conflicting entry points.

This is a specific continuation point, not a rejection or a request to redo already recorded checks. Completed local evidence is retained separately from outstanding whole-head obligations.

Batch review, attribution and evidence. Thank you for the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants