Crate forward2#249
Closed
nikomatsakis wants to merge 17 commits into
Closed
Conversation
Rewrite user-facing docs and design docs to reflect the proposed crate-forward plugin model: installed crates, dependency allow lists, auto-install, workspace as plugin source, and new predicates. Co-authored-by: Claude <claude@anthropic.com>
Completed:
- Phase 3 source acquisition: path/git/crate registry resolver and
Cargo dependency-table probing.
- Phase 3.5 source graph API: installed/workspace provenance, dedupe,
workspace members.
- Phase 4 discovery model: synthesized root manifests, default
skill/plugin declarations, optional names, default where.crates =
["*"], shared where parsing, [[plugins]] source.path/git/crate
declarations, nested independent manifests, fixture migration to
skills/.
Verification:
- cargo fmt --check
- cargo test passed fully.
Deviation audit:
- Phase 3 per-source freshness metadata remains deferred; existing git/
Cargo cache behavior covers current callers, and repeated graph
resolution is not wired until later phases.
- Phase 3.5 dependency provenance is represented but automatic
dependency expansion is deferred to Phase 7 as planned.
- Phase 4 parses/retains git and crate plugin-source declarations but
only resolves path declarations, matching the Phase 7 deferral.
- workspace() is represented as a built-in predicate now; full source-
context semantics remain for Phase 5.
Co-authored-by: Codex codex@openai.com (codex@openai.com)
Phase 5: Add `dependency()` and `installed()` predicates alongside `workspace()`. All three evaluate against a per-plugin provenance set (`BTreeSet<SourceProvenance>`) on `PredicateContext`, updated before each plugin's predicates are evaluated. Thread provenance through all call sites: skills, hooks, MCP servers, subcommands. Phase 6: Wire sync to use `ResolvedSourceGraph`. New `resolve_sync_sources` combines installed sources (best-effort), workspace root/members, and legacy `[[plugin-source]]` entries. `load_registry_from_graph` scans each node, stamps provenance, and deduplicates manifests by canonical path (unioning provenance when the same SYMPOSIUM.toml is reached from multiple roots). Key design decisions: - Workspace root only added when it has an explicit SYMPOSIUM.toml (avoids unbounded recursive search through project tree) - `agents-syncing = false` disables workspace source scanning - Installed crate resolution is best-effort (warns and skips) - Hooks/help/subcommands remain on legacy `load_registry` for now Co-authored-by: Claude <claude@anthropic.com>
Add specificity-based allow/deny policy evaluation for workspace dependency auto-discovery, and iterative graph expansion that follows recursive [[plugins]] source.crate/source.git declarations. Key changes: - New `discovery.rs` module with CollectedPolicy engine (3-tier specificity: global wildcard < registry wildcard < specific crate) - `expand_source_graph()` in crate_sources iterates until convergence: collects policy from resolved plugins, evaluates workspace dep candidates, follows recursive source declarations - Plugin manifests can declare [discovery.allow]/[discovery.deny] - Sync calls expand_source_graph after initial graph construction Tests: 10 unit tests for policy logic, 3 integration tests proving discovery-allow installs, recursive source.crate resolves, default deny blocks, and workspace() predicate gates recursive provenance. Co-authored-by: Claude <claude@anthropic.com>
- Cycle termination: A source.crate→B, B source.crate→A doesn't loop and both skills install - Chained expansion: discovery allows mid-crate, mid-crate's [[plugins]] source.crate resolves leaf-crate, leaf skill installs - Provenance growth: plugin-x is both installed and discovered as dep, dependency()-gated skill installs because provenance is unioned Co-authored-by: Claude <claude@anthropic.com>
Replace the round-based iteration (which re-scanned all nodes each pass) with a simple worklist: push specs to resolve, pop and process one at a time. A scanned set tracks (path, provenance) so we only re-process a node when new provenance arrives. Key properties: - Each node is scanned at most once per provenance combination - Cycle termination: A→B→A resolves B, sees A already scanned, stops - Provenance propagation: when discovery adds Dependency to an already- Installed node, children are re-pushed with the grown provenance - Policy growth: when a newly-scanned plugin contributes allow rules, workspace dep candidates are immediately re-evaluated - Safety limit of 1000 iterations (far above any real graph) No behavioral changes — all 6 expansion integration tests pass unchanged. Co-authored-by: Claude <claude@anthropic.com>
Plugin hooks and subcommands can now reference crate binary targets by name without an explicit [[installations]] entry. After validating the manifest, `merge_implicit_installations` reads `Cargo.toml` from the manifest's directory and adds: - One Installation per [[bin]] target (or the inferred default target) - A `crate` alias pointing to the package-name binary Explicit installations always take precedence — implicit entries are only added for names not already declared. Tests: parse_binary_targets (explicit + inferred), implicit merge with `crate` alias, explicit-takes-precedence. Co-authored-by: Claude <claude@anthropic.com>
New read-only command that shows the resolved plugin/skill state without installing anything. Resolves the same source graph as sync (installed + workspace + legacy + discovery + recursive), evaluates predicates, and reports: - Source nodes with provenance (installed/workspace/dependency) - Active/inactive plugins and skill groups - Summary (source count, plugin count, skill count, agent count) - Configured agents Supports --json for structured output via the existing report layer. Tests: integration test proving status does NOT create skill dirs. Co-authored-by: Claude <claude@anthropic.com>
Addresses discrepancies identified in plan review: Discovery policy (Phase 7): - Add 6 precedence tests covering all tier combinations: tier 0 vs tier 1 (registry beats global in both directions), tier 1 vs tier 2 (specific beats registry wildcard in both), and multi-source same-specificity (deny wins regardless of order) Implicit installations (Phase 8): - Move implicit binary injection into validate_manifest() so hooks and subcommands can reference them during validation (was a bug: post-validation merge meant hooks couldn't find implicit names) - Add test: hook command = "helper-bin" resolves against [[bin]] target - Add test: subcommand command = "crate" resolves to default binary - Add implicit-binary0 fixture for potential integration use Co-authored-by: Claude <claude@anthropic.com>
… rejection, status tests 1. Reject where/crates/predicates on [[plugins]] source.git/source.crate with a clear error (predicate-gated recursive sources are a planned future extension). Path sources still support predicates. 2. Extract ResolvedSourceGraph::build_initial() as the shared entry point for both sync and status, eliminating the duplicated build_source_graph in status.rs. 3. Add status JSON schema test: verifies status_source, status_plugin, status_summary events are present with correct shapes. 4. Add status inactive plugin test: verifies reporting works with workspace-noserde0 fixture. Co-authored-by: Claude <claude@anthropic.com>
help_render, discovery_hint (hooks), and dispatch_external now use ResolvedSourceGraph::build_initial + expand_source_graph + load_registry_from_graph instead of the legacy load_registry(sym). This means they see the same source model as sync/status: installed sources, workspace members, discovery-allowed deps, and recursive [[plugins]] source.* declarations — with full provenance. help_text and render_help are now async (the binary main and testlib are already async). register_hooks (called from init, no workspace) remains on the legacy path until Phase 10 removes it. Co-authored-by: Claude <claude@anthropic.com>
The crate registry resolver returned paths from cargo metadata without canonicalizing, which could cause the same directory (reached via a symlink) to appear as separate graph nodes when also installed through the path registry. Add canonicalization in resolve_crate and a test that exercises the symlink case. Also includes Phase 10 completion: remove legacy [[plugin-source]], [defaults], source="crate" mechanisms; switch help/hooks/subcommand dispatch to graph-based registry; add predicate-gated recursive source expansion; migrate all fixtures and docs to the new config model. Co-authored-by: Claude <claude@anthropic.com>
Replace the internal is_default flag with a user-facing warn.if_empty field on [[skills]] groups. Defaults to true (explicit groups warn when empty), but the implicit default groups (skills/, .agents/skills/) set it to false. Plugin authors can also set warn.if_empty = false on their own explicit skill groups to suppress the warning. Co-authored-by: Claude <claude@anthropic.com>
On macOS, /var is a symlink to /private/var, so canonicalize() returns a different path than tempfile provides. Update test assertions to compare against canonicalized paths, matching the contract that resolve_crate now guarantees. Co-authored-by: Claude <claude@anthropic.com>
Rename the CLI commands (`cargo agents install` → `cargo agents use`, `cargo agents uninstall` → `cargo agents remove`), the config section (`[installed]` → `[used]`), and the `installed()` predicate to `used()` for a consistent mental model throughout. Co-authored-by: Claude <claude@anthropic.com>
…tion Implement directory-scoped plugin sources so `cargo agents use` scopes entries to the current workspace by default (opt out with `--global`). Unify config and manifest plugin source resolution into a single fixed-point loop that handles custom-predicate-gated entries correctly. Key changes: - New `directory()` predicate with exact and `/**` prefix forms - Config schema migrated from `[used]` to `[[plugins]]` array-of-tables (legacy format silently upgraded on read) - `--global` flag on `cargo agents use`; default is directory-scoped - `expand_source_graph()` tracks custom predicate definitions and defers entries gated on unknown customs, retrying when definitions appear - Fixes latent bug where custom predicates on [[plugins]] source declarations were silently skipped Co-authored-by: Claude <claude@anthropic.com>
Member
Author
|
This PR is too big. Going to revisit it with the #250 RFD process |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Registry-ready plugin source system
Replace the git-repository-based plugin source model with a registry-based system where
path,git, andcrateregistries each resolve plugin source declarations into concrete source trees for Symposium to scan.What changed
New config model — Plugin sources are now installed via
[installed.crates](Cargo dependency syntax),installed.paths, andinstalled.git, replacing the old[[plugin-source]]and[defaults]config. The default install issymposium-recommendations = "1"as a crate, not a git source.Install/uninstall commands —
cargo agents install <CRATE>[@<VERSION>],--path,--gitandcargo agents uninstalllet users manage installed sources from the CLI.Source graph with provenance — A
ResolvedSourceGraphtracks how each source was reached (installed, workspace, dependency) with non-exclusive provenance flags.workspace(),dependency(), andinstalled()predicates evaluate against these flags, enabling skills that only activate for crate developers vs. downstream users.Discovery policy — Plugins can declare
[discovery.allow]/[discovery.deny]rules to curate which workspace dependencies are auto-discovered as plugin sources. Specificity-based matching: specific beats wildcard, deny wins ties.Recursive plugin sources —
[[plugins]] source.crate,source.git, andsource.pathdeclarations are followed transitively with cycle detection and provenance propagation.Implicit binary installations — Crate binary targets from
Cargo.tomlare automatically available as installations, referenced by name in hooks/subcommands with no explicit[[installations]]block needed.Status command —
cargo agents statusexposes the full resolved state (installed sources, active/inactive plugins, predicate explanations, configured agents) without mutating skill directories.Unified registry path — Help rendering, hook dispatch, subcommand dispatch, and MCP registration all use the same graph-based resolution as sync.
Legacy removal —
[[plugin-source]],[defaults],source = "crate"skill groups,[package.metadata.symposium], and the old provider commands are removed with schema-level rejection.Smaller improvements
warn.if_emptyfield on[[skills]]groups (defaults suppress the "no SKILL.md" warning; explicit groups warn; authors can opt out)symposium-sdkcrate withWorkspaceDepscaching andSymposiumDirsfor shared path resolutionTest plan
cargo agents plugin validateruns clean against migratedsymposium-recommendationscargo clippy --all --workspaceclean (pre-existing warnings only)cargo agents syncwith localsymposium-recommendationspath source resolves and installs skills correctlyDisclosure questions
AI disclosure.
Questions for reviewers.