showcase: add Kleos Research - #110
Conversation
Kleos Research builds the layer between agents and the models they run on: 0xCopilot, a local-first desktop harness that runs on your own key or a Virtuals ACP key, and Kaleidoscope, filesystem-native memory for agents. The manifest declares `acp` and `token`. Both are earned: the harness routes inference through compute.virtuals.io on an ACP key, discovering the model catalogue live so per-Mtok pricing is visible before a task is spent rather than after; and $CPILOT is tokenised through Virtuals (listing 113720). Proof is inspectable rather than promotional. The headline artifact is the harness benchmark, measured on the packaged app against a live model and scored from the same records the product bills from. It reports a real win — an inherited step ceiling was terminating live work, and raising it took task completion from 3-of-4 to 4-of-4 for +0.1% tokens — and, in the same file, the correction of an earlier version of itself that had declared that finding falsified using a metric blind to the failure it was measuring. Kaleidoscope's numbers come from held-out results published as a PDF: a 100,000-memory scan in 3.04 ms with no index, 99.7% of retrieval quality retained at 32 bytes per memory, and 96% of the recall of a model four times its size from 7.2 MB. No comparison against another memory system is quoted. One looked significant and was withdrawn after an aggregation fault was found — one side had been scored across nine abilities and the other across ten — and the corrected interval crosses zero. It is left out because it is not settled. Validated with `node scripts/validate-showcase.mjs`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
This is a fantastic submission — the level of detail in the README and the transparent, honest framing of the results (including the retractions) is exactly what we want to see. Really well done. Just one small thing to align with the showcase conventions: the (Totally optional nit: The tagline and description are great but a little long for the card format. No need to change, but if you see an easy way to shorten them, it might help them land a bit better.) Also flagged:
Refreshed after a new push. Any inline suggestions from the first pass may now be stale. |
- topics now draw from tags already in use across the showcase (acp, compute, virtuals, token, robinhood-chain, developer-tools, privacy) rather than package-specific coinages. `memory` is kept as the one addition: it is the primary category of half this submission and no existing tag covers it. - links.share points at the 0xCopilot launch post rather than the profile. - adds skills/0xcopilot-local-workspace — bring up a local agent workspace on Virtuals compute, with the credential entry marked as a human-only handoff and the CLI's lack of a batch-run verb stated under "when NOT to use this". - names the BEAM 100K tier and the open-source suite that measures it, and adds the CLI package and the memory-recall announcement as artifacts. No Kaleidoscope skill: the evaluation suite is open source but needs the `kscope` binary on PATH, and Kaleidoscope is not yet published. A skill whose first step no reader can perform is worse than no skill. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Thank you — pushed in TopicsAligned. I took the replacements from what is actually in use across the other
Mapping from the flagged values: I kept One small note in case it is useful for the guide rather than for this PR: the Share linkNow the 0xCopilot launch post rather than the profile: The memory-recall paper announcement is added as an artifact alongside it, since A skill, after allYou did not ask for one, but the template pushes for it and the earlier "n/a" It is deliberately a workspace skill, not a task-runner. The CLI is a launcher There is deliberately no Kaleidoscope skill. The evaluation suite is open BEAMThe manifest and README now name the measurement — the BEAM 100K tier, ten |
1 participant
Showcase Project
What shipped
kleos-researchacp,tokenbenchmark findings, the Virtuals compute integration source, the held-out
memory results (PDF), the evaluation suite, and the live
$CPILOTlisting.All linked from the manifest and reachable today.
Why
acpandtoken. The harness routes inference throughcompute.virtuals.ioon a Virtuals ACP key, discovering the model cataloguelive so per-Mtok pricing is visible before a task is spent rather than after;
the integration source is linked as an artifact.
$CPILOTis tokenised throughVirtuals (listing 113720).
One thing worth flagging for the reviewer: this is not an ACP commerce
offering with buyer/seller job round-trips, as several neighbouring packages
are. It is a local-first harness that consumes Virtuals compute, plus the memory
layer underneath it. If
acpreads as overclaiming for that shape, say so andI will drop it to
tokenalone.Project package
showcase/kleos-research/showcase.jsonskillsis[]skills/skills[].sourcePath, per the abovehiddenis not set — this package is meant to publishsoul.mdis not linkedSkill standard
Skill path: none. This submission is a project package without a reusable
skill, which the field reference permits (
skillsmay be empty). The remainingcheckboxes in this section do not apply.
Safety and redaction
prompts, wallet material, or private account records are published
a real run producing a document artifact; it contains no credentials, no
account data, and no private workspace contents
is public and was checked to resolve before submitting
soul.mdA note on the claims
Two claims a reviewer might expect here are deliberately absent.
No comparison against another memory system. One looked significant and was
withdrawn: an aggregation fault had scored one side across nine abilities and
the other across ten, and the corrected interval crosses zero. A later harness
fix also showed the underlying measurement was non-uniformly corrupting
between-arm comparisons. It is left out because it is not settled, not because
it is unflattering.
No cost-per-task reduction. The harness benchmark shows 97% of tokens are
cache reads and concludes that prompt-trimming pays only in proportion to
cold-start frequency, so a blanket "cheaper per task" claim would contradict our
own published data. What is claimed is that cost is metered and visible, and
that task completion improved measurably — 3-of-4 to 4-of-4 for +0.1% tokens
after raising a step ceiling that had been terminating live work.
The findings file linked as the headline artifact publishes its own retractions
next to its results, including the correction of an earlier version of itself.
Validation
visual.posterUrlpoints atraw.githubusercontent.com/.../showcase/kleos-research/assets/hero-card.png,which resolves once this merges — matching the convention used by the existing
packages.