From d37824fcf1e351a05cc480414f92eccb749345d8 Mon Sep 17 00:00:00 2001 From: Jeff Otterson Date: Fri, 24 Jul 2026 15:27:14 -0600 Subject: [PATCH] Add Sabot to LLM & MCP Testing Sabot injects one controlled fault into a running multi-agent pipeline and scores whether that pipeline's own reviewer and guardrail stages detect it, rather than whether an external harness can. Apache-2.0 harness, pre-registered specification, and the full raw trace corpus are published so the scores are recomputable offline. --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 6b24efd..d53b0c3 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,7 @@ Collection of awesome Python resources for testing and generating test data. ## LLM & MCP Testing - [mcp-server-fuzzer](https://github.com/Agent-Hellboy/mcp-server-fuzzer) - A comprehensive fuzzing tool designed specifically for testing Model Context Protocol (MCP) servers. It supports both tool argument fuzzing and protocol type fuzzing across multiple transport protocols. +- [Sabot](https://github.com/Jott2121/sabot) - Plants controlled faults inside running multi-agent pipelines (LangGraph, CrewAI, AutoGen) and scores whether each pipeline's own reviewer and guardrail stages detect them. Pre-registered spec, deterministic adjudication, published raw traces. - [Tenro](https://github.com/tenro-ai/tenro-python) - An open-source, provider-agnostic testing framework for AI agents that integrates with pytest. It simulates LLM and tool calls to test edge cases, failure paths, and agent logic without live API calls. ## Load Testing