Research Agent Guardrails

[sahilds1]

Title

Research Agent Guardrails

Background

Keep the agent safe in production

Existing Behavior

server/api/views/assistant/sanitizer.py:

https://github.com/CodeForPhilly/balancer-main/blob/develop/server/api/views/assistant/sanitizer.py

437 Build an input sanitizer

Acceptance Criteria

• []

Approach

References

OpenAI Cookbook Guardrails:
https://developers.openai.com/cookbook/topic/guardrails

OpenAI Developer Documentation Safety:
https://developers.openai.com/api/docs/guides/safety-best-practices

OpenAI Moderation API:
https://developers.openai.com/api/docs/guides/moderation

Open AI Privacy Filter Model:
https://openai.com/index/introducing-openai-privacy-filter/

Open AI Guardrails Python Package:
https://github.com/openai/openai-guardrails-python

Risks and Rollback

Screenshots / Recordings

[mahirhir]

For structuring the agent guardrails and sanitizer pipeline in server/api/views/assistant/sanitizer.py:

1. Layered Sanitization Pipeline:

   • Stage 1 (O(1) Structural Injection Pre-filter):
     Strip instruction-override tags and block common prompt-injection syntax before context assembly:

     import re
     from typing import Tuple
     
     INJECTION_PATTERNS = [
         r"(?i)\bignore\s+(all\s+)?(previous|prior)\s+instructions\b",
         r"(?i)\bsystem\s*:\s*override\b",
         r"(?i)\byou\s+are\s+now\s+in\s+developer\s+mode\b"
     ]
     
     def prefilter_user_input(text: str) -> Tuple[bool, str]:
         for pattern in INJECTION_PATTERNS:
             if re.search(pattern, text):
                 return False, "Input flagged by injection guardrail."
         return True, text

   • Stage 2 (PII Redaction):
     Replace emails, phone numbers, and identifying tokens with deterministic placeholders ([REDACTED_EMAIL], [REDACTED_PHONE]) before storing or sending to the LLM.
   • Stage 3 (Async Moderation Gate):
     Evaluate openai.moderations.create in parallel with vector retrieval, aborting immediately if safety thresholds are violated.

2. Pre-flight Tool Execution Validation:

   • Validate tool invocation arguments against strict Pydantic models before executing backend database queries or calculations.

We use this layered data-sanitization and schema-enforced tool receipt pattern in https://github.com/TraceFold/tracefold (architecture in docs/TRACEFOLD_TR.md §4.1) for production agent safety.