Inkdown
Start writing

Study

70 filesยท12 subfolders

Shared Workspace

Study
AI eng

05_GUARDRAILS

Shared from "Study" on Inkdown

Guardrails - Comprehensive Deep Dive

Overview

Guardrails are safety checks that validate input and output at various points in an agent run. Think of guardrails as "security checkpoints" or "quality gates" that ensure the agent is operating within safe and acceptable boundaries. They can prevent harmful content, validate data formats, enforce business rules, and provide custom validation logic.

Core Concepts

What are Guardrails?

Guardrails are functions that run at specific points during agent execution to:

  • Validate - Check that data meets certain criteria
  • Filter - Remove or modify inappropriate content
  • Block - Prevent execution when safety thresholds are crossed
  • Transform - Modify content to meet requirements
Types of Guardrails
  1. Input Guardrails - Run before the agent processes input
basic-ques
core
Revision w/ Whiteboard
CN Basics - 1
CN Basics - 2
DNS
Event loop
programming-language-concepts.md
zero-language-explanation.md
DB
Quick
databases-deep-dive.md
01-introduction.md
02-relational-databases.md
03-database-design.md
04-indexing.md
05-transactions-acid.md
06-nosql-databases.md
07-query-optimization.md
08-replication-ha.md
09-sharding-partitioning.md
10-caching-strategies.md
11-cap-theorem.md
12-connection-pooling.md
13-backup-recovery.md
14-monitoring.md
15-database-selection.md
README.md
JS
core topics
Event loop
Merlin Backend
01-Orchestration.md
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft
Mobile
Build Alternative
Bundling
metro-bundler-deep-dive.md
OpenAI Agents Python
00_OVERVIEW.md
01_AGENT_SYSTEM.md
02_RUNNER_SYSTEM.md
03_TOOL_SYSTEM.md
04_ITEMS_SYSTEM.md
05_GUARDRAILS.md
06_HANDOFFS.md
07_MEMORY_SESSIONS.md
08_MODEL_PROVIDERS.md
09_SANDBOX_SYSTEM.md
10_TRACING.md
11_RUN_STATE.md
12_CONTEXT.md
13_LIFECYCLE_HOOKS.md
14_CONFIGURATION.md
15_ERROR_HANDLING.md
16_STREAMING.md
17_EXTENSIONS.md
18_MCP_INTEGRATION.md
19_BEST_PRACTICES.md
20_ARCHITECTURE_PATTERNS.md
opencode-study
context-handling
core
Python
Alembic
Basics
sqlalchemy - fastapi
SQLAlchemy overview
tweets
system_design_for_agentic_apps.md
Agent Loop
  • Output Guardrails - Run after the agent produces output
  • Tool Input Guardrails - Run before tool execution
  • Tool Output Guardrails - Run after tool execution
    • Guardrail Anatomy

    Every guardrail has:

    Python

    Tripwire Concept:

    • When tripwire_triggered = False - Execution continues normally
    • When tripwire_triggered = True - Execution is halted with an exception
    • The output_info can contain details about what was checked

    Input Guardrails

    Purpose

    Input guardrails validate or filter input before it reaches the LLM. They run:

    • Only on the first turn of a run
    • Only for the starting agent (not for handoffs)
    • Optionally in parallel with the agent (default) or before the agent starts
    Basic Input Guardrail
    Python
    Input Guardrail with Structured Input
    Python
    Parallel vs Sequential Execution

    Input guardrails can run in parallel (default) or sequentially:

    Python

    When to use parallel:

    • Independent checks that don't depend on each other
    • Faster execution
    • No order dependency

    When to use sequential:

    • Guardrails depend on previous guardrail results
    • Order matters (e.g., check A must pass before check B)
    • Need to short-circuit on first failure
    Async Input Guardrails

    Guardrails can be async:

    Python
    Input Guardrail Parameters
    Python

    Parameters:

    • name - Custom name (defaults to function name)
    • run_in_parallel - Whether to run in parallel (default: True)

    Output Guardrails

    Purpose

    Output guardrails validate the agent's final output before it's returned. They run:

    • After the agent produces a final output
    • For every agent that produces output
    • Before the result is returned to the user
    Basic Output Guardrail
    Python
    Structured Output Validation
    Python
    Content Filtering
    Python
    Business Rule Validation
    Python

    Tool Input Guardrails

    Purpose

    Tool input guardrails validate arguments before a tool is executed. They run:

    • After the model calls a tool
    • Before the tool is actually executed
    • For each tool call individually
    Basic Tool Input Guardrail
    Python
    Security Validation
    Python
    Schema Validation
    Python

    Tool Output Guardrails

    Purpose

    Tool output guardrails validate the results after tool execution. They run:

    • After the tool completes
    • Before the result is sent back to the model
    • For each tool call individually
    Basic Tool Output Guardrail
    Python
    Output Sanitization
    Python
    Format Validation
    Python

    Guardrail Execution Flow

    Input Guardrail Flow
    Python
    Output Guardrail Flow
    Python
    Tool Guardrail Flow
    Python

    Guardrail Exceptions

    InputGuardrailTripwireTriggered

    Raised when an input guardrail tripwire is triggered:

    Python
    OutputGuardrailTripwireTriggered

    Raised when an output guardrail tripwire is triggered:

    Python
    ToolInputGuardrailTripwireTriggered

    Raised when a tool input guardrail tripwire is triggered:

    Python
    ToolOutputGuardrailTripwireTriggered

    Raised when a tool output guardrail tripwire is triggered:

    Python

    Guardrail Configuration

    Agent-Level Guardrails

    Guardrails can be set on individual agents:

    Python
    Run-Level Guardrails

    Guardrails can be set for an entire run:

    Python
    Guardrail Priority

    Guardrails are applied in this priority:

    1. Tool-level guardrails (for tool input/output)
    2. Agent-level guardrails
    3. Run-level guardrails

    All guardrails at the same level run (unless configured otherwise).

    Guardrail Best Practices

    1. Clear Tripwire Conditions

    Make it clear when a guardrail should trigger:

    Python
    2. Descriptive Output Info

    Provide helpful information in output_info:

    Python
    3. Use Parallel When Possible

    For independent checks, use parallel execution:

    Python
    4. Handle Errors Gracefully

    Guardrails should handle their own errors:

    Python
    5. Keep Guardrails Focused

    Each guardrail should have a single responsibility:

    Python

    Common Guardrail Patterns

    1. Content Moderation
    Python
    2. PII Detection
    Python
    3. Format Validation
    Python
    4. Business Rule Enforcement
    Python
    5. Resource Limits
    Python

    Guardrail and Tracing

    Guardrail Spans

    Guardrails create trace spans for observability:

    Python
    Guardrail Metadata

    Guardrail results are included in traces:

    Python

    Summary

    Guardrails provide safety and validation for agent runs. Key takeaways:

    1. Input guardrails validate input before agent processing
    2. Output guardrails validate output after agent processing
    3. Tool input guardrails validate tool arguments
    4. Tool output guardrails validate tool results
    5. Tripwires halt execution when triggered
    6. Parallel execution improves performance for independent checks
    7. Sequential execution for dependent checks
    8. GuardrailFunctionOutput is the standard return type
    9. Exceptions indicate guardrail violations
    10. Agent-level guardrails apply to specific agents
    11. Run-level guardrails apply to entire runs
    12. Priority determines which guardrails apply
    13. Async guardrails support external API calls
    14. Structured validation for complex data
    15. Content filtering for safety
    16. Business rules for domain validation
    17. Resource limits prevent abuse
    18. Error handling should be graceful
    19. Tracing includes guardrail results
    20. Single responsibility keeps guardrails focused

    Guardrails are essential for building safe, reliable, and compliant agent systems.