Inkdown
Start writing

Study

70 filesยท12 subfolders

Shared Workspace

Study
AI eng

08_MODEL_PROVIDERS

Shared from "Study" on Inkdown

Model Providers - Comprehensive Deep Dive

Overview

Model Providers are the abstraction layer that connects the OpenAI Agents SDK to various Large Language Model (LLM) APIs. Think of Model Providers as "translators" or "adapters" - they translate the SDK's standardized requests into the specific format required by different LLM providers (OpenAI, Anthropic, Google, etc.).

Core Concepts

What is a Model Provider?

A Model Provider is responsible for:

  • Resolving model names to concrete Model instances
  • Managing model connections and resources
  • Providing a consistent interface across different LLM APIs
  • Handling provider-specific features and quirks
Why Model Providers Matter
  1. Provider Agnostic - Switch between LLM providers without changing agent code
  2. Flexibility - Use the best model for each task
basic-ques
core
Revision w/ Whiteboard
CN Basics - 1
CN Basics - 2
DNS
Event loop
programming-language-concepts.md
zero-language-explanation.md
DB
Quick
databases-deep-dive.md
01-introduction.md
02-relational-databases.md
03-database-design.md
04-indexing.md
05-transactions-acid.md
06-nosql-databases.md
07-query-optimization.md
08-replication-ha.md
09-sharding-partitioning.md
10-caching-strategies.md
11-cap-theorem.md
12-connection-pooling.md
13-backup-recovery.md
14-monitoring.md
15-database-selection.md
README.md
JS
core topics
Event loop
Merlin Backend
01-Orchestration.md
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft
Mobile
Build Alternative
Bundling
metro-bundler-deep-dive.md
OpenAI Agents Python
00_OVERVIEW.md
01_AGENT_SYSTEM.md
02_RUNNER_SYSTEM.md
03_TOOL_SYSTEM.md
04_ITEMS_SYSTEM.md
05_GUARDRAILS.md
06_HANDOFFS.md
07_MEMORY_SESSIONS.md
08_MODEL_PROVIDERS.md
09_SANDBOX_SYSTEM.md
10_TRACING.md
11_RUN_STATE.md
12_CONTEXT.md
13_LIFECYCLE_HOOKS.md
14_CONFIGURATION.md
15_ERROR_HANDLING.md
16_STREAMING.md
17_EXTENSIONS.md
18_MCP_INTEGRATION.md
19_BEST_PRACTICES.md
20_ARCHITECTURE_PATTERNS.md
opencode-study
context-handling
core
Python
Alembic
Basics
sqlalchemy - fastapi
SQLAlchemy overview
tweets
system_design_for_agentic_apps.md
Agent Loop
  • Cost Optimization - Use cheaper models for simple tasks
  • Redundancy - Fallback to alternative providers
  • Experimentation - Easy A/B testing of different models

    • Provider Architecture

    Model Interface

    The base Model interface defines what all models must implement:

    Python
    ModelProvider Interface

    The ModelProvider interface defines how models are resolved:

    Python

    OpenAI Provider

    OpenAIProvider

    The default provider for OpenAI models:

    Python
    OpenAI Chat Completions Model

    Uses the OpenAI Chat Completions API:

    Python

    When to use:

    • You want to use the Chat Completions API
    • You need compatibility with older OpenAI integrations
    • You want more control over the API
    OpenAI Responses Model

    Uses the newer OpenAI Responses API:

    Python

    When to use:

    • You want the latest OpenAI features
    • You need better tool support
    • You want server-managed conversations
    • You need prompt caching
    OpenAI Responses WebSocket Model

    Uses WebSocket transport for Responses API:

    Python

    When to use:

    • You want real-time streaming
    • You need lower latency
    • You're building real-time applications
    Default Model Selection

    The SDK has a default model:

    Python
    GPT-5 Special Handling

    GPT-5 models require special reasoning settings:

    Python

    The SDK automatically adjusts settings when you specify a GPT-5 model.

    MultiProvider

    Using Multiple Providers

    MultiProvider allows using multiple model providers:

    Python
    Provider Priority

    Providers are tried in order:

    Python
    Model Name Resolution

    Different providers use different model names:

    Python

    Custom Model Providers

    Creating a Custom Provider

    Implement the ModelProvider interface:

    Python
    Creating a Custom Model

    Implement the Model interface:

    Python
    Using Custom Provider
    Python

    Model Settings

    ModelSettings Class

    Configure model-specific parameters:

    Python
    Temperature

    Controls randomness:

    Python

    Use cases:

    • Low (0.0-0.3): Factual responses, code generation
    • Medium (0.4-0.7): General conversation
    • High (0.8-1.5): Creative writing, brainstorming
    Max Tokens

    Limit response length:

    Python
    Penalties

    Control repetition:

    Python

    Model Tracing

    ModelTracing Enum

    Controls tracing behavior:

    Python
    Tracing in Model Calls
    Python

    Model Retry

    Retry Advice

    Models can provide retry advice:

    Python
    Retry Policies

    Configure retry behavior:

    Python

    Provider-Specific Features

    OpenAI Features

    Server-Managed Conversations:

    Python

    Prompt Caching:

    Python

    Reasoning Models:

    Python
    Anthropic Features (via LiteLLM)

    Using Anthropic through LiteLLM:

    Python
    Google Features (via LiteLLM)

    Using Google through LiteLLM:

    Python

    Model Selection Strategies

    Per-Agent Model Selection

    Different agents can use different models:

    Python
    Run-Level Model Override

    Override model for a specific run:

    Python
    Dynamic Model Selection

    Select model based on context:

    Python

    Model Configuration

    Global Default Configuration

    Set global defaults:

    Python
    Agent-Level Configuration

    Configure model on agent:

    Python
    Run-Level Configuration

    Configure model for a run:

    Python
    Configuration Priority

    Priority (highest to lowest):

    1. RunConfig.model
    2. Agent.model
    3. Global default

    Model Best Practices

    1. Choose Appropriate Models

    Select the right model for the task:

    Python
    2. Configure Temperature Appropriately

    Set temperature based on task:

    Python
    3. Set Reasonable Token Limits

    Control response length:

    Python
    4. Handle Model Failures

    Implement fallback logic:

    Python
    5. Monitor Model Usage

    Track model usage and costs:

    Python

    Common Patterns

    1. Tiered Model Usage

    Use different models for different complexity:

    Python
    2. Model A/B Testing

    Test different models:

    Python
    3. Cost Optimization

    Use cheaper models when possible:

    Python
    4. Model-Specific Prompts

    Adjust prompts per model:

    Python
    5. Provider Redundancy

    Ensure availability with multiple providers:

    Python

    Model and Tracing

    Model-Level Tracing

    Models can emit traces:

    Python
    Sensitive Data Handling

    Control what's traced:

    Python

    Summary

    Model Providers enable flexible LLM integration. Key takeaways:

    1. Model Providers abstract LLM API differences
    2. Model interface defines the contract for all models
    3. ModelProvider interface defines model resolution
    4. OpenAIProvider is the default provider
    5. OpenAIChatCompletionsModel uses the Chat Completions API
    6. OpenAIResponsesModel uses the newer Responses API
    7. OpenAIResponsesWSModel uses WebSocket transport
    8. MultiProvider enables using multiple providers
    9. Custom providers can integrate any LLM API
    10. ModelSettings configures model parameters
    11. Temperature controls randomness
    12. Max tokens limits response length
    13. Penalties control repetition
    14. ModelTracing controls observability
    15. Retry advice provides error recovery hints
    16. Provider-specific features like server-managed conversations
    17. LiteLLM enables using 100+ LLM providers
    18. Per-agent model selection for different tasks
    19. Run-level overrides for specific runs
    20. Configuration priority determines which settings apply

    Model Providers are essential for building flexible, cost-effective, and resilient agent systems.