Inkdown
Start writing

Study

70 filesยท12 subfolders

Shared Workspace

Study
AI eng

05-Streaming

Shared from "Study" on Inkdown

Streaming Architecture

Overview

The streaming system delivers real-time Server-Sent Events (SSE) to clients, enabling live updates as AI processes requests, executes tools, and generates responses.


SSE (Server-Sent Events) Protocol

Plain text

SSE Format:

basic-ques
core
Revision w/ Whiteboard
CN Basics - 1
CN Basics - 2
DNS
Event loop
programming-language-concepts.md
zero-language-explanation.md
DB
Quick
databases-deep-dive.md
01-introduction.md
02-relational-databases.md
03-database-design.md
04-indexing.md
05-transactions-acid.md
06-nosql-databases.md
07-query-optimization.md
08-replication-ha.md
09-sharding-partitioning.md
10-caching-strategies.md
11-cap-theorem.md
12-connection-pooling.md
13-backup-recovery.md
14-monitoring.md
15-database-selection.md
README.md
JS
core topics
Event loop
Merlin Backend
01-Orchestration.md
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft
Mobile
Build Alternative
Bundling
metro-bundler-deep-dive.md
OpenAI Agents Python
00_OVERVIEW.md
01_AGENT_SYSTEM.md
02_RUNNER_SYSTEM.md
03_TOOL_SYSTEM.md
04_ITEMS_SYSTEM.md
05_GUARDRAILS.md
06_HANDOFFS.md
07_MEMORY_SESSIONS.md
08_MODEL_PROVIDERS.md
09_SANDBOX_SYSTEM.md
10_TRACING.md
11_RUN_STATE.md
12_CONTEXT.md
13_LIFECYCLE_HOOKS.md
14_CONFIGURATION.md
15_ERROR_HANDLING.md
16_STREAMING.md
17_EXTENSIONS.md
18_MCP_INTEGRATION.md
19_BEST_PRACTICES.md
20_ARCHITECTURE_PATTERNS.md
opencode-study
context-handling
core
Python
Alembic
Basics
sqlalchemy - fastapi
SQLAlchemy overview
tweets
system_design_for_agentic_apps.md
Agent Loop
Plain text

Streamer Core Functions

File: src/server/repositories/streamer/streamer.ts

1. Initialize SSE Connection

File: src/server/repositories/streamer/streamer.ts:129

TypeScript

Sets up the SSE connection with proper headers.

2. Stream V1 (Legacy)

File: src/server/repositories/streamer/streamer.ts:208

TypeScript

Key Features:

  • Adaptive Pacing: Slows down when buffer is full (better UX)
  • Tool Call Assembly: Accumulates multi-chunk tool calls
  • Citation Handling: Special format for source citations
  • Graceful Errors: Resolves with partial results on failure
3. Stream V2 (Current)

File: src/server/repositories/streamer/streamer.ts:469

TypeScript

V2 Improvements:

  • Index-based: Each chunk has explicit index for ordering
  • Tool Result Streaming: Sub-agents stream through parent
  • Reasoning Display: Shows model reasoning to user
  • Simplified: Removed adaptive pacing (handled elsewhere)

EventManager: Progress Tracking

File: src/server/repositories/streamer/streamer.ts:1031

Manages UI progress events:

TypeScript

Usage Example:

TypeScript

SSEProgressEvent: Individual Steps

File: src/server/repositories/streamer/streamer.ts:794

TypeScript

Event Types

Streamed to Client:
TypeScript
Event Types Sent:
TypeScript

Stream End Handling

File: src/server/repositories/streamer/streamer.ts:771

TypeScript

Sends final event: end to signal completion.


Citation Handling

File: src/server/repositories/streamer/streamer.ts:151

TypeScript

Citation Format: [citation: 1: 2] means:

  • Document #1 (first attachment)
  • Citation #2 within that document

Index Management (CRITICAL)

File: src/server/endpoints/unified/orchestrator/helpers/baseUtils.ts

TypeScript

Why This Matters:

  • Prevents multi-agent streaming conflicts
  • Ensures correct chunk ordering
  • Global index synchronized across all agents

Integration with Orchestrator

TypeScript

Summary

The streaming architecture:

  1. SSE Protocol: Text/event-stream with structured data
  2. Dual APIs: V1 (legacy) and V2 (indexed, current)
  3. EventManager: Hierarchical progress tracking
  4. Adaptive Pacing: V1 slows when buffer full
  5. Index Synchronization: Prevents multi-agent conflicts
  6. Graceful Errors: Resolves with partial results
  7. Citation Support: Automatic attachment linking

Key Principle: Users see progress in real-time, even during multi-step tool execution. Never make them wait for the final result.