Inkdown
Start writing

Merlin Backend

12 files·0 subfolders

Shared Workspace

Merlin Backend
01-Orchestration.md

05-Streaming

Shared from "Merlin Backend" on Inkdown

Streaming Architecture

Overview

The streaming system delivers real-time Server-Sent Events (SSE) to clients, enabling live updates as AI processes requests, executes tools, and generates responses.

SSE (Server-Sent Events) Protocol

Plain text

SSE Format:

02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft
Plain text

Streamer Core Functions

File: src/server/repositories/streamer/streamer.ts

1. Initialize SSE Connection

File: src/server/repositories/streamer/streamer.ts:129

TypeScript

Sets up the SSE connection with proper headers.

2. Stream V1 (Legacy)

File: src/server/repositories/streamer/streamer.ts:208

TypeScript

Key Features:

  • Adaptive Pacing: Slows down when buffer is full (better UX)
  • Tool Call Assembly: Accumulates multi-chunk tool calls
  • Citation Handling: Special format for source citations
  • Graceful Errors: Resolves with partial results on failure
3. Stream V2 (Current)

File: src/server/repositories/streamer/streamer.ts:469

TypeScript

V2 Improvements:

  • Index-based: Each chunk has explicit index for ordering
  • Tool Result Streaming: Sub-agents stream through parent
  • Reasoning Display: Shows model reasoning to user
  • Simplified: Removed adaptive pacing (handled elsewhere)

EventManager: Progress Tracking

File: src/server/repositories/streamer/streamer.ts:1031

Manages UI progress events:

TypeScript

Usage Example:

TypeScript

SSEProgressEvent: Individual Steps

File: src/server/repositories/streamer/streamer.ts:794

TypeScript

Event Types

Streamed to Client:
TypeScript
Event Types Sent:
TypeScript

Stream End Handling

File: src/server/repositories/streamer/streamer.ts:771

TypeScript

Sends final event: end to signal completion.

Citation Handling

File: src/server/repositories/streamer/streamer.ts:151

TypeScript

Citation Format: [citation: 1: 2] means:

  • Document #1 (first attachment)
  • Citation #2 within that document

Index Management (CRITICAL)

File: src/server/endpoints/unified/orchestrator/helpers/baseUtils.ts

TypeScript

Why This Matters:

  • Prevents multi-agent streaming conflicts
  • Ensures correct chunk ordering
  • Global index synchronized across all agents

Integration with Orchestrator

TypeScript

Summary

The streaming architecture:

  1. SSE Protocol: Text/event-stream with structured data
  2. Dual APIs: V1 (legacy) and V2 (indexed, current)
  3. EventManager: Hierarchical progress tracking
  4. Adaptive Pacing: V1 slows when buffer full
  5. Index Synchronization: Prevents multi-agent conflicts
  6. Graceful Errors: Resolves with partial results
  7. Citation Support: Automatic attachment linking

Key Principle: Users see progress in real-time, even during multi-step tool execution. Never make them wait for the final result.