Inkdown
Start writing

Study

70 filesยท12 subfolders

Shared Workspace

Study
AI eng

01-Orchestration

Shared from "Study" on Inkdown

Orchestration Architecture

Overview

The orchestration layer is the brain of the Arcane system. It manages multi-agent workflows, handles iterative tool calling, maintains conversation state, and coordinates all AI-driven operations.


Architecture Flow

Plain text
basic-ques
core
Revision w/ Whiteboard
CN Basics - 1
CN Basics - 2
DNS
Event loop
programming-language-concepts.md
zero-language-explanation.md
DB
Quick
databases-deep-dive.md
01-introduction.md
02-relational-databases.md
03-database-design.md
04-indexing.md
05-transactions-acid.md
06-nosql-databases.md
07-query-optimization.md
08-replication-ha.md
09-sharding-partitioning.md
10-caching-strategies.md
11-cap-theorem.md
12-connection-pooling.md
13-backup-recovery.md
14-monitoring.md
15-database-selection.md
README.md
JS
core topics
Event loop
Merlin Backend
01-Orchestration.md
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft
Mobile
Build Alternative
Bundling
metro-bundler-deep-dive.md
OpenAI Agents Python
00_OVERVIEW.md
01_AGENT_SYSTEM.md
02_RUNNER_SYSTEM.md
03_TOOL_SYSTEM.md
04_ITEMS_SYSTEM.md
05_GUARDRAILS.md
06_HANDOFFS.md
07_MEMORY_SESSIONS.md
08_MODEL_PROVIDERS.md
09_SANDBOX_SYSTEM.md
10_TRACING.md
11_RUN_STATE.md
12_CONTEXT.md
13_LIFECYCLE_HOOKS.md
14_CONFIGURATION.md
15_ERROR_HANDLING.md
16_STREAMING.md
17_EXTENSIONS.md
18_MCP_INTEGRATION.md
19_BEST_PRACTICES.md
20_ARCHITECTURE_PATTERNS.md
opencode-study
context-handling
core
Python
Alembic
Basics
sqlalchemy - fastapi
SQLAlchemy overview
tweets
system_design_for_agentic_apps.md
Agent Loop

Core: ToolOrchestrator Class

File: src/server/endpoints/unified/orchestrator/toolOrchestrator.ts:69

TypeScript

Purpose:

  • Maintains state across multiple tool-calling iterations
  • Coordinates between LLM, tools, and streaming
  • Manages token usage and context limits
  • Supports sub-agents (Deep Research spawns ResearcherAgent)

The Main Loop (Heart of the System)

File: src/server/endpoints/unified/orchestrator/toolOrchestrator.ts:237

This is where the magic happens - iterative reasoning with tools:

TypeScript

Why This Loop Matters:

  • Iterative: LLM sees tool results, then decides next action
  • Stateful: Each iteration builds on previous results
  • Streaming: Real-time updates to client
  • Resilient: Tool errors don't crash the system

State Management

File: src/server/endpoints/unified/orchestrator/toolOrchestrator.ts:129

TypeScript

Critical Field: currentContentIndex

This prevents streaming conflicts in multi-agent scenarios. Each chunk sent to client has an index:

  • Main thread uses indices 0, 1, 2, 3...
  • Sub-agent starts where parent left off
  • Prevents race conditions where chunks arrive out of order

Agent Configuration System

Location: src/server/endpoints/unified/orchestrator/configs/

TypeScript

Three Built-in Agents:

  1. MainThreadAgent (mainThread.config.ts)

    • Standard chat
    • 8-15 iterations (plan-based)
    • Parallel tools enabled
    • Full data storage
  2. DeepResearchSupervisor (deepResearch.config.ts)

    • Complex research orchestration
    • Time-limited (e.g., 5 minutes)
    • Sequential execution
    • Custom tool filtering
  3. ResearcherAgent (researcher.config.ts)

    • Individual research tasks
    • Spawned by supervisor
    • Tool-specific focus

Parallel Tool Execution

File: src/server/endpoints/unified/orchestrator/toolOrchestrator.ts:929

TypeScript

Tools run in parallel, results stream as they arrive. No waiting for slow tools.


Individual Tool Lifecycle

File: src/server/endpoints/unified/orchestrator/toolOrchestrator.ts:962

TypeScript

Tool Result Types:

  • tool:start - Execution begins
  • tool:stream - Streaming partial results (sub-agents)
  • tool:done - Success with final result
  • tool:error - Failed (caught gracefully)
  • tool:progress - Overall batch tracking

Multi-Agent Coordination

Plain text

Sub-Agent Spawning Example:

TypeScript

Integration Points

  1. Engine (src/server/repositories/engine/engine.ts)

    • Context trimming after each iteration
    • Layout selection algorithm
    • Token counting
  2. Streamer (src/server/repositories/streamer/streamer.ts)

    • SSE streaming to client
    • EventManager integration
    • Index synchronization
  3. Provider (src/server/repositories/provider/provider.ts)

    • LLM API calls via Rune
    • Token cost calculation
    • Model selection
  4. ToolRegistry (src/server/endpoints/unified/tools/toolRegistry.ts)

    • Tool availability per agent
    • Dynamic tool loading

Summary

The orchestration system:

  • Iterative reasoning: Multi-step tool calling
  • Multi-agent: Different agents for different tasks
  • Parallel execution: Tools run concurrently
  • Streaming-first: Real-time progress
  • Resilient: Errors contained, not catastrophic
  • Token-aware: Context trimming prevents overflow

Key insight: This isn't single-pass Q&A. It's an iterative reasoning engine where the LLM can take multiple actions, see results, and continue thinking - just like a human problem-solving approach.