Orchestration Layer

Edward Orchestration Workflow - End to End

Simple Technical KT for Engineers

Table of Contents

1. What is Orchestration in Edward?
2. The 5-Minute Overview
3. Step-by-Step Flow
4. Deep Dive: Each Layer
5. Key Files to Read
6. Common Questions

1. What is Orchestration in Edward?

Orchestration = How Edward coordinates all the pieces to turn a user's chat message into working code.

Think of it like a conductor leading an orchestra:

• Conductor = Orchestration layer
• Musicians = LLM, Docker sandbox, file system, package manager, build tools
• Music = The generated code

The orchestration layer makes sure everyone plays at the right time, in the right order.

2. The 5-Minute Overview

The Big Picture

The 3 Main Phases

3. Step-by-Step Flow

Step 1: User Sends Message

File: apps/web/stores/chatStream/useStartStream.ts

Step 2: API Admission

File: apps/api/services/runs/messageOrchestrator.service.ts

Key Points:

• Admission control prevents overload (global + per-user + per-chat limits)
• Run is persisted BEFORE execution (durable)
• Browser gets runId immediately for tracking

Step 3: Worker Picks Up Job

File: apps/api/services/runs/agent-run-worker/processor.ts

Key Points:

• Worker is independent (can restart without losing progress)
• Cancel signal via Redis pub/sub (fast)
• Events persisted to DB as they happen (resumable)

Step 4: Stream Session Setup

File: apps/api/services/chat/session/orchestrator/runStreamSession.orchestrator.ts

Key Points:

• Framework resolved before LLM call (better prompts)
• Token budget checked BEFORE calling LLM (fail fast)
• Post-generation validation + autofix (quality control)

Step 5: Agent Loop (Multi-Turn)

File: apps/api/services/chat/session/loop/agentLoop.runner.ts

Why Multiple Turns?

Loop Continues When:

• Tools were called but no file output yet
• No <done> tag received
• Under turn budget

Loop Stops When:

• Code/file output detected
• <done> tag received
• Tool budget exceeded
• Max turns reached
• Client aborted

Step 6: Turn Execution (Stream + Parse)

File: apps/api/services/chat/session/loop/agentLoop.stream.ts

Key Points:

• Chunks processed as they arrive (not waiting for full response)
• Parser converts raw text → structured events
• Events trigger immediate side effects

Step 7: Parser State Machine

File: apps/api/lib/llm/parser.ts

Parser States:

Why State Machine?

• Chunks can split tags across boundaries
• Need to handle incomplete output safely
• Can't just regex over full string

Step 8: Event Handler (Side Effects)

File: apps/api/services/chat/session/events/handler.ts

Event Types:

Step 9: Sandbox Write Flow (Buffered)

File: apps/api/services/sandbox/write/buffer.ts + flush.ts

Write (Buffered to Redis)

Flush (Redis → Container)

Why Buffer?

Benefits:

• Resilient to partial failures
• Can batch multiple writes
• Can replay/repair on failure

Step 10: Install Task Queue

File: apps/api/services/chat/session/loop/agentLoop.runner.ts

Why Serialize Installs?

Step 11: Turn Outcome Decision

File: apps/api/services/chat/session/loop/agentLoop.turnOutcome.ts

Decision Tree:

Step 12: Finalize

File: apps/api/services/chat/session/orchestrator/runStreamSession.finalize.ts

4. Deep Dive: Each Layer

Layer 1: Message Orchestrator

Purpose: Admission control + queue + stream handoff

Key Functions:

• unifiedSendMessage() - Entry point
• createAdmittedRun() - Create run with limits
• enqueueAdmittedRun() - Queue to worker
• streamRunEventsFromPersistence() - SSE to browser

What Could Go Wrong:

• API key decryption fails
• Model/provider mismatch
• Run admission rejected (limits)
• Queue enqueue fails

Layer 2: Stream Session

Purpose: Framework resolve + message prep + token budget + finalize

Key Functions:

• resolveFramework() - Detect/prefer framework
• prepareBaseMessages() - Build LLM context
• composePrompt() - System prompt
• computeTokenUsage() - Budget check
• finalizeStreamSession() - Persist results

What Could Go Wrong:

• Context limit exceeded
• Framework detection fails
• Finalize persistence fails

Layer 3: Agent Loop

Purpose: Multi-turn execution + outcome decisions

Key Functions:

• runAgentLoop() - Main loop
• executeAgentTurnStream() - Single turn
• resolveTurnOutcome() - Continue/stop decision

What Could Go Wrong:

• Turn budget exceeded
• Max turns reached
• Abort signal received
• Continuation prompt fails

Layer 4: Parser

Purpose: Chunk → event conversion

Key Functions:

• createStreamParser() - State machine
• process() - Parse chunk
• flush() - Handle incomplete output

What Could Go Wrong:

• Tag split across chunks
• Incomplete output
• State machine stuck

Layer 5: Event Handler

Purpose: Side effect execution

Key Functions:

• handleParserEvent() - Dispatch by type
• handleFileContent() - Buffer writes
• handleInstallContent() - Queue installs
• handleCommandEvent() - Run commands

What Could Go Wrong:

• Sandbox not provisioned
• File write fails
• Install conflicts
• Command timeout

Layer 6: Sandbox Write

Purpose: Buffered writes to container

Key Functions:

• writeSandboxFile() - Buffer to Redis
• flushSandbox() - Redis → container
• scheduleSandboxFlush() - Debounced flush

What Could Go Wrong:

• Redis unavailable
• Docker exec fails
• Lock acquisition fails
• Container stopped

5. Key Files to Read

Core Orchestration

Parser + Events

Sandbox

6. Common Questions

Q: Why multi-turn loop instead of one LLM call?

A: Complex tasks need multiple steps:

Q: Why buffer writes to Redis instead of writing directly?

A: Three reasons:

1. Resilience: If Docker fails, buffer survives in Redis
2. Performance: One flush vs many small writes
3. Batching: Multiple chunks → one file write

Q: Why serialize installs?

A: Prevent race conditions:

Q: How does cancellation work?

A: Two mechanisms:

1. Redis pub/sub (fast):

   Plain text

2. DB polling (backup):

   Plain text

Q: What happens if worker crashes mid-turn?

A: Checkpoint system allows resume:

Q: How are tokens budgeted?

A: Multiple levels:

Q: How does framework detection work?

A: Three sources:

Q: What's the difference between run_event and message?

A: Different purposes:

7. Debugging Guide

Trace a Turn

Common Failures

8. Summary

The Orchestration Flow in One Diagram

Key Takeaways

1. Orchestration is layered - Each layer has a clear responsibility
2. Multi-turn is essential - Complex tasks need iteration
3. Buffering matters - Redis buffers make writes resilient
4. Budgets prevent runaway - Token, tool, turn limits
5. Events are durable - Persisted for replay/resume
6. Cancellation is dual - Pub/sub + DB polling

End of Document