Inkdown
Start writing

Merlin Backend

12 filesยท0 subfolders

Shared Workspace

Merlin Backend
01-Orchestration.md

07-MemoryAndContext

Shared from "Merlin Backend" on Inkdown

Memory and Context Management Architecture

Overview

The context management system prevents token overflow while preserving critical information. It intelligently trims conversation history using multiple strategies based on token costs.

The Engine: Context Trimming

File: src/server/repositories/engine/engine.ts:201

TypeScript
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft

Layout Selection Algorithm

File: src/server/repositories/engine/engine.ts:38

TypeScript

Layout Options:

TypeScript

How It Works:

  1. Calculate token cost for each layout option
  2. Start with cheapest (FULL everywhere)
  3. If total > contextLimit, try next layout
  4. Continue until find one that fits
  5. If none fit, throw LAYOUT_NOT_FOUND error

Token Calculation

File: src/server/repositories/engine/engine.ts:148

TypeScript

Handler Implementations

1. Full Handler (No Trimming)

File: src/server/repositories/engine/engine.ts:133

TypeScript

Returns messages unchanged.

2. Tool-Provided Summary Handler

File: src/server/repositories/engine/engine.ts:94

TypeScript

Uses tool-provided summary if shouldIncludeInHistory: false.

3. LLM Summary Handler

File: src/server/repositories/engine/engine.ts:46

TypeScript

Uses LLM to generate summary of conversation history.

Handler Map

File: src/server/repositories/engine/engine.ts:142

TypeScript

Research Memory System

File: src/server/endpoints/unified/features/deepResearch/memory/index.ts

For deep research, maintains research context across steps:

TypeScript
Memory Structure
TypeScript
Context Scoring
TypeScript

Why Context Scoring:

  • Deep research generates many insights
  • Not all relevant to current step
  • Scoring prioritizes important context
  • Prevents context overflow

Integration with Orchestrator

TypeScript

Context Limit by Plan

Different plans get different context limits:

TypeScript

Summary

The context management system:

  1. Three Sections: HISTORY, IN_LOOP, CURRENT_MESSAGE
  2. Three Strategies: FULL, SUMMARY, TOOL_PROVIDED_SUMMARY
  3. Optimal Layout: Cheapest that fits in contextLimit
  4. LLM Summarization: When tool summaries insufficient
  5. Research Memory: Insights, gaps, contradictions with scores
  6. Context Scoring: Prioritize relevant information
  7. Plan-Based Limits: Free (16k), Pro (32k), Ultra (full)

Key Principle: Preserve maximum useful information while staying within token limits. Summarize only when necessary, use tool summaries when available.