Shared from "Study" on Inkdown
The LLM provider system routes requests to multiple AI models through a unified abstraction layer called Rune. It handles model selection, token cost calculation, caching, and provider-specific transformations.
File: src/server/repositories/provider/provider.ts:24
File: src/server/constantsSchemasAndTypes/endpointConstants.ts
Rune acts as a unified gateway to multiple LLM providers.
Models are defined with costs, prompts, and capabilities:
Cost Multipliers Applied:
| Operation | Base Cost | Multiplier | Final Cost |
|---|---|---|---|
| Standard chat | 100 | 1× | 100 |
| With web search | 100 | 2× | 200 |
| With data analysis | 100 | 1× + 15 | 115 |
| RAG (Project + ProFinder) | 100 | 1× + 250 | 350 |
| Multiple searches (3×) | 100 | (3+1)× | 400 |
File: src/server/repositories/provider/rune.ts
Why Transform:
Token Counting:
Usage Config:
File: src/server/repositories/provider/provider.ts:322
For simple queries without conversation context:
Use Case:
File: src/server/repositories/provider/functions/helpers.ts
Error Types:
content_filter: Input/output blocked by safety filtersrate_limited: Too many requests to providercontext_length_exceeded: Input too long for modelserver_error: Provider downtimePrompt Caching:
Why Only Last Message:
The multi-provider LLM architecture:
Key Principle: One unified interface, multiple providers, accurate cost tracking, seamless failover between models.