Inkdown
Start writing

Merlin Backend

12 files·0 subfolders

Shared Workspace

Merlin Backend
01-Orchestration.md

10-TaskQueue

Shared from "Merlin Backend" on Inkdown

Task Queue Architecture

Overview

The task scheduling system allows users to create recurring or one-time tasks that execute in the background. It uses Google Cloud Tasks for reliable scheduling with Firebase for persistence.

Architecture

Plain text
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft

Google Cloud Tasks Setup

File: src/server/services/gcloudTasks.ts

TypeScript

Task Creation Flow

File: src/server/utilities/tasks/baseUtils.ts:303

TypeScript
Task Creation Handler
TypeScript

RRULE Schedule Generation

File: src/server/utilities/tasks/functions/iCalSchedule.ts

TypeScript

RRULE Examples:

TypeScript

Wake-Up Call Pattern

Problem: Google Cloud Tasks max schedule is 30 days. Solution: Chain of wake-up calls for longer schedules.

Plain text

Code:

TypeScript

Task Execution

File: src/server/endpoints/tasks/executeTask.ts

TypeScript
Next Execution Calculation

File: src/server/utilities/tasks/baseUtils.ts:109

TypeScript

Scheduling Request Format

File: src/server/utilities/tasks/baseUtils.ts:253

TypeScript

Task Data Model

Firestore Schema:

TypeScript

Executions Subcollection:

TypeScript

Summary

The task queue system:

  1. LLM-Powered: Natural language → structured tasks
  2. RRULE Scheduling: Complex recurrence patterns
  3. Wake-Up Pattern: Handle >30 day schedules
  4. Time Zone Aware: User's local time, UTC storage
  5. Cost Tracking: Estimated + actual query costs
  6. Multi-Channel Delivery: Email, web, mobile push
  7. Execution History: Full audit trail
  8. Chain Scheduling: Recurring tasks auto-reschedule

Key Principle: Background tasks should feel like natural conversation continuations, scheduled and executed autonomously.