Inkdown
Start writing

Study

70 filesยท12 subfolders

Shared Workspace

Study
AI eng

09-sharding-partitioning

Shared from "Study" on Inkdown

09 - Sharding & Partitioning

The Scaling Problem

As data grows, a single database server eventually hits limits:

  • CPU: Query processing, connection handling
  • Memory: Cache, working set size
  • Disk: Storage capacity, IOPS
  • Network: Bandwidth for replication
Plain text

Partitioning vs Sharding

basic-ques
core
Revision w/ Whiteboard
CN Basics - 1
CN Basics - 2
DNS
Event loop
programming-language-concepts.md
zero-language-explanation.md
DB
Quick
databases-deep-dive.md
01-introduction.md
02-relational-databases.md
03-database-design.md
04-indexing.md
05-transactions-acid.md
06-nosql-databases.md
07-query-optimization.md
08-replication-ha.md
09-sharding-partitioning.md
10-caching-strategies.md
11-cap-theorem.md
12-connection-pooling.md
13-backup-recovery.md
14-monitoring.md
15-database-selection.md
README.md
JS
core topics
Event loop
Merlin Backend
01-Orchestration.md
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft
Mobile
Build Alternative
Bundling
metro-bundler-deep-dive.md
OpenAI Agents Python
00_OVERVIEW.md
01_AGENT_SYSTEM.md
02_RUNNER_SYSTEM.md
03_TOOL_SYSTEM.md
04_ITEMS_SYSTEM.md
05_GUARDRAILS.md
06_HANDOFFS.md
07_MEMORY_SESSIONS.md
08_MODEL_PROVIDERS.md
09_SANDBOX_SYSTEM.md
10_TRACING.md
11_RUN_STATE.md
12_CONTEXT.md
13_LIFECYCLE_HOOKS.md
14_CONFIGURATION.md
15_ERROR_HANDLING.md
16_STREAMING.md
17_EXTENSIONS.md
18_MCP_INTEGRATION.md
19_BEST_PRACTICES.md
20_ARCHITECTURE_PATTERNS.md
opencode-study
context-handling
core
Python
Alembic
Basics
sqlalchemy - fastapi
SQLAlchemy overview
tweets
system_design_for_agentic_apps.md
Agent Loop
Plain text

Table Partitioning

Range Partitioning
Sql
List Partitioning
Sql
Hash Partitioning
Sql
Sub-partitioning
Sql
Partition Maintenance
Sql

Database Sharding

Sharding Strategies
1. Hash Sharding
Plain text
Python
2. Range Sharding
Plain text
Python
3. Directory-Based (Lookup) Sharding
Plain text
Sharding Architecture
Plain text
Cross-Shard Operations
Plain text
Consistent Hashing
Plain text
Re-sharding (Adding Shards)
Plain text

When to Shard

Plain text

Next: Caching Strategies