Inkdown
Start writing

Study

70 filesยท12 subfolders

Shared Workspace

Study
AI eng

01-introduction

Shared from "Study" on Inkdown

01 - Introduction to Databases

What is a Database?

At its core, a database is an organized collection of data that can be easily accessed, managed, and updated. Think of it as a digital filing system that's smarter, faster, and more reliable than storing data in files.

Why Not Just Use Files?
Python
basic-ques
core
Revision w/ Whiteboard
CN Basics - 1
CN Basics - 2
DNS
Event loop
programming-language-concepts.md
zero-language-explanation.md
DB
Quick
databases-deep-dive.md
01-introduction.md
02-relational-databases.md
03-database-design.md
04-indexing.md
05-transactions-acid.md
06-nosql-databases.md
07-query-optimization.md
08-replication-ha.md
09-sharding-partitioning.md
10-caching-strategies.md
11-cap-theorem.md
12-connection-pooling.md
13-backup-recovery.md
14-monitoring.md
15-database-selection.md
README.md
JS
core topics
Event loop
Merlin Backend
01-Orchestration.md
02-DeepResearch.md
03-Search.md
04-Scraping.md
05-Streaming.md
06-MultiProviderLLM.md
07-MemoryAndContext.md
08-ErrorHandling.md
09-RateLimiting.md
10-TaskQueue.md
11-SecurityAndAuth.md
Orchestration-2nd-draft
Mobile
Build Alternative
Bundling
metro-bundler-deep-dive.md
OpenAI Agents Python
00_OVERVIEW.md
01_AGENT_SYSTEM.md
02_RUNNER_SYSTEM.md
03_TOOL_SYSTEM.md
04_ITEMS_SYSTEM.md
05_GUARDRAILS.md
06_HANDOFFS.md
07_MEMORY_SESSIONS.md
08_MODEL_PROVIDERS.md
09_SANDBOX_SYSTEM.md
10_TRACING.md
11_RUN_STATE.md
12_CONTEXT.md
13_LIFECYCLE_HOOKS.md
14_CONFIGURATION.md
15_ERROR_HANDLING.md
16_STREAMING.md
17_EXTENSIONS.md
18_MCP_INTEGRATION.md
19_BEST_PRACTICES.md
20_ARCHITECTURE_PATTERNS.md
opencode-study
context-handling
core
Python
Alembic
Basics
sqlalchemy - fastapi
SQLAlchemy overview
tweets
system_design_for_agentic_apps.md
Agent Loop

Types of Data Storage

1. Flat Files (CSV, JSON, XML)
  • Good for: Small datasets, simple configurations
  • Bad for: Concurrent access, complex queries, relationships
2. Relational Databases (SQL)
  • Good for: Structured data, complex relationships, ACID transactions
  • Examples: PostgreSQL, MySQL, Oracle, SQL Server
3. NoSQL Databases
  • Good for: Unstructured/semi-structured data, massive scale, rapid iteration
  • Examples: MongoDB, Cassandra, Redis, DynamoDB
4. Specialized Databases
  • Time-series: InfluxDB, TimescaleDB (metrics, IoT)
  • Graph: Neo4j, Amazon Neptune (relationships)
  • Search: Elasticsearch, OpenSearch (full-text search)
  • Vector: Pinecone, Weaviate (AI embeddings)

The Database Management System (DBMS)

A DBMS is software that interacts with end-users, applications, and the database itself to capture and analyze data. It provides:

Plain text

Key Components Explained

1. Storage Engine

The heart of the database - decides HOW data is stored and retrieved.

B-Tree Engine (InnoDB, PostgreSQL)

Plain text

LSM-Tree Engine (Cassandra, RocksDB)

Plain text
2. Query Processor
Sql
3. Transaction Manager

Ensures data integrity when multiple operations happen together.

Database Architecture Patterns

1. Single Node (Monolith)
Plain text
  • Simplest setup
  • Good for: Startups, small applications (< 1000 users)
  • Limits: CPU, memory, disk on one machine
2. Primary-Replica (Read Replicas)
Plain text
  • Primary handles writes
  • Replicas handle reads
  • Good for: Read-heavy applications
3. Sharded Architecture
Plain text
  • Data partitioned across multiple servers
  • Each shard holds a subset of data
  • Good for: Very large datasets, high write throughput

Database Selection Flowchart

Plain text

Important Terminology

TermDefinitionAnalogy
SchemaStructure/blueprint of databaseHouse floor plan
Table/CollectionGroup of related dataExcel sheet
Row/DocumentSingle recordOne row in Excel
Column/FieldAttribute of dataColumn in Excel
IndexData structure for fast lookupBook index
Primary KeyUnique identifier for each rowSocial Security Number
Foreign KeyReference to another table's PKPointer to related data
QueryRequest for dataAsking a question
TransactionGroup of operations treated as oneBank transfer (debit + credit)

Performance Metrics to Know

  1. Throughput: Queries per second (QPS) / Transactions per second (TPS)
  2. Latency: Time for a single query (p50, p95, p99 percentiles)
  3. Concurrency: Number of simultaneous connections
  4. Storage: Disk space used, I/O operations per second (IOPS)

Next Steps

Now let's dive deeper into each topic:

  • 02 - Relational Databases & SQL
  • 03 - Database Design & Normalization
  • 04 - Indexing Deep Dive
  • 05 - Transactions & ACID
  • 06 - NoSQL Databases
  • 07 - Query Optimization
  • 08 - Replication & High Availability
  • 09 - Sharding & Partitioning
  • 10 - Caching Strategies
  • 11 - CAP Theorem & Distributed Systems
  • 12 - Connection Pooling & Performance
  • 13 - Backup & Recovery
  • 14 - Monitoring & Observability
  • 15 - Choosing the Right Database