Drills: System Design Interviews

# Writes per day
uploads/day = 10,000,000 * 2        = 20,000,000 / day
write QPS   = 20,000,000 / 100,000s = 200 QPS
peak QPS    ~ 200 * 3                ~ 600 QPS

# Storage per day
storage/day = 20,000,000 * 3 MB     = 60,000,000 MB = 60 TB / day
# -> ~22 PB/year. Object storage (S3), not a database.

Use 100,000 seconds/day and round freely — the interviewer wants the order of magnitude (hundreds of QPS, tens of TB/day), not a precise figure.

(a) Banking ledger    -> SQL (Postgres)
    transactions, strong consistency, ACID — never lose/double money.

(b) IoT firehose      -> NoSQL (wide-column / document)
    huge write volume, flexible schema, horizontal scale, eventual is fine.

(c) Semantic search   -> Vector store (pgvector to start)
    nearest-neighbour over embeddings — neither SQL nor NoSQL does this natively.

The senior move: start with the simplest store that works (often Postgres + pgvector) and only reach for specialised systems when scale or access patterns force it.

# Cache the assembled product page (or its query results) in Redis
key = f"product:{id}"
val = redis.get(key) or load_from_db_and_set(key, ttl=3600)

# Invalidation — data changes rarely, so:
1. Write-through / explicit bust: on product update, delete the key
2. Plus a TTL as a safety net (e.g. 1h) in case a bust is missed

Read-heavy + rarely-changing is the ideal cache case. Pair explicit invalidation (precise) with a TTL (forgiving) — belt and braces.

(a) Chat answer   -> SYNC
    the user is waiting on this exact response; ~3s is acceptable.

(b) PDF ingestion -> ASYNC (queue + workers)
    slow (parse/chunk/embed), depends on rate-limited LLM API.
    Return status:"processing" instantly, finish in the background.

(c) Ready email   -> ASYNC
    nobody is blocking on it; fire it from a worker after ingestion.

Rule of thumb: if a human is staring at the screen waiting for this result, it's sync. If the work is slow, can fail/retry, or nobody's blocked — push it to a queue.

Bottleneck: the single Postgres primary — three problems at once.
  1. Read load saturates it  -> add READ REPLICAS, route reads to them.
  2. File BLOBs bloat it      -> move files to OBJECT STORAGE (S3),
                                 keep only the s3_key in the DB.
  3. Write throughput ceiling -> cache hot reads (Redis); partition/shard
                                 later if writes still exceed one primary.

Storing files in the DB is the smell to spot instantly. The general lesson: a single primary doing everything is the classic first bottleneck — separate concerns (files → S3, reads → replicas, hot data → cache).

# 1. REQUIREMENTS
Functional:     upload doc, ask questions over your own docs, see history.
Non-functional: ~100k users; reads light; answer <3s; uploads can be
                eventually-searchable; never lose a file; LLM cost matters.

# 2. ESTIMATES
100k users * 5 q/day = 500k/day -> ~5 avg / ~15 peak QPS (tiny).
100k * 20 docs * 2MB = 4 TB files -> S3.
2M docs * 40 chunks  = 80M vectors * 6KB ~ 480 GB -> the real constraint.
Cost: 500k LLM answers/day -> cache, batch, fall back.

# 3. API
POST /documents | GET /documents/:id | POST /chat | GET /chat/history

# 4. DATA MODEL (managed Postgres + pgvector)
users(id, org_id, ...)
documents(id, user_id, org_id, s3_key, status, content_hash)
chunks(id, document_id, user_id, org_id, text, embedding vector(1536))
messages(id, user_id, conversation_id, role, content, created_at)

# 5. ARCHITECTURE (split fast read path / slow write path)
CDN -> Load Balancer -> stateless App Servers
App -> Redis (cache + rate limit), Postgres primary (+ read replicas
       for pgvector search), S3 (files), Queue
Queue -> Workers: parse -> chunk -> embed (LLM) -> store vectors

Upload: app stores file in S3, writes documents row status="processing",
        enqueues a job, returns instantly. Workers (idempotent on
        content_hash) finish; user polls until status="ready".

# 6. DEEP-DIVE: vector search scaling
Start pgvector + HNSW -> read replicas -> partition by org_id (also gives
tenant isolation) -> dedicated vector store only when recall/latency demand.

# 7. BOTTLENECKS
Ingestion throughput (LLM rate limit) -> more workers + batching + queue.
Vector search at 100s of millions      -> partition, then dedicated store.

# 8. TRADE-OFFS (say them!)
Eventual consistency on upload: not instantly searchable, but fast/resilient.
pgvector now vs dedicated later: a future migration, but ships faster today.
Multi-tenancy: every query scoped by user_id/org_id — isolation is mandatory.
Reliability: timeouts + retries(backoff/jitter) + circuit breaker, model fallback.
Observability: log QPS, p95 latency, queue depth, LLM spend, worker failures.

The grade is in the shape: did you clarify before designing, let the math pick the hard problems, separate read/write paths with a queue, make workers idempotent, and name your trade-offs unprompted? If yes, you ran a senior interview.