System Design Interview Framework

Example: "Design Twitter"

Ask:
 Post tweets? (Yes)
 Follow users? (Yes)
 Timeline: home feed + user profile? (Yes, both)
 Like/retweet? (Nice-to-have, out of scope)
 Search tweets? (Out of scope)
 Direct messages? (Out of scope)
 Trending topics? (Out of scope)

Result: Focus on core features only.

Ask:
 Scale: How many users? (100M daily active users)
 Availability: More important than consistency? (Yes, eventual consistency OK)
 Latency: How fast? (Timeline loads < 1 second)
 Read vs Write: More reads or writes? (10:1 read-heavy)

Why these matter:
- 100M users → Need distributed system
- Availability > Consistency → Use NoSQL, caching
- Read-heavy → Focus on read optimization (caching, CDN)
- < 1s latency → Pre-compute timelines, use CDN

Example: Twitter-like System

Given:
- 100M daily active users (DAU)
- Each user views timeline 5 times/day
- Each timeline shows 20 tweets
- Users post 0.5 tweets/day on average

Read (Timeline Views):
- 100M users × 5 views = 500M timeline requests/day
- 500M / 86,400 seconds = ~6,000 requests/second (QPS)
- Peak (3x average) = 18,000 QPS

Write (Posting Tweets):
- 100M users × 0.5 tweets = 50M tweets/day
- 50M / 86,400 = ~600 tweets/second
- Peak = 1,800 tweets/second

Result: Read-heavy (10:1 ratio). Optimize reads with caching!

Tweets:
- 50M tweets/day × 280 chars × 2 bytes (Unicode) = ~28 GB/day text
- Plus metadata (user ID, timestamp, etc.) = ~30 GB/day
- 30 GB × 365 days = ~11 TB/year

Media (photos/videos):
- 20% of tweets have media
- 50M × 0.2 = 10M media uploads/day
- Avg 200 KB per image = 10M × 200 KB = 2 TB/day
- 2 TB × 365 = 730 TB/year

Total: ~750 TB/year
Result: Need distributed storage (S3, blob storage). Can't fit on 1 server!

Incoming:
- 30 GB text + 2 TB media = ~2 TB/day
- 2 TB / 86,400 seconds = ~24 MB/second

Outgoing (users viewing tweets):
- 500M timeline views × 20 tweets × 300 bytes (avg) = 3 TB/day text
- Plus media views (assume 50% of media): 1 TB/day media
- 4 TB / 86,400 = ~46 MB/second

Result: Outgoing > incoming (read-heavy confirms earlier assumption)

┌──────────┐        ┌──────────┐        ┌──────────┐
│  Client  │───────▶│  Server  │───────▶│ Database │
└──────────┘        └──────────┘        └──────────┘

Works for:
 100 users
 100M users (single point of failure, can't scale)

┌──────────┐
                   │   Load   │
   ┌───────────────│ Balancer │
   │               └──────────┘
   │                     │
   │         ┌───────────┼───────────┐
   ▼         ▼           ▼           ▼
┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐
│Server 1│ │Server 2│ │Server 3│ │Server N│
└────────┘ └────────┘ └────────┘ └────────┘
     │         │           │           │
     └─────────┴───────────┴───────────┘
                     │
                ┌────▼─────┐
                │ Database │
                └──────────┘

Improvements:
 Horizontal scaling: add more servers
 No single point of failure (if 1 server dies, others handle)
 Database still bottleneck

┌──────────┐
                   │   Load   │
                   │ Balancer │
                   └──────────┘
                        │
         ┌──────────────┼──────────────┐
         ▼              ▼              ▼
    ┌────────┐     ┌────────┐     ┌────────┐
    │Server 1│     │Server 2│     │Server 3│
    └────────┘     └────────┘     └────────┘
         │              │              │
         └──────────────┼──────────────┘
                        │
                   ┌────▼─────┐
                   │  Redis   │  ◄── Cache hot data
                   │  Cache   │
                   └──────────┘
                        │
                   ┌────▼─────┐
                   │ Database │  ◄── Cold storage
                   └──────────┘

Benefits:
 80% of reads from cache (< 1ms latency)
 Database load reduced 5x
 Redis: in-memory, very fast

┌────────┐      ┌─────────┐      ┌────────────┐
│  User  │─────▶│   CDN   │─────▶│ Static     │
│(Browser)│      │(Images, │      │ Assets     │
└────────┘      │ JS, CSS)│      │ (S3/Blob)  │
    │           └─────────┘      └────────────┘
    │
    │           ┌──────────┐
    └──────────▶│   Load   │
                │ Balancer │
                └──────────┘
                     │
         ┌───────────┼───────────┐
         ▼           ▼           ▼
    ┌────────┐  ┌────────┐  ┌────────┐
    │ Write  │  │ Write  │  │  Read  │
    │Server 1│  │Server 2│  │Servers │
    └────────┘  └────────┘  └────────┘
         │           │           │
         └───────────┼───────────┘
                     │
         ┌───────────┴───────────┐
         ▼                       ▼
    ┌─────────┐           ┌──────────┐
    │ Primary │──────────▶│ Replicas │
    │Database │ Replicate │ (Readers)│
    │(Writer) │           └──────────┘
    └─────────┘

Benefits:
 CDN: assets served from nearest edge location (20-200ms saved)
 Write servers: optimized for inserts (no caching)
 Read servers: optimized for queries (heavy caching)
 Database replication: reads scale horizontally

// Post a tweet
POST /api/v1/tweets
Request Body:
{
  "user_id": "uuid",
  "text": "Hello world!",
  "media_urls": ["https://cdn.example.com/img1.jpg"]
}
Response:
{
  "tweet_id": "uuid",
  "created_at": "2025-02-12T10:30:00Z"
}

// Get user timeline (home feed)
GET /api/v1/timeline?user_id={uuid}&cursor={cursor}&limit=20
Response:
{
  "tweets": [
    {
      "tweet_id": "uuid",
      "user_id": "uuid",
      "username": "alice",
      "text": "...",
      "created_at": "...",
      "media_urls": [...],
      "likes_count": 42,
      "retweets_count": 10
    },
    // ... 19 more
  ],
  "next_cursor": "base64_encoded_timestamp"
}

// Follow a user
POST /api/v1/follow
Request Body:
{
  "follower_id": "uuid",
  "followee_id": "uuid"
}
Response:
{
  "success": true
}

// Get user profile
GET /api/v1/users/{user_id}
Response:
{
  "user_id": "uuid",
  "username": "alice",
  "bio": "...",
  "followers_count": 1000,
  "following_count": 500,
  "tweets_count": 2000
}

-- PostgreSQL: User data (needs transactions)
CREATE TABLE users (
  user_id UUID PRIMARY KEY,
  username VARCHAR(50) UNIQUE NOT NULL,
  email VARCHAR(255) UNIQUE NOT NULL,
  created_at TIMESTAMP DEFAULT NOW(),
  bio TEXT,
  profile_image_url TEXT
);

CREATE TABLE follows (
  follower_id UUID REFERENCES users(user_id),
  followee_id UUID REFERENCES users(user_id),
  created_at TIMESTAMP DEFAULT NOW(),
  PRIMARY KEY (follower_id, followee_id)
);

-- Why SQL: Need to enforce unique username, email constraints. 
-- Following relationships need joins for "mutual follows" queries.

// DynamoDB (NoSQL): Tweets and Timeline (needs scale + speed)
{
  TableName: "Tweets",
  PartitionKey: "tweet_id",  // UUID
  SortKey: "created_at",      // Timestamp
  Attributes: {
    user_id: "UUID",
    username: "String",       // Denormalized for fast reads!
    text: "String",
    media_urls: ["String"],
    likes_count: "Number",
    retweets_count: "Number"
  }
  // Global Secondary Index: user_id + created_at (for user profile view)
}

{
  TableName: "Timeline",
  PartitionKey: "user_id",    // Owner of timeline
  SortKey: "created_at",      // Latest first
  Attributes: {
    tweet_id: "UUID",
    // Fan-out on write: when user tweets, add to all followers' timelines
  }
  // TTL: 7 days (auto-delete old timeline entries)
}

// Why NoSQL:
// - 50M tweets/day needs horizontal scaling
// - Schema may evolve (polls, videos, etc.)
// - Read-heavy: denormalize for speed (store username in tweet)
// - Fan-out on write: pre-compute timelines in Timeline table

Problem: When user posts tweet, how do 10,000 followers see it?

Option A: Fan-out on Write (Push)
- Store tweet in each follower's timeline immediately
- Read: Fast (just query user's timeline table)
- Write: Slow for celebrities (1M followers = 1M writes)

Option B: Fan-out on Read (Pull)
- Store tweets only in user's own table
- Read: Slow (join tweets from all followed users)
- Write: Fast (1 write only)

Hybrid Solution (What Twitter Actually Does):
- Regular users (<10K followers): Fan-out on write
- Celebrities (>10K followers): Fan-out on read
- At read time: merge pre-computed timeline + celebrity tweets
- Best of both worlds!

What to Cache:
 Timeline: Top 100 tweets per user (Redis Sorted Set, TTL 5min)
 User profiles: Hot users (celebrities) cached (TTL 1hr)
 Tweet metadata: Likes/retweets count (updated async)

Cache Invalidation:
- New tweet: invalidate author's timeline + followers' timelines
- Use pub/sub (Redis) to notify cache servers
- Accept slight delay (eventual consistency)

Cache-Aside Pattern:
1. App checks cache
2. Cache miss: query database
3. Store in cache with TTL
4. Return to user

Problem: Postgres has 1 billion users, can't fit on 1 server.

Shard by user_id:
- Hash(user_id) % N → determines which database shard
- Shard 1: user_id 0-249M
- Shard 2: user_id 250M-499M
- Shard 3: user_id 500M-749M
- Shard 4: user_id 750M-999M

Pros:
 Even distribution
 Each shard handles 250M users

Cons:
 Cross-shard queries hard (e.g., "users who follow user A and B")
 Rebalancing when adding shards is complex

Mitigation:
- Use consistent hashing to minimize re-sharding
- Denormalize to avoid cross-shard queries

Metrics to Track:
 Server health: CPU, memory, disk I/O per instance
 API latency: p50, p95, p99 per endpoint
 Error rates: 4xx, 5xx by endpoint
 Database: Query time, connection pool size, replication lag
 Cache: Hit rate (target >80%), eviction rate

Alerts:
 P99 latency > 2 seconds
 Error rate > 1%
 Replication lag > 10 seconds
 Cache hit rate < 70%

Tools:
- Prometheus + Grafana: Metrics dashboards
- Jaeger: Distributed tracing
- ELK Stack: Centralized logging