High Level Design (HLD)

High Level Design (HLD) defines the architecture of a software system — which components exist, how they communicate, and how the overall system meets its functional and non-functional requirements. In interview contexts, HLD is synonymous with System Design.

HLD vs LLD: HLD is the architect's view — boxes and arrows showing services, databases, load balancers, CDNs, message queues. LLD is the engineer's view — classes, interfaces, and code structure within a single service. HLD asks "what does the system look like from 10,000 feet?"; LLD asks "how is this one service built?"

What interviewers assess in an HLD round: - Do you ask the right clarifying questions before designing? - Can you estimate scale (QPS, storage, bandwidth) to inform design choices? - Do you know when to use SQL vs NoSQL, REST vs gRPC, synchronous vs async? - Can you identify and resolve bottlenecks (single points of failure, hot partitions)? - Do you understand trade-offs and can you articulate why you made each choice?

The most important skill is not memorizing patterns — it's demonstrating that every decision is a deliberate trade-off.

A 45-minute HLD interview has a predictable structure. The framework below keeps you on track and signals seniority to interviewers.

Minutes 0–5: Clarify functional requirements. What are the core use cases? What is explicitly out of scope? Who are the users? Don't skip this — interviewers penalize candidates who jump to drawing boxes.

Minutes 5–8: Clarify non-functional requirements. Target scale: how many daily active users? Peak QPS? Read/write ratio? Latency SLA (P99 < 200ms?)? Availability target (99.9% = 8.7 hours/year downtime)? Geographic distribution? These numbers drive every architectural decision.

Minutes 8–12: Back-of-envelope estimation. Estimate: QPS (daily requests / 86,400), storage (bytes per item × items/day × retention), bandwidth (QPS × response size), cache size (20% of daily data × hot data ratio). Write these on the board — they tell you if you need sharding, CDN, read replicas.

Minutes 12–17: Define the API. Write 3–5 core REST endpoints. Include request/response shapes. This forces precision about what the system actually does.

Minutes 17–22: Data model. List the core entities, their key fields, and which database(s) store them. Choose SQL or NoSQL and justify it. Identify which fields need indexes.

Minutes 22–35: High-level architecture diagram. Draw the components: client → load balancer → API servers → cache → database → message queue → workers. Add CDN, auth service, monitoring as needed. Walk through the data flow for each core use case.

Minutes 35–45: Deep dives. Pick the 1–2 hardest parts (feed generation, consistency, search) and go deep. Discuss failure modes, bottlenecks, and how you'd resolve them.

A monolith is a single deployable unit containing all the application's functionality. A microservices architecture splits the application into small, independently deployable services that communicate over a network.

Monolith advantages: simple to develop (one codebase, one deployment), easy to test end-to-end, no network overhead between components, easy to debug (one log stream, one trace). Ideal for small teams, early-stage products, and bounded problem domains.

Monolith disadvantages: as the team and codebase grow, deployments become risky (one bug redeploys everything), scaling requires scaling the whole app, different parts can't use different tech stacks, and large teams step on each other.

Microservices advantages: independent scaling (scale only the service under load), independent deployment (ship UserService without touching PaymentService), independent tech stacks, clear team ownership. Enables large organizations to move fast.

Microservices disadvantages: distributed systems complexity (network failures, latency, partial failures), service discovery, distributed tracing, the need for API gateways, data consistency across services (no single transaction), operational overhead (many services to monitor and deploy).

Service mesh (Istio, Linkerd): infrastructure layer handling service-to-service communication — mutual TLS, load balancing, circuit breaking, observability — without changing application code. A sidecar proxy (Envoy) is injected into each pod.

Migration path: start with a modular monolith (clear module boundaries inside one deployment), extract the highest-traffic or most frequently changed module as a service first, then continue incrementally.

The API is the contract between your service and its callers. Getting it right matters because APIs are hard to change once clients depend on them.

REST (Representational State Transfer): resource-based URLs (/users/123), standard HTTP verbs (GET, POST, PUT, PATCH, DELETE), stateless requests, JSON responses. Widely understood, human-readable, cacheable. Best for public-facing APIs and CRUD-heavy services.

GraphQL: clients specify exactly what data they need in a query. Eliminates over-fetching (getting too much) and under-fetching (needing a second request). Ideal for complex, nested data (social networks, e-commerce). Disadvantages: harder to cache, complex query optimization needed, N+1 problem if not careful.

gRPC: uses Protocol Buffers (binary serialization) over HTTP/2. Very fast (10x smaller payloads than JSON), strongly typed contracts, bidirectional streaming. Ideal for internal service-to-service communication where performance matters. Disadvantages: binary format is not human-readable, harder to debug.

REST best practices: use nouns for resources (/orders, not /getOrders), plural forms, HTTP status codes semantically (200 OK, 201 Created, 400 Bad Request, 401 Unauthorized, 404 Not Found, 429 Too Many Requests, 500 Server Error). Use query params for filters (/orders?status=pending&limit=20). Version your API (/v1/orders).

Idempotency: a request is idempotent if making it multiple times has the same effect as making it once. GET, PUT, DELETE are idempotent. POST is not (two POST /orders creates two orders). For POST operations that should be idempotent (payment), accept an Idempotency-Key header and store the result keyed by that ID.

Pagination: offset-based (/items?page=2&limit=20) is simple but slow for large offsets. Cursor-based (/items?after=cursor123&limit=20) uses a stable pointer and is O(1) regardless of position — preferred for large datasets.

A load balancer distributes incoming traffic across multiple backend servers to prevent any single server from becoming a bottleneck, and provides automatic failover when a server goes down.

L4 vs L7 load balancing: L4 (transport layer) operates on TCP/UDP — routes based on IP and port. Very fast, can't inspect content. L7 (application layer) operates on HTTP — can route based on URL path, headers, cookies, and content. More powerful but adds latency.

Load balancing algorithms: - Round Robin: requests distributed in rotation. Simple, works well when servers are uniform. - Weighted Round Robin: servers with more capacity get proportionally more requests. - Least Connections: routes to the server with the fewest active connections. Good when request duration varies. - IP Hash: same client IP always routes to the same server (session stickiness). Useful for stateful apps, but breaks if the server pool changes. - Consistent Hashing: hash requests to a ring, route to the nearest server clockwise. Adding/removing a server only remaps 1/N of traffic.

Health checks: the load balancer periodically probes backends (HTTP GET /health or TCP ping). Servers that fail health checks are removed from the pool automatically. Active health checks probe constantly; passive health checks monitor live traffic.

Global load balancing (anycast, GeoDNS): route users to the nearest data center based on geographic location. Reduces latency and provides disaster recovery at the regional level.

SSL termination at the load balancer: decrypts HTTPS at the LB, sends plain HTTP to backends. Simpler certificate management, backends don't need SSL overhead. For end-to-end encryption, re-encrypt before sending to backends.

Caching stores frequently accessed data in a fast store (usually in-memory) to reduce latency and database load. A 1ms cache hit vs a 10ms database query is a 10x improvement.

Cache-aside (lazy loading): the application checks the cache first. On a miss, it fetches from the database, stores in cache, then returns. The cache only holds data that's actually been requested. Cons: first request always slow (cold start), data can be stale.

Read-through: the cache sits in front of the database. On a miss, the cache itself fetches from the database and stores the result. The application always talks to the cache. Consistent code path; cons: cache provider must know how to fetch data.

Write-through: data is written to the cache and database simultaneously. Cache is always consistent with the database. Cons: writes are slower (two writes), cache fills with data that may never be read.

Write-behind (write-back): data is written to the cache first, and asynchronously flushed to the database. Writes are very fast. Cons: data loss risk if the cache crashes before flush, complex consistency.

Redis vs Memcached: Redis supports rich data structures (strings, lists, sets, sorted sets, hashes), persistence, Lua scripting, pub/sub, and clustering. Memcached is simpler, multi-threaded, and slightly faster for simple key-value workloads. Use Redis almost always — its versatility is worth the slight overhead.

Cache eviction policies: LRU (Least Recently Used — evict the item accessed longest ago), LFU (Least Frequently Used — evict the least-accessed item), FIFO (first in, first out — ignores access patterns). LRU is the default for most caches.

Cache invalidation is the hardest problem. Strategies: TTL (expire after N seconds — simple, may serve stale data), event-driven invalidation (publish a cache-invalidation event when data changes — consistent but complex), version keys (embed a version in the cache key — guaranteed fresh but fills the cache).

At scale, a single database instance cannot handle the read/write load or store all the data. Scaling strategies build on each other.

Read replicas: a primary (master) handles all writes and replicates to one or more replicas. Reads are distributed across replicas. Replication lag means replicas may be slightly behind the primary — acceptable for reads that tolerate slight staleness (analytics, profile views), not acceptable for reads that must see their own writes (post a comment, immediately see it).

CQRS (Command Query Responsibility Segregation): split the data model for writes (commands) and reads (queries). The write side uses a normalized relational model optimized for consistency. The read side uses a denormalized model (or a separate read store like Elasticsearch) optimized for the specific queries your UI needs. Changes propagate from write to read store asynchronously.

Event sourcing: instead of storing current state, store a log of events that produced that state. The current state is derived by replaying events. Benefits: complete audit log, time travel (replay to any point), easy event-driven integration. Cons: complexity, eventual consistency, event schema evolution is hard.

Polyglot persistence: use the right database for each use case. PostgreSQL for transactional data, Redis for sessions and caching, Elasticsearch for full-text search, Cassandra for time-series append-heavy workloads, Neo4j for graph queries, S3 for blob storage. The tradeoff is operational complexity.

Connection pooling: database connections are expensive to open. A connection pool maintains N open connections and leases them to requests. PgBouncer for PostgreSQL, HikariCP for Java applications. Without pooling, 10,000 concurrent users = 10,000 database connections = crash.

Message queues and event streams decouple producers from consumers, enabling asynchronous processing, load leveling, and event-driven architectures.

Message queue (RabbitMQ, SQS, ActiveMQ): a producer sends a message to a queue; a consumer reads and processes it. Messages are typically consumed once (point-to-point). Use for: background jobs, email sending, asynchronous task processing, microservice communication that doesn't need a response.

Event streaming (Kafka, Kinesis): a persistent, ordered, replayable log of events. Multiple consumers can read the same stream independently. Events are retained for days or weeks. Use for: event sourcing, real-time analytics, change data capture (CDC), audit logs, streaming pipelines.

Kafka key concepts: topics (named streams), partitions (parallelism unit — one consumer per partition), offsets (position in partition), consumer groups (each group gets all messages; consumers within a group share partitions). Increasing partitions increases throughput linearly.

Delivery guarantees: at-most-once (fire and forget — fast, may lose messages), at-least-once (retry on failure — messages may be processed twice, requires idempotent consumers), exactly-once (Kafka transactions or idempotent producers + deduplication — hardest to achieve).

Dead letter queue (DLQ): messages that fail processing repeatedly are moved to a DLQ for manual inspection. Prevents bad messages from blocking the queue indefinitely.

Backpressure: when consumers are slower than producers, the queue grows. Solutions: scale consumers, add circuit breakers, shed load (drop low-priority messages), or use reactive streams with backpressure propagation.

Distributed systems face a fundamental tension between consistency (all nodes see the same data at the same time) and availability (every request gets a response). The CAP theorem formalizes this.

CAP Theorem: a distributed system can guarantee at most two of three properties: Consistency (every read sees the most recent write or an error), Availability (every request receives a response — not necessarily the most recent), Partition Tolerance (the system continues operating when network partitions occur). Since partitions are unavoidable in real networks, you choose between CP and AP.

CP systems (HBase, ZooKeeper, MongoDB with majority writes): sacrifice availability during partitions — if nodes can't agree, they return an error rather than stale data. Use for: financial transactions, inventory management, anything where stale data causes real harm.

AP systems (Cassandra, DynamoDB, CouchDB): remain available during partitions but may return stale data. Eventually consistent — all nodes will converge when the partition heals. Use for: social feeds, recommendations, analytics, anything where slight staleness is acceptable.

PACELC extends CAP: even when there's no partition (P), you trade off latency (L) vs consistency (C). Some systems sacrifice consistency for low latency even in normal operation.

Quorum reads/writes (Cassandra model): with N replicas, a write succeeds when W replicas acknowledge, a read succeeds when R replicas respond. Strong consistency requires W + R > N (typically W=R=ceil(N/2)+1). Tunable consistency lets you choose per-query.

Saga pattern: for distributed transactions spanning multiple services, use a saga — a sequence of local transactions where each step publishes an event triggering the next. If a step fails, compensating transactions roll back the previous steps. Choreography (services react to events) vs Orchestration (a central coordinator directs services).

A Content Delivery Network (CDN) is a geographically distributed network of servers that cache and serve content from the location closest to the user, reducing latency and origin server load.

How CDNs work: when a user requests content, DNS resolves to the nearest CDN Point of Presence (PoP). If the PoP has the content cached (cache hit), it serves it directly. If not (cache miss), it fetches from the origin server, caches it, and serves it. Subsequent requests from the same region are served from cache.

Push CDN: you proactively push content to CDN nodes before users request it. Good for known static assets (CSS, JS, images). Full control, but requires re-pushing on every update.

Pull CDN: CDN fetches from origin on first request per PoP, then caches. Simple — just point the CDN at your origin. Good for less predictable access patterns. First user per region is slower (cache miss).

Cache invalidation: the hardest CDN problem. Options: TTL-based (content expires after N seconds — simple, may serve stale content), URL versioning (/app.v2.js — cache forever, update the URL when content changes — recommended), API-based purge (call CDN API to invalidate a URL — fast but can be expensive for many files).

Edge computing: run code at CDN PoPs instead of the origin. Cloudflare Workers, AWS Lambda@Edge, Vercel Edge Functions. Use cases: A/B testing at the edge, authentication before traffic hits the origin, response transformation, geo-blocking, personalization headers. Reduces round trips to the origin — logic runs 10ms from the user instead of 100ms.

What to put on CDN: static assets (images, CSS, JS, fonts), large media files (video, audio), API responses that are cacheable (product catalog, public data), edge-rendered HTML for performance. Do NOT put on CDN: user-specific data, auth tokens, real-time data that must always be fresh.

Observability is the ability to understand the internal state of a system by examining its external outputs. The three pillars are metrics, logs, and traces.

Metrics: numerical measurements over time. CPU utilization, QPS, error rate, P99 latency, active connections, cache hit rate. Stored in time-series databases (Prometheus, InfluxDB). Visualized in dashboards (Grafana). Cheap to collect, easy to alert on. Metrics tell you something is wrong; logs and traces tell you why.

Logs: timestamped records of discrete events. Structured logs (JSON) are machine-readable and queryable. Log aggregation (ELK stack: Elasticsearch, Logstash, Kibana) or cloud-native (CloudWatch, Datadog, Splunk). Include request IDs in every log line so you can correlate all logs for one request.

Distributed traces: track a single request across multiple services. Each service adds a span to the trace. A trace shows the full call graph, timing at each step, and which service caused the overall latency. Tools: Jaeger, Zipkin, Datadog APM, AWS X-Ray. Requires propagating a trace ID in every request header.

SLI, SLO, SLA: Service Level Indicator (SLI) is a metric (availability, P99 latency). Service Level Objective (SLO) is the target (99.9% availability, P99 < 200ms). Service Level Agreement (SLA) is the contractual promise with consequences for breach. Set SLOs tighter than SLAs — if your SLA is 99.9%, your SLO should target 99.95% so you have a buffer.

Alerting philosophy: alert on symptoms (user-visible impact — high error rate, high latency), not causes (high CPU, disk 80% full). Too many alerts → alert fatigue → alerts are ignored. Every alert must be actionable — if the response is "wait and see", it shouldn't page. Use error budgets to gate risky deployments.

Security in distributed systems goes beyond securing a single app — you need to secure every service-to-service hop, every user session, and every piece of data at rest and in transit.

Zero-trust architecture: never trust, always verify. Even traffic inside your VPC is verified. Every service authenticates requests from other services (mutual TLS). No implicit trust based on network location. Principle of least privilege: every service and user gets only the permissions they need, nothing more.

OAuth 2.0 + OIDC flow: the user logs in with an identity provider (Google, Okta). The IDP issues an access token (for API authorization) and an ID token (for identity, via OIDC). The client presents the access token to your API. Your API validates the token (signature, expiry, scope) without calling the IDP for every request.

JWT vs session tokens: JWTs are self-contained (the API validates them locally without a database lookup — stateless, scalable). Sessions require a database or cache lookup on every request (stateful, but instantly revocable). JWTs cannot be revoked without a blocklist; sessions can be deleted. Use JWTs for stateless microservices; use sessions for user-facing web apps where revocation matters.

API gateway authentication: place auth at the gateway. Internal services trust that the gateway validated the request — they receive a forwarded user identity header. Services don't re-validate tokens, reducing latency and centralizing auth logic.

Secrets management: never store secrets (API keys, DB passwords, TLS certs) in environment variables on disk or in code. Use a secrets manager (HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager). Rotate secrets automatically. Audit every secrets access.

SQL injection prevention: use parameterized queries / prepared statements, never string concatenation. Input validation at the boundary. Principle of least privilege for DB users (the API user shouldn't have DROP TABLE permission). Row-level security in PostgreSQL.

Encryption at rest and in transit: TLS 1.2+ for all network traffic (including internal). Encrypt database volumes (transparent database encryption). Encrypt sensitive fields at the application level for PII data — even database admins can't read it in plaintext.

The top 15 HLD questions and the key components and trade-offs for each.

Design a URL Shortener (TinyURL) — Key components: hash function (MD5 truncated or base62 encoded counter), read-heavy (redirect) vs write-light (shorten), cache top URLs in Redis, separate read and write services. Trade-off: collision handling with hashing vs sequential IDs with a counter.

Design Instagram / Photo Sharing — Key components: image upload via presigned S3 URLs (never stream through your servers), CDN for image delivery, metadata in PostgreSQL, fanout-on-write vs fanout-on-read for feed generation.

Design a Twitter / News Feed — Key: pull model (query posts of followed users at read time — simple but slow) vs push model (fanout writes to each follower's feed at write time — fast reads, expensive for users with millions of followers). Hybrid for celebrities.

Design WhatsApp / Chat System — Key: WebSocket connections for real-time delivery, message queue (Kafka) for reliability, message status (sent/delivered/read) via acknowledgements, end-to-end encryption, unread count as a separate counter.

Design YouTube — Key: video upload pipeline (chunked upload → transcoding service → multiple resolutions → CDN), view count as an approximate counter (Redis HyperLogLog or batch aggregation), recommendations via offline ML pipeline.

Design Google Search Autocomplete — Key: Trie data structure per prefix, sharding the trie by prefix range, pre-computing top-K suggestions per node, serving from in-memory cache, updating suggestions asynchronously.

Design a Rate Limiter — Key: token bucket (smooth bursts) vs leaky bucket (strict rate) vs fixed/sliding window counter. Redis + Lua script for atomic increment. Distributed rate limiting (share state across API servers via Redis).

Design Uber / Ride Sharing — Key: geospatial index (QuadTree or Geohash) for nearby driver queries, WebSocket for real-time location updates, surge pricing as a multiplier on a base price formula, matching algorithm (weighted score of distance + rating).

Design a Notification System — Key: multiple channels (push, email, SMS) behind a unified API, channel-specific adapters (FCM for Android, APNs for iOS, Twilio for SMS), retry queue for failed deliveries, user preference service (which channels are enabled).

Design a Distributed Cache (Redis clone) — Key: consistent hashing for partitioning across nodes, replication for high availability, eviction policy (LRU), persistence (AOF vs RDB), CAP trade-off (Redis is CP — primary goes down, replica doesn't auto-promote without Sentinel/Cluster).