Why Cache?
Databases are slow (disk seeks take milliseconds). Reading from RAM is 100,000× faster. Caching stores frequently accessed data in memory so subsequent reads hit the cache instead of the DB. A 95% cache hit rate means 95% of reads skip the DB entirely.
Cache hierarchy (fastest to slowest): CPU L1 cache (0.5 ns) → L2 cache (5 ns) → RAM (100 ns) → SSD (100 μs) → HDD (10 ms) → network (1-10 ms) → remote DB (10-100 ms). Cache at the level that removes your bottleneck.
Common caches in web systems: Application-level (Redis, Memcached) for DB queries, session data, and computed results. CDN cache for static assets and API responses. Browser cache for static files. DNS cache for domain lookups.
Cache-Aside (Lazy Loading)
The most common pattern. Application code manages the cache: on a read, check cache first. If hit, return cached value. If miss, read from DB, store result in cache, return result. Application is responsible for keeping cache in sync with DB.
Pros: only requested data is cached (no unnecessary preloading), tolerates cache failures (fall back to DB). Cons: first request is slow (cache miss), cache can become stale if DB is updated without invalidating the cache.
Implementation in Redis: GET user:{user_id} → if nil, query DB → SET user:{user_id} JSON_data EX 3600. EX 3600 sets a 1-hour TTL — key automatically evicts. This is the "TTL as invalidation" strategy: accept possible staleness up to TTL duration.
Write-Through and Write-Around
Write-Through: on every DB write, also update the cache. Cache is always in sync. Pros: no stale reads. Cons: extra write latency, cache polluted with data that may never be read.
Write-Around: write directly to DB, bypass the cache. Cache is populated on the next read (cache-aside). Pros: good for write-heavy workloads where written data is rarely re-read (logs, events). Cons: first read after write is a cache miss.
Write-Back (Write-Behind): write to cache only; asynchronously persist to DB. Ultra-low write latency. Risk: data loss if cache crashes before DB flush. Used for analytics counters (view count, like count) where small inaccuracy is acceptable.
Eviction Policies
When the cache is full, an eviction policy decides what to remove to make room for new entries.
LRU (Least Recently Used): evict the entry that was accessed least recently. Best for most web workloads — recently accessed items are likely to be accessed again soon. Implementation: doubly linked list + hash map. O(1) put and get.
LFU (Least Frequently Used): evict the entry with the lowest access count. Better for workloads with stable long-term popularity (e.g., product catalog where bestsellers stay popular). More complex to implement.
FIFO: evict the oldest entry. Simple, but doesn't account for access patterns — a heavily used 1-minute-old entry gets evicted before a stale 2-minute-old entry.
TTL (Time-To-Live): entries expire automatically after a fixed time regardless of access pattern. Not technically an eviction policy but the most important staleness control. Combine TTL with LRU for most production caches.
Cache Stampede and Thundering Herd
Cache Stampede: when a popular cache key expires, all concurrent requests miss the cache simultaneously and all rush to the DB. The DB is suddenly hit with 1,000+ queries for the same data. Solution: (1) Jitter — randomize TTLs slightly so keys don't expire at the same time. (2) Locking — only one request queries the DB; others wait. (3) Probabilistic early expiration — before TTL expires, probabilistically recompute and refresh the cache entry.
Thundering Herd: when a cache server restarts and the entire cache is cold, all requests miss. Warm the cache gradually by reading from a backup cache or by slowly promoting requests to the new cache.
Distributed Cache — Redis Cluster
A single cache server has limited RAM and is a single point of failure. Redis Cluster partitions keys across multiple nodes using consistent hashing. Each node owns a subset of the key space (16,384 hash slots). Clients hash the key to determine the slot and then the node. Each node has a replica for fault tolerance. If the primary node fails, its replica is promoted.
Memcached vs Redis: Memcached is simpler (pure key-value, multi-threaded), Redis is richer (sorted sets, lists, pub/sub, persistence). Use Memcached for simple object caching; use Redis when you need data structures (leaderboards, rate limiting, real-time features).