07 · Database Caching — Serve Hot Data from Memory Layer¶
Scaling Reads · Topic 7 of 7
Why Cache?¶
A cache avoids repeated database work for hot data. CPU memory access, local storage I/O and a remote cache request are different latency layers: a networked cache does not serve end-to-end requests in nanoseconds. Measure hit latency, miss latency and database work avoided on your actual deployment.
Cache Strategies¶
Cache-Aside (Lazy Loading)¶
Application checks cache first. On miss, reads from DB and populates cache.
def get_user(user_id):
user = cache.get(f"user:{user_id}")
if user is None:
user = db.query("SELECT * FROM users WHERE id = %s", user_id)
cache.set(f"user:{user_id}", user, ttl=300)
return user
Write-Through¶
Every write goes to both cache and DB simultaneously.
- Centralizes cache updates in the write path, but correctness depends on ordering, failures and bypass writers
- Adds write-path work; latency is not necessarily doubled
Write-Behind (Write-Back)¶
Write to cache first; DB write happens asynchronously.
- ✅ Fast writes
- ❌ Risk of data loss if cache fails before DB write
Read-Through¶
Cache handles DB reads automatically (cache-aside done by the cache layer, not app).
Cache Invalidation Strategies¶
| Strategy | How |
|---|---|
| TTL expiry | Cache entry expires after N seconds |
| Event-driven invalidation | DB change triggers cache delete (via CDC) |
| Cache-aside + versioning | Cache key includes version number |
Cache invalidation is hard
"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton
Cloud Implementations¶
- In-memory key-value store
- Data structures: strings, hashes, sorted sets, lists
- Persistence: RDB snapshots + AOF (Append-Only File)
- Clustering: Redis Cluster (horizontal sharding)
- DynamoDB Accelerator — in-memory cache fully managed by AWS
- Microsecond latency for cached reads
- Understand item-cache versus query-cache freshness; writes bypassing DAX and cached query results can remain stale until expiry
- Spanner has no native caching layer
- Application-managed Redis/Memcached for hot reads
- In-memory analysis layer for sub-second dashboard queries
- Automatically caches frequently queried data
- WiredTiger cache (50% of RAM by default)
- Working set must fit in cache for optimal performance
Cache Stampede (Thundering Herd)¶
When a popular cache key expires, thousands of requests hit the DB simultaneously.
Solutions:
- Probabilistic early expiration: refresh before TTL expires
- Mutex lock: only one request refreshes, others wait
- Stale-while-revalidate: serve stale and refresh in background
The Stale Refill Race¶
A reader misses the cache and reads version 1. A writer commits version 2 and deletes the cache key. The reader then stores version 1, reintroducing stale data. "Update DB then delete cache" alone does not eliminate this race. Use a documented freshness budget, TTL, version-aware writes or an invalidation protocol appropriate to the application.
Protect the database during cache outages with bounded concurrency, request coalescing and load shedding; otherwise every miss becomes a database request. Add TTL jitter to spread expirations. Negative caching needs a short TTL and tenant-aware keys, especially for authorization-sensitive data.