Skip to content

07 · Database Caching — Serve Hot Data from Memory Layer

Scaling Reads · Topic 7 of 7


Why Cache?

A cache avoids repeated database work for hot data. CPU memory access, local storage I/O and a remote cache request are different latency layers: a networked cache does not serve end-to-end requests in nanoseconds. Measure hit latency, miss latency and database work avoided on your actual deployment.


Cache Strategies

Cache-Aside (Lazy Loading)

Application checks cache first. On miss, reads from DB and populates cache.

def get_user(user_id):
    user = cache.get(f"user:{user_id}")
    if user is None:
        user = db.query("SELECT * FROM users WHERE id = %s", user_id)
        cache.set(f"user:{user_id}", user, ttl=300)
    return user

Write-Through

Every write goes to both cache and DB simultaneously.

  • Centralizes cache updates in the write path, but correctness depends on ordering, failures and bypass writers
  • Adds write-path work; latency is not necessarily doubled

Write-Behind (Write-Back)

Write to cache first; DB write happens asynchronously.

  • ✅ Fast writes
  • ❌ Risk of data loss if cache fails before DB write

Read-Through

Cache handles DB reads automatically (cache-aside done by the cache layer, not app).


Cache Invalidation Strategies

Strategy How
TTL expiry Cache entry expires after N seconds
Event-driven invalidation DB change triggers cache delete (via CDC)
Cache-aside + versioning Cache key includes version number

Cache invalidation is hard

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton


Cloud Implementations

  • In-memory key-value store
  • Data structures: strings, hashes, sorted sets, lists
  • Persistence: RDB snapshots + AOF (Append-Only File)
  • Clustering: Redis Cluster (horizontal sharding)
  • DynamoDB Accelerator — in-memory cache fully managed by AWS
  • Microsecond latency for cached reads
  • Understand item-cache versus query-cache freshness; writes bypassing DAX and cached query results can remain stale until expiry
  • Spanner has no native caching layer
  • Application-managed Redis/Memcached for hot reads
  • In-memory analysis layer for sub-second dashboard queries
  • Automatically caches frequently queried data
  • WiredTiger cache (50% of RAM by default)
  • Working set must fit in cache for optimal performance

Cache Stampede (Thundering Herd)

When a popular cache key expires, thousands of requests hit the DB simultaneously.

Solutions:

  • Probabilistic early expiration: refresh before TTL expires
  • Mutex lock: only one request refreshes, others wait
  • Stale-while-revalidate: serve stale and refresh in background

The Stale Refill Race

A reader misses the cache and reads version 1. A writer commits version 2 and deletes the cache key. The reader then stores version 1, reintroducing stale data. "Update DB then delete cache" alone does not eliminate this race. Use a documented freshness budget, TTL, version-aware writes or an invalidation protocol appropriate to the application.

Protect the database during cache outages with bounded concurrency, request coalescing and load shedding; otherwise every miss becomes a database request. Add TTL jitter to spread expirations. Negative caching needs a short TTL and tenant-aware keys, especially for authorization-sensitive data.

Check Your Understanding

#

Question 1

Is a 99% hit ratio enough to prove the cache is healthy?

#

Question 2

When may a stale cache be used for a balance or permission decision?