All notes
Programming

One key expires, a thousand queries hit the database: Redis caching patterns, TTL and cache stampede

What happens in the second a hot cache key expires, and how to tame it: cache-aside, write-through, jitter, locks, XFetch early refresh and invalidation in on_commit.

It is 20:00, a promo push notification has just gone out, and the product page for a discounted phone is getting a few thousand requests per second. The page is fast because the product card lives in Redis with a five-minute TTL. Then, at 20:05:00, the key expires. For the next few hundred milliseconds every request sees an empty cache, every one of them goes to PostgreSQL with the same query, and every one of them writes the same value back into Redis. The database connection pool runs dry, latency jumps, and some requests start timing out. Nothing was "broken": the cache simply did exactly what we told it to.

This is the cache stampede (also called thundering herd or dog-piling). In this post we will build caching in Django step by step: cache-aside, write-through and write-behind, how to choose a TTL, three ways to tame a stampede, and what you should never put in a cache.

Setup: Django's built-in Redis backend

Since Django 4.0 there is a built-in Redis backend; it needs the redis package (redis-py) installed. No third-party backend is required:

# settings.py
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "TIMEOUT": 300,          # default: 5 minutes
        "KEY_PREFIX": "shop",
        "VERSION": 1,
    }
}

Two details that bite people: TIMEOUT is in seconds, None means "never expire", and 0 means "expire immediately" (effectively, do not cache). The real key in Redis looks like shop:1:product:card:42, that is KEY_PREFIX:VERSION:key. Bumping VERSION is a cheap way to invalidate everything after a deploy that changes the shape of cached data.

The naive version and why it breaks

This is what most code looks like on day one:

from django.core.cache import cache

from .models import Product


def get_product_card(product_id: int) -> dict | None:
    return cache.get_or_set(
        f"product:card:{product_id}",
        lambda: Product.objects.filter(pk=product_id)
        .values("id", "name", "price", "stock")
        .first(),
        timeout=300,
    )

It works, and under low traffic it is fine. Under load it has three problems:

  1. Read-compute-write is not atomic. get_or_set does a get, then computes, then writes. Between the miss and the write, every concurrent request also sees a miss and also runs the query. That is the stampede.
  2. All keys share the same TTL. If you warmed a thousand products at the same moment (after a deploy, after a flush), they all expire in the same second too. A single hot key becomes a thousand hot keys.
  3. The cost of a miss is invisible. If the query takes 5 ms, a stampede is noise. If it takes 800 ms (an aggregation, a call to an external API), it can take down the database.

The cache is not a feature that makes things fast. It is a promise that the expensive path will run rarely. A stampede breaks exactly that promise.

Three patterns: cache-aside, write-through, write-behind

Pattern Who writes to the cache Strength Risk
Cache-aside (lazy loading) The reader, on a miss Simple, caches only what is actually read Stampede on a miss, stale data until TTL or invalidation
Write-through The writer, right after the DB write Reads almost never miss Every write pays the cost, unread data gets cached too
Write-behind (write-back) The writer writes only to the cache; the DB is updated later in the background Very fast writes Data loss if the cache dies before the flush

Cache-aside is the default in Django projects. Here is a more careful version of it:

import random

from django.core.cache import cache

from .models import Product

_MISS = object()


def jittered(base: int, spread: float = 0.1) -> int:
    """300 -> a random value between 270 and 330."""
    delta = int(base * spread)
    return base + random.randint(-delta, delta)


def card_key(product_id: int) -> str:
    return f"product:card:{product_id}"


def get_product_card(product_id: int) -> dict | None:
    key = card_key(product_id)
    cached = cache.get(key, _MISS)
    if cached is not _MISS:
        return cached  # may be None: "this product does not exist"

    card = (
        Product.objects.filter(pk=product_id)
        .values("id", "name", "price", "stock")
        .first()
    )
    # negative caching: remember "not found" too, but briefly
    ttl = 30 if card is None else jittered(300)
    cache.set(key, card, timeout=ttl)
    return card

What changed compared to the naive version:

  • A sentinel instead of None. cache.get(key) returns None both when the key is missing and when you stored None. With _MISS we can tell them apart, which lets us cache "this product does not exist" (negative caching). Without it, a bot requesting non-existent IDs would hit the DB on every request.
  • Jitter in the TTL. ±10% spreads expirations out in time, so keys warmed together do not expire together.
  • Caching a dict, not a model instance. A pickled model is tied to the model's code; after a deploy that changes the model, old values in Redis may no longer fit. A plain dict with exactly the fields you need is smaller and safer.

Write-through means updating the cache in the same code path that writes to the DB. In Django it is important to do it after commit (more on that below):

def set_price_write_through(product_id: int, price) -> None:
    def refresh() -> None:
        card = (
            Product.objects.filter(pk=product_id)
            .values("id", "name", "price", "stock")
            .first()
        )
        cache.set(card_key(product_id), card, timeout=jittered(300))

    with transaction.atomic():
        Product.objects.filter(pk=product_id).update(price=price)
        transaction.on_commit(refresh)

Write-behind is when the application writes only to Redis and a background worker (for example Celery) periodically flushes changes to PostgreSQL. It is useful for counters like views or likes, where losing a few increments is acceptable. For money, orders and statuses, it is not acceptable: if Redis restarts before the flush, the data is gone, and PostgreSQL is no longer the source of truth.

Choosing a TTL

There is no "correct" TTL; there is a question: how long can a user see stale data without real harm?

  • Product catalog, categories, public settings: minutes to hours, plus explicit invalidation on change.
  • Heavy aggregates (dashboards, reports): as long as the business accepts, often recomputed in the background.
  • Negative results ("not found"): short, seconds to a minute, otherwise a newly created object stays "invisible".
  • Anything with timeout=None: only with explicit invalidation that you are sure about, otherwise the stale value lives forever.

A TTL is your safety net. Even when you do invalidation on every write, keep a finite TTL: if one invalidation is lost (a crash between commit and delete), the data heals itself.

Taming the stampede

1. A lock: only one process recomputes

The idea: on a miss, one process takes a short lock and computes the value; everyone else waits a bit and reads the result from the cache. In Django, cache.add() is a natural fit, because with the Redis backend it maps to SET ... NX, which is atomic: only one caller gets True.

import time
from collections.abc import Callable
from typing import Any

from django.core.cache import cache

_MISS = object()


def get_or_compute_locked(
    key: str,
    compute: Callable[[], Any],
    ttl: int = 300,
    lock_ttl: int = 10,
    wait: float = 3.0,
) -> Any:
    value = cache.get(key, _MISS)
    if value is not _MISS:
        return value

    lock_key = f"lock:{key}"
    deadline = time.monotonic() + wait
    while True:
        # add() == SET NX: only one process gets True
        if cache.add(lock_key, "1", timeout=lock_ttl):
            try:
                value = cache.get(key, _MISS)  # double-check
                if value is _MISS:
                    value = compute()
                    cache.set(key, value, timeout=jittered(ttl))
                return value
            finally:
                cache.delete(lock_key)

        time.sleep(0.05)
        value = cache.get(key, _MISS)
        if value is not _MISS:
            return value
        if time.monotonic() >= deadline:
            return compute()  # do not hang forever; fall back to the DB

(jittered is the helper from the cache-aside example above.)

Important details: the double-check after taking the lock (someone could have filled the cache while we were waiting), a TTL on the lock itself (if the process dies, the lock disappears on its own), and a deadline for waiters (better to go to the DB once than to hang the request).

This version has one subtle weakness: if compute() takes longer than lock_ttl, the lock expires, another process takes it, and our finally deletes someone else's lock. For short computations this is acceptable. For long ones, use a lock with an owner token. redis-py has one built in: on release it checks the token with a Lua script and deletes the key only if it still belongs to us.

import redis

r = redis.Redis.from_url("redis://127.0.0.1:6379/1")


def rebuild_report(report_id: int) -> None:
    lock = r.lock(f"lock:report:{report_id}", timeout=30, blocking_timeout=5)
    if not lock.acquire():
        return  # someone else is already rebuilding it
    try:
        ...  # heavy computation + cache.set(...)
    finally:
        # deletes the key only if our token is still there (Lua script)
        lock.release()

2. Jitter: do not let keys expire together

We already added it to jittered(). It does not help with one very hot key, but it removes "synchronized" stampedes after a deploy or a cache flush. It costs one line, so use it everywhere.

3. Early refresh: recompute before the key expires

A lock makes users wait during recomputation. A better option for hot keys is to never let them expire under load. The probabilistic early expiration algorithm (XFetch, from the paper "Optimal Probabilistic Cache Stampede Prevention" by Vattani, Chierichetti and Lowenstein) works like this: together with the value we store how long it took to compute (delta) and when it logically expires. On each read, a request "rolls the dice", and the closer we are to expiry, and the more expensive the recomputation, the higher the chance that this particular request recomputes the value early, while everyone else still gets the old one.

import math
import random
import time
from collections.abc import Callable
from typing import Any

from django.core.cache import cache


def get_xfetch(
    key: str,
    compute: Callable[[], Any],
    ttl: int = 300,
    beta: float = 1.0,
) -> Any:
    entry = cache.get(key)
    if entry is not None:
        value, delta, expires_at = entry
        # 1 - random() is in (0, 1], so log() never sees zero
        jitter = -delta * beta * math.log(1.0 - random.random())
        if time.time() + jitter < expires_at:
            return value  # still fresh enough

    started = time.time()
    value = compute()
    delta = time.time() - started  # how long the recompute took
    cache.set(key, (value, delta, time.time() + ttl), timeout=ttl)
    return value

-log(u) for uniform u in (0, 1] is a positive random number, usually small but occasionally large. So far from expiry almost nobody recomputes, and in the last few delta before expiry one or two requests do it, while the key is still present. beta > 1 makes refreshes earlier, beta < 1 later. Another variant of the same idea is a background refresh: a Celery beat task recomputes popular keys every N minutes and users never see a miss at all.

Invalidation and transactions

The most common production bug with caching in Django is not a stampede; it is invalidation at the wrong moment.

from django.core.cache import cache
from django.db import transaction

from .models import Product


# WRONG: the cache changes before the transaction is committed
def set_price_wrong(product_id: int, price) -> None:
    with transaction.atomic():
        Product.objects.filter(pk=product_id).update(price=price)
        cache.delete(card_key(product_id))
        # ...more work here; if it raises, the DB rolls back,
        # but a concurrent reader may already have re-cached the old row


# RIGHT: invalidate only after COMMIT
def set_price(product_id: int, price) -> None:
    with transaction.atomic():
        Product.objects.filter(pk=product_id).update(price=price)
        transaction.on_commit(lambda: cache.delete(card_key(product_id)))

(card_key is from the cache-aside example.)

Why the first version is wrong: until COMMIT, other transactions still see the old row (READ COMMITTED). If you delete the key inside the transaction, a concurrent reader gets a miss, reads the old price from the DB and puts it back into the cache for the next five minutes. transaction.on_commit() runs the callback only after a successful commit, and if the transaction is rolled back, it does not run at all.

Even with on_commit there is a rare race: a reader read the old row before our commit, but wrote it into the cache after our delete. That is exactly why the finite TTL from the previous section is not optional.

What not to cache

  • Balances, payment and order statuses, stock levels when you decide whether to sell. Those decisions must read the source of truth inside a transaction (with select_for_update if needed). A cache can show a "~ in stock" badge, but must not decide whether a payment goes through.
  • Idempotency results and anything that must survive a restart. Redis with an eviction policy can delete a key at any moment; that is its right.
  • Per-user data under a shared key. cache_page on a page that shows the user's name or a CSRF token will serve one user's page to another. Either do not cache such views or make the key include the user.
  • Permissions and roles with a long TTL. A revoked access that still works for an hour is a security bug.
  • Huge values. A multi-megabyte value is slow to (de)serialize and to transfer over the network on every read. Cache a narrow dict, not the entire queryset.

Production pitfalls

  • maxmemory-policy. Redis's default policy is noeviction: when memory is full, writes fail with an error. For a pure cache, allkeys-lru or allkeys-lfu is usually what you want.
  • One Redis for cache and Celery. maxmemory and the eviction policy apply to the whole instance, not to a database number. With allkeys-lru, Redis can evict your Celery queues. Keep the broker and the cache in separate instances.
  • Redis being down. Django's built-in backend does not silently swallow connection errors. Decide deliberately: should the site go to the DB (and survive the extra load?), or return an error?
  • KEYS * for invalidation by pattern. It blocks Redis on a large keyspace. Use SCAN, or better, key versioning (VERSION or a version number in the key itself).
  • Measure, do not guess. Look at the hit ratio and evictions:
# hit/miss counters since the last restart
redis-cli INFO stats | grep -E "keyspace_(hits|misses)|evicted_keys"

# what happens when memory runs out?
redis-cli CONFIG GET maxmemory
redis-cli CONFIG GET maxmemory-policy

# remaining TTL of a specific key (Django key = prefix:version:key)
redis-cli TTL "shop:1:product:card:42"

If evicted_keys grows steadily, the cache is too small for your working set, and the TTL tuning above is not the problem.

Checklist

  • Cache dicts with the fields you need, not model instances or whole querysets.
  • Use a sentinel to distinguish "missing" from "cached None"; cache negative results briefly.
  • Add jitter to every TTL.
  • For expensive or hot keys: a lock with cache.add() / redis.lock(), or early refresh.
  • Invalidate in transaction.on_commit(), never inside the transaction.
  • Always keep a finite TTL as a safety net, even with explicit invalidation.
  • Do not make money decisions based on cache.
  • Separate Redis for cache and for the Celery broker; set maxmemory-policy deliberately.
  • Watch keyspace_hits, keyspace_misses and evicted_keys.

Takeaway

A cache does not make an expensive query cheap; it only makes it rare, and a stampede is the moment when "rare" turns into "all at once". Cache-aside with a sentinel, jittered TTLs and invalidation in on_commit covers most Django projects. For hot and expensive keys add a lock or early refresh, and keep anything that decides about money in PostgreSQL.

// comments

Loading comments...

    // next step

    Let's build something together.

    Message me on Telegram
    aslbek@asldev: ~
    ❯