Cache invalidation: the key was deleted, the page still shows the old price
Why deleting the cache in post_save puts the old price right back, how on_commit fixes it, generation keys for whole groups, and what to do about local caches in 12 workers.
Imagine a shop page. An operator in the admin panel changes the price of a product from 12 000 to 15 000 so'm and presses "Save". The code is correct: right after the save, a post_save signal deletes the product's key from Redis. Yet ten minutes later a customer still sees 12 000 on the product page. The cache was deleted, nobody wrote to it manually, and the old price is still there.
The famous joke says there are only two hard things in computer science: cache invalidation and naming things. The joke survives because it is true. Putting something in a cache is easy; knowing exactly when that copy stopped being true, and removing it at exactly the right moment, is where things break. In the previous article we talked about cache patterns, TTL and cache stampede. Here we focus on one question only: how to make the cache forget old data correctly.
Why invalidation is hard
A cache is a second copy of data. As soon as there are two copies, there are two sources that can disagree. The database is the truth, the cache is a snapshot from some moment in the past. Invalidation is the act of saying "this snapshot is no longer valid".
Three things make it hard:
- Concurrency. Writers and readers work at the same time, in different processes, often on different servers. The order "update DB, then delete the cache" is not atomic: between those two steps anything can happen.
- Transactions. Your code may think it has changed the data, while the database has not committed it yet. Other connections still see the old row.
- Derived data. One product sits in its own key, in a category listing, on the home page, in search results, in a cached template fragment. Deleting
product:42does not touch any of those.
Here is the starting point, a classic cache-aside read:
# catalog/cache.py
from django.core.cache import cache
from catalog.models import Product
PRODUCT_TTL = 60 * 10 # 10 minutes
def product_key(product_id: int) -> str:
return f"product:{product_id}"
def get_product_data(product_id: int) -> dict:
key = product_key(product_id)
data = cache.get(key)
if data is None:
product = Product.objects.get(pk=product_id)
data = {
"id": product.pk,
"name": product.name,
"price": str(product.price),
}
cache.set(key, data, PRODUCT_TTL)
return data
Approach 1: rely on TTL only
The simplest invalidation is none at all: every key has a TTL and the data is allowed to be stale for at most that long. This is honest and surprisingly often enough. A product description that updates 10 minutes late harms nobody.
The problems start when the business says "the price must change immediately". Shortening the TTL to 5 seconds kills the hit ratio, and you are back to hitting the database. TTL is a ceiling for staleness, not an invalidation strategy. But keep it in mind: every key should still have a TTL, because every other method described below can fail, and TTL is the safety net that limits the damage.
Approach 2: delete in a signal, and why it breaks
The natural next step is to delete the key whenever the model is saved:
# catalog/signals.py -- WRONG
from django.core.cache import cache
from django.db.models.signals import post_save
from django.dispatch import receiver
from catalog.cache import product_key
from catalog.models import Product
@receiver(post_save, sender=Product)
def drop_product_cache(sender, instance, **kwargs):
# Runs inside the transaction, BEFORE commit
cache.delete(product_key(instance.pk))
# catalog/services.py
from django.db import transaction
from catalog.models import PriceHistory, Product
def change_price(product_id: int, new_price) -> None:
with transaction.atomic():
product = Product.objects.select_for_update().get(pk=product_id)
product.price = new_price
product.save(update_fields=["price"]) # post_save -> cache.delete()
PriceHistory.objects.create(product=product, price=new_price)
# COMMIT happens only here
Look at where post_save fires: right after the UPDATE statement, but still inside transaction.atomic(). The commit happens only when the with block exits. With PostgreSQL's default isolation level, READ COMMITTED, other connections do not see uncommitted changes. So we get this timeline:
| Step | Request A (admin, writer) | Request B (customer, reader) |
|---|---|---|
| 1 | UPDATE price = 15000 (not committed) |
|
| 2 | post_save → cache.delete("product:42") |
|
| 3 | INSERT INTO pricehistory ... |
cache miss for product:42 |
| 4 | SELECT → sees the old row, 12000 |
|
| 5 | cache.set("product:42", 12000) |
|
| 6 | COMMIT |
After step 6 the database says 15 000, the cache says 12 000, and nothing will fix it until the TTL expires. The window between steps 2 and 6 is as long as the rest of the transaction: one extra INSERT, a call to an external API, a slow query. With ATOMIC_REQUESTS = True the whole view is one transaction, so the window lasts until the end of the request. On a popular product that window will be hit.
There is a second, quieter bug: if the transaction rolls back, we have deleted a perfectly valid cache entry for nothing. That is harmless for correctness, but it shows the signal runs at the wrong moment.
The fix: transaction.on_commit
Django has a hook exactly for "do this after the data is really in the database": transaction.on_commit(). The callback is stored and executed only after the outermost transaction commits successfully. If the transaction rolls back, the callback is discarded. If there is no active transaction (autocommit mode), it runs immediately.
# catalog/signals.py
from functools import partial
from django.core.cache import cache
from django.db import transaction
from django.db.models.signals import post_delete, post_save
from django.dispatch import receiver
from catalog.cache import product_key
from catalog.models import Product
@receiver(post_save, sender=Product)
@receiver(post_delete, sender=Product)
def drop_product_cache(sender, instance, using, **kwargs):
transaction.on_commit(
partial(cache.delete, product_key(instance.pk)),
using=using,
robust=True,
)
# catalog/apps.py
from django.apps import AppConfig
class CatalogConfig(AppConfig):
name = "catalog"
def ready(self):
from catalog import signals # noqa: F401
A few details that matter:
using=using. The signal gives you the database alias. With several databases the callback must be attached to the transaction of the right connection.robust=True(Django 4.2+). If Redis is unavailable, the exception is logged and otheron_commitcallbacks still run. The data is already committed anyway; failing the request at this point would only confuse the user.partialinstead oflambda.partialcomputesproduct_key(instance.pk)right now. Alambdawould readinstance.pklater, at commit time. Forpost_deleteinside an outer transaction, Django setsinstance.pktoNoneafter deleting, so a lambda would happily delete the keyproduct:None.
Nested atomic() blocks behave as expected: a callback registered inside a savepoint runs after the outermost commit, and if that savepoint is rolled back, its callbacks are dropped.
Invalidate after commit, never before. Before commit, other connections still see the old data and will happily put it back into the cache.
The remaining race and the "delayed double delete"
on_commit closes the big window, but not every window. A reader can still do this: miss the cache, read the old row before the commit, get stuck for a moment (GC pause, slow network), and only write the old value to the cache after our delete has already run. The window is small, usually milliseconds, but it exists.
If your data can tolerate it, the TTL covers it. If it cannot, a common technique is to delete twice: once immediately after commit and once more a little later, when any slow reader has surely finished.
# catalog/tasks.py
from celery import shared_task
from django.core.cache import cache
@shared_task
def delete_cache_keys(keys: list[str]) -> None:
cache.delete_many(keys)
# catalog/signals.py (fragment)
from catalog.tasks import delete_cache_keys
def invalidate_product(product_id: int, category_id: int) -> None:
key = product_key(product_id)
cache.delete(key)
bump_generation(f"category:{category_id}")
# Second delete catches a reader that filled the cache with old data
delete_cache_keys.apply_async(args=[[key]], countdown=2)
This is not a guarantee either: it just makes the window much narrower. Prefer delete over set on writes, by the way. If two writers both put new values into the cache, they can finish in the wrong order and leave the older value there. A delete has no such ordering problem: the next reader always goes to the database.
Signals do not see everything
Signals fire only through Model.save() and Model.delete(). A lot of real code changes data without them:
# None of these send post_save:
Product.objects.filter(category_id=7).update(is_active=False)
Product.objects.bulk_update(products, ["price"])
Product.objects.bulk_create(new_products)
# catalog/services.py
from functools import partial
from django.core.cache import cache
from django.db import transaction
from catalog.cache import product_key
from catalog.models import Product
def deactivate_category(category_id: int) -> None:
with transaction.atomic():
ids = list(
Product.objects.select_for_update()
.filter(category_id=category_id)
.values_list("pk", flat=True)
)
Product.objects.filter(pk__in=ids).update(is_active=False)
keys = [product_key(pk) for pk in ids]
transaction.on_commit(partial(cache.delete_many, keys), robust=True)
QuerySet.update(), bulk_update(), bulk_create(), raw SQL, a migration, another service writing to the same database: none of them send post_save. (QuerySet.delete() does send pre_delete/post_delete for each object, which is why Django has to load them first.) For bulk operations, invalidate explicitly in the same service function, still through on_commit.
If other systems write to the database directly, signals cannot help at all. Then you either route all writes through one service, or you listen to the database itself (for example triggers with LISTEN/NOTIFY, or logical replication). That is a bigger project; for many teams an honest TTL is the right answer there.
Versioned keys: invalidating a group
Deleting product:42 is easy. But what about all pages of the category listing, with every possible page number and filter? You cannot list those keys, and you should not try with KEYS:
# Never in production: KEYS blocks Redis while it walks every key
redis-cli KEYS 'product:*'
# If you really must iterate, SCAN walks the keyspace in small steps
redis-cli --scan --pattern ':1:product:*' --count 1000
The cleaner trick is a generation number in the key. Each namespace (for example a category) has a counter. Every cache key for that namespace includes the current value. To invalidate the whole group, you increment the counter: new reads build new keys, and the old keys simply become unreachable and die by TTL.
# catalog/generations.py
import time
from django.core.cache import cache
def _gen_key(namespace: str) -> str:
return f"gen:{namespace}"
def get_generation(namespace: str) -> int:
key = _gen_key(namespace)
gen = cache.get(key)
if gen is None:
# Never restart from 1: if this key was evicted, old data keys
# may still live under small generation numbers.
cache.add(key, time.time_ns(), timeout=None)
gen = cache.get(key)
return gen
def bump_generation(namespace: str) -> int:
key = _gen_key(namespace)
try:
return cache.incr(key)
except ValueError: # key does not exist
cache.add(key, time.time_ns(), timeout=None)
return cache.incr(key)
# catalog/listing.py
from django.core.cache import cache
from catalog.generations import get_generation
from catalog.models import Product
LIST_TTL = 60 * 5
PAGE_SIZE = 20
def category_page(category_id: int, page: int) -> list[dict]:
gen = get_generation(f"category:{category_id}")
key = f"category:{category_id}:g{gen}:page:{page}"
data = cache.get(key)
if data is None:
start = (page - 1) * PAGE_SIZE
rows = (
Product.objects.filter(category_id=category_id, is_active=True)
.order_by("name", "pk")
.values("pk", "name", "price")[start:start + PAGE_SIZE]
)
data = [{**row, "price": str(row["price"])} for row in rows]
cache.set(key, data, LIST_TTL)
return data
# catalog/signals.py
from functools import partial
from django.core.cache import cache
from django.db import transaction
from django.db.models.signals import post_delete, post_save
from django.dispatch import receiver
from catalog.cache import product_key
from catalog.generations import bump_generation
from catalog.models import Product
def invalidate_product(product_id: int, category_id: int) -> None:
cache.delete(product_key(product_id))
bump_generation(f"category:{category_id}")
@receiver(post_save, sender=Product)
@receiver(post_delete, sender=Product)
def drop_product_cache(sender, instance, using, **kwargs):
transaction.on_commit(
partial(invalidate_product, instance.pk, instance.category_id),
using=using,
robust=True,
)
Why time.time_ns() and not 1 as the starting value? If Redis evicts the counter under memory pressure while the data keys g1, g2, g3 are still alive, restarting from 1 would bring old data back to life. A start value based on the clock is practically always larger than any previous one. Also configure Redis with a volatile-* eviction policy if you can: then keys without a TTL, like these counters, are not evicted.
The price is one extra Redis round trip per read (to fetch the generation). For a page with several namespaces, fetch the generations with cache.get_many() in one call. And remember the edge case: if a product moves from one category to another, both the old and the new category must be bumped, so you need the old category_id (for example from pre_save or a value stored on the instance when it was loaded).
Django also has a built-in version argument and cache.incr_version(), but those work on one key at a time. For "forget a whole group", the explicit generation counter is simpler to reason about.
Several servers and a local cache
Things get harder when there is a cache inside the process. LocMemCache, functools.lru_cache, a module-level dictionary: all of them live in one worker process. With 4 gunicorn workers on 3 servers that is 12 independent copies. Deleting a key in Redis does not touch any of them, and on_commit on one server knows nothing about memory on another.
The practical options:
| Option | Freshness | Complexity |
|---|---|---|
| Short TTL on the local layer (seconds) | stale for up to the TTL | minimal |
| Check a generation in Redis on every read | fresh, one Redis call per read | medium |
| Broadcast invalidation via Redis Pub/Sub | usually fresh, messages can be lost | high |
For most cases a two-level cache with a very short local TTL is enough:
# settings.py
CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.redis.RedisCache",
"LOCATION": "redis://127.0.0.1:6379/1",
"TIMEOUT": 600,
},
"local": {
"BACKEND": "django.core.cache.backends.locmem.LocMemCache",
"LOCATION": "l1",
"TIMEOUT": 5,
},
}
# config/flags.py
from django.core.cache import caches
from config.models import FeatureFlag
def get_flags() -> dict:
local = caches["local"]
flags = local.get("flags")
if flags is None:
shared = caches["default"]
flags = shared.get("flags")
if flags is None:
flags = dict(FeatureFlag.objects.values_list("name", "enabled"))
shared.set("flags", flags, 600)
local.set("flags", flags) # lives 5 s (TIMEOUT of "local")
return flags
Redis Pub/Sub looks attractive, but it is fire-and-forget: a worker that was reconnecting at the moment of publishing never receives the message. So even with broadcasting you keep the short TTL as a backstop. The same thinking applies to read replicas: if the cache is filled from a replica that lags behind the primary, a cache miss right after a write can put old data back. Fill critical keys from the primary.
Accept eventual consistency, consciously
The final step is not technical. Not all data needs the same freshness, and pretending otherwise leads to either a useless cache or wrong data. Decide per type of data:
- Money decisions (balance at payment time, final price at checkout, limits): never decide from the cache. Read from the database, inside the transaction, with
select_for_update()where needed. The cache can show a balance on a dashboard, it must not approve a payment. - Catalog data (names, descriptions, listings):
on_commitinvalidation plus TTL of minutes. - Configuration and flags: generation or short local TTL; a few seconds of staleness is fine.
Write the allowed staleness down next to the cache key. "This may be up to 5 minutes old" is a requirement the business can agree to; "the cache is always fresh" is a promise nobody can keep.
Testing invalidation
Django's TestCase wraps every test in a transaction that is never committed, so on_commit callbacks never run by default. Use captureOnCommitCallbacks(execute=True):
# catalog/tests.py
from decimal import Decimal
from django.test import TestCase
from catalog.cache import get_product_data
from catalog.models import Category, Product
from catalog.services import change_price
class PriceChangeTests(TestCase):
@classmethod
def setUpTestData(cls):
cls.category = Category.objects.create(name="Drinks")
def test_cache_dropped_after_commit(self):
product = Product.objects.create(
name="Tea", price=Decimal("12000.00"), category=self.category
)
get_product_data(product.pk) # warm the cache
with self.captureOnCommitCallbacks(execute=True):
change_price(product.pk, Decimal("15000.00"))
self.assertEqual(get_product_data(product.pk)["price"], "15000.00")
Use LocMemCache in test settings so tests do not share a real Redis with each other or with development.
Checklist
- Every cache key has a TTL, even if you also invalidate explicitly.
- Invalidation runs in
transaction.on_commit(), not directly inpost_save. - Use
partialor capture values immediately; do not readinstance.pkinside a lazy lambda. update(),bulk_*, raw SQL: invalidate explicitly in the service function.- On writes, delete keys instead of setting new values.
- Groups of keys: generation counters, not
KEYS/pattern deletes. - Generation counters never restart from 1; prefer a
volatile-*eviction policy. - In-process caches have a short TTL; you know how many copies exist.
- Money decisions read from the database, never from the cache.
- Tests use
captureOnCommitCallbacks(execute=True).
Takeaway
Cache invalidation goes wrong mostly in timing: deleting before the commit, missing writes that bypass signals, forgetting copies in other processes. Delete after commit, keep a TTL as the safety net, use generation counters for groups, and write down how stale each piece of data is allowed to be. Then the cache becomes a well-understood trade-off instead of a source of mysterious bugs.
// comments
Loading comments...