Caching in Django to improve site performance

June 23, 2022 - by Themba Mahlangu - 7 min read

In this article, you will learn to make your Django website faster using caching. We will cover the core concepts of caching, cache stores, and different caching strategies in Django. We will briefly look at techniques for invalidating your cache.

¶

Prerequisites

Code

Basic knowledge of Django

LEVEL -

💻

Beginner - Intermediate - We will cover basic concepts and add some examples. This post includes some tips at the end for advanced users.

¶

Caching and Dynamic Frameworks like Django

A cache is a temporary storage area. Caching is the process of storing data in a cache. The objective is to save the result of an expensive calculation so you don’t have to save it later.

A downside of dynamic frameworks like Django is that each time user requests a page, the webserver needs to make a number of calculations such as making database queries, template rendering, and other business logic. For small websites this is fine, but for bigger sites with higher traffic, it is important to reduce this work as much as possible to improve page load times for your users.

Fortunately, Django has a robust, built-in cache system that lets you control which pages, views or fragments should be saved so they do not have to be calculated for each request.

¶

Where should cached data be stored

Cached data can be stored in a database, filesystem, or directly in memory. Memory-based stores like

Memcached

&

Redis

will usually be the fastest and can handle high loads. You can also use database caching but this works best if you have a fast well-indexed database server. The choice of storage is important as it affects cache performance. See the

Django docs

for more information on this.

¶

Setting up the cache in Django

To set up the cache system in Django, add your cache preferences to the CACHES setting in your settings file. We will be using Redis as the cache database in this example.

Code

 text

Code

`CACHES = {
    'default': {
        'BACKEND':'django.core.cache.backends.redis.RedisCache',
        'LOCATION':'redis://127.0.0.1:6739'
     }
}`

Notes

BACKEND
- we are using the RedisCache backend. This backend is only supported natively in Django 4. For previous versions, you will need to install the

django-redis

library.

LOCATION
- the URL of the Redis server. This can be on the same instance as the app or a remote server.

¶

How to use caching

¶

Per site cache

The simplest way to use caching is to

cache the entire site

. This is done by adding appropriate middleware in the Middleware setting. This blanket approach may not be appropriate for all sites.

Code

 text

Code

`MIDDLEWARE = [
     'django.middleware.cache.UpdateCacheMiddleWare',
     'django.middleware.cache.CommonMiddleware',
     'django.middleware.cache.FetchFromCacheMiddle',     
]
`

Notes

Code

The
[
 order of the middleware
](https://docs.djangoproject.com/en/4.0/topics/cache/#order-of-middleware)
matters because each middleware runs at different parts of the request, response cycle. During the request phase, the middleware is run from top to bottom, this is when FetchFromCacheMiddleware will run. During the response phase, the middleware is run from bottom to top, therefore UpdateCacheMiddleware must be on top as it should be run last.

You can also customize the
`
 CACHE_MIDDLEWARE_ALIAS
`
,
`
 CACHE_MIDDLEWARE_KEY_PREFIX
`
and
`
 CACHE_MIDDLEWARE_SECONDS
`
flags to your preferences in the same file.

¶

Per view caching

Use this to cache the output of individual views. For example, you could use this to cache the output of your landing pages such as the home page, pricing page, and other marketing pages where site speed is important.

* views.py
*

Code

 text

Code

`from django.views.decorators.cache import cache_page

@cache_page(60 * 15)
def pricing_view(request):
    some expensive calculations here

`

* urls.py
*

Code

 text

Code

`from django.views.decorators.cache import cache_page

urlpatterns = [

    path('pricing/', pricing_view),
    path('home/', cache_page(60 * 60 * 24, cache="marketing")(TemplateView.as_view(template_name="home.html"))
]
`

Notes

Code

views.py cache_page - Django provides a cache decorator that will automatically cache the response for your. The pricing_view in this example is cached for 15 minutes.

urls.py - the decorator can also be used on Class based views.

options - the cache_page decorator accepts additional arguments;

  cache for specifying which cache to use.

  key_prefix to be added to the cache key.

¶

Template fragment caching.

Fragments of a template can also be cached using the cache template tag. The cache template tag caches the contents of a section for a given period of time.

Code

 text

Code

`{% load cache %}
{% cache 500 profile_card request.user.email %}
    .. profile card..
{% endcache %}`

Notes

Code

load cache - the template tag first has to be loaded

{% cache 500 .. %}
- the cache template tag accepts at least arguments. The timeout in seconds, and the name of the fragment. In this example, the profile card will be cached separately for each user.

¶

Low-level cache API

Django exposes a low-level cache API for when you need more control on what to cache. For example when you want to refresh the cache at specified intervals.

Code

 text

Code

`from django.core.cache import cache

def get_expensive_calc_data(force_refresh=False):
    data = cache.get('my_key')
    if data is None or force_refresh:
        data = calculate_expensive_thing here
        cache.set('my_key', data, timeout=3000)
    return data

def my_view(request):
    data = get_expensive_calc_data()
    ...`

Notes

get_expensive_calc_data
- fetches data from cache if it exists, else does the calculation and saves it to cache storage for future requests.

force_refresh
- The force_refresh flag can be set to true to invalidate/refresh the cached results.

cache.set(..
- This method expects the key to be a string and the value can be any pickable Python object. The timeout argument is optional and defaults to the timeout set in the CACHES setting explained above.

¶

Invalidating or refreshing your cache

In this next section, we will look at techniques for refreshing your cached content. Please note that this only refers to your own data, i.e. data that is returned from the webserver and not “

downstream

” caches like a proxy cache or browser.

¶

Automated refresh using the timeout flag

Django will automatically invalidate the cache at specific intervals depending on the timeout seconds specified. Specifying a timeout can be done at all levels of caching i.e. site, view, template fragment, or low-level API.

¶

Low level API and template fragments

To refresh the cache manually you will have to know the cache key beforehand. If you are using the low-level API this is straightforward since you set the key yourself.

For template fragment caching, Django provides a method to retrieve a fragment key.

Code

 text

Code

`from django.core.cache import cache
from django.core.cache.utils import make_template_fragment_key

def invalidate_cached_template_fragment(key:str, vary_on:List) -&gt; bool:
    key = make_template_fragment_key(key, vary_on)
    return cache.delete(key)

# use it
invalidate_cached_template_fragment('profile_card', [user.email])`

Notes

invalidate_cached_template_fragment
- in the spirit of keeping things

DRY

, we make a small function we can call from anywhere to invalidate any template fragment (DRY). It will return
True,
if the key was deleted or
False
if not.

¶

Invalidating cached view content

This one is a bit more complex as cache keys for cached views are generated based on request URL, query, and headers. Django provides a method to get a key based on the request URL and query, but we need to construct a fake request object to fetch the correct key.

* utils.py
*

Code

 text

Code

`from typing import Dict, Optional

from django.conf import settings
from django.core.handlers.wsgi import WSGIRequest

def fake_request(path, vary_headers: Optional[Dict[str, str]] = None,):
    """
    Construct a request object, can be used to generate a valid cache key
    :param path: object base path
    :param vary_headers:
    :return: request
    """
    request = WSGIRequest(
        {
            "PATH_INFO": path,
            "REQUEST_METHOD": "GET",
            "HTTP_HOST": settings.HOST,
            "wsgi.input": None,
            "wsgi.url_scheme": "http" if settings.DEBUG else "https"
        }
    )
    if vary_headers:
        request.META.update(vary_headers)
    return request`

Notes

Code

fake_request - constructs a fake request that mimics a real WSGIRequest object.

path - the path for the view for example 'home/'.

settings.HOST - this host should be specified in your settings file. This is your naked domain test.com or including the port if in your dev environment localhost:8000.

vary_headers - a vary header defines which request headers a cache mechanism should take into account when building the cache key. You can read more about it
[
 here
](https://docs.djangoproject.com/en/4.0/topics/cache/#using-vary-headers)
.

Now we can use Django get_cache_key util to retrieve and invalidate the cache for a view.

Code

 text

Code

`from django.utils.cache import get_cache_key
from .utils import fake_request

path = 'home/'
request = fake_request(path)
key = cache.get(key)
cache.delete(key)
 `

Notes

Code

This part is simple. We construct the request using our method from before, fetch and delete the key.

You may want to automate the last bit, for example, you can use a management command to wrap up this logic and then trigger it via a cron, webhook, or some other event in your app.

¶

Conclusion

That’s it. In this tutorial, we have covered the basics of caching, cache stores, and different caching strategies for caching in Django. We have also looked at techniques for refreshing the cache manually.