I recently added sorl-thumbnail to a project for creating smaller image files. I chose to configure with a Redis Key-Value Store, as I’ve heard Redis is super hip, but ran into trouble during deployment.
In this post, I’ll document how I set up Redis locally with Docker, then in production on Heroku.
Local Docker Setup
The logical place to start is with the sorl-thumbnail documentation.
Install the package
python3 -m pip install sorl-thumbnail
Add package to settings.py
# settings.py INSTALLED_APPS += ['sorl.thumbnail']
python manage.py migrate
This should create a model called KVStore.
Add redis server
If we navigate to the documentation’s requirements page, we’ll see we need some sort of key-value store and image processing library.
As the title suggests, we’re going to use Redis. Let’s add that in our
# docker-compose.yml version: "3.7" services: web: build: ./django command: python /code/manage.py runserver 0.0.0.0:8000 env_file: - ./.env.dev volumes: - ./django:/code ports: - 8000:8000 - 443:443 depends_on: - db - redis db: image: postgres volumes: - postgres_data:/var/lib/postgresql environment: - POSTGRES_HOST_AUTH_METHOD=trust - POSTGRES_USER=username - POSTGRES_PASS=supercomplexpasswordthatishardtocrack - POSTGRES_DBNAME=pg ports: - "5432:5432" redis: restart: always image: redis:latest ports: - "6379:6379" volumes: - redis_data:/var/lib/redis volumes: postgres_data: redis_data: networks: default:
I’m assuming here that your Django web and db services and volumes are already configured. Use this tutorial to get started with Docker, Django, and Postgres.
First, we’re adding a redis service (
redis:) and volume (
redis_data). Then, we’re telling the web service that we need redis for things to work properly.
Add Redis client
python3 -m pip install redis
Here’s a link to this redis-py project on Github.
Add a Python image library
If you’ve gotten to this page, you probably already have Pillow installed, but you can use a few different image libraries.
Connect Redis server and client
Lastly, sorl-thumbnail has some settings to configure to link it up to your Redis key-value store.
# settings.py THUMBNAIL_KVSTORE = 'sorl.thumbnail.kvstores.redis_kvstore.KVStore'
But if we try to
docker-compose up with just this settings, we receive an error from Django:
Connection error. Error 99 connecting to localhost:6379. Cannot assign requested address.
The sorl-thumbnail package tries to use some sensible default values for connecting to the Redis server:
The problem is that we called our Redis host redis, so sorl-thumbnail cannot connect.
I like to be explicit about variables at the expense of verbosity. Here’s all my sorl-thumbnail settings:
# settings.py # sorl-thumbnail THUMBNAIL_KVSTORE = 'sorl.thumbnail.kvstores.redis_kvstore.KVStore' THUMBNAIL_REDIS_DB = 0 # the default THUMBNAIL_REDIS_HOST = 'redis' THUMBNAIL_REDIS_PORT = 6379 # the default
Then you should be all set locally!
Heroku Production Setup
Production is a little different, but the principles still apply:
- Redis server
- Redis client
- Connect sorl-thumbnail to server
Sorl-thumbnail uses the redis-py package (the Redis client) to connect to the Redis server. At the time of writing, sorl-thumbnail’s reference documentation only showed our above settings as configurable.
But I went and looked at the code and found a nice little secret! We’ll use it later.
Provision heroku-redis add-on
The Heroku documentation shows you how to create a new heroku-redis add-on from the command line or Heroku dashboard.
heroku addons:create heroku-redis:hobby-dev -a your-app-name
The hobby-dev tier is free.
This will add a
REDIS_URL configuration or environment variable to your Heroku app. It should look like “redis://…”.
But we didn’t use this variable to connect our project to Redis in our local environment. How will we use REDIS_URL?
That’s the secret I just mentioned at the end of the previous section!
Configure Django settings to use
Here’s a link to the sorl-thumbnail code used to connect to Redis. It first searches for a
THUMBNAIL_REDIS_URL environment variable, which contains the Redis username, password, host, and port.
We want to use the
REDIS_URL variable set by heroku-redis because that URL might change in the future without us knowing, but will always be stored as
We need to replace our previous Django settings for sorl-thumbnail and Redis connections.
# sorl-thumbnail THUMBNAIL_KVSTORE = 'sorl.thumbnail.kvstores.redis_kvstore.KVStore' THUMBNAIL_REDIS_URL = os.environ.get('REDIS_URL')
And that should be all! Hopefully everything works for you on your first try and you feel like a genius.
I’ve opened an issue for including the
THUMBNAIL_REDIS_URL in the Reference documentation.
Overall lesson today is to keep your eye out for strange anomalies like this. Sometimes there is a way to do what you need to do, but someone hasn’t gotten around to documenting it yet.
Maybe that someone could be you!