Lack of Volume Persistence in kartoza/postgis Docker Container

Starting Point

My knowledge of Docker comes entirely from Will Vincent’s Django for Professionals book. In there, he builds from the official postgres Docker image within a docker-compose.yml file.

version: "3.7"

services:
  web:
    build: .
    command: python /code/manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - 8000:8000
    depends_on:
      - db
  db:
    image: postgres:11
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - "POSTGRES_HOST_AUTH_METHOD=trust"

volumes:
  postgres_data:

Learning to Use GeoDjango

Luckily, I found a great tutorial over at Real Python to introduce myself to GeoDjango: “Make a Location-Based Web App With Django and GeoDjango” by Ahmed Bouchefra. If you’re totally new to GeoDjango, I highly recommend this article.

Ahmed even uses Docker in the article, so super win if you’re coming from Vincent’s book like me. PostGIS is an extension of PostgreSQL, so it’s kind of like a standard postgres database you might use for your Django project, but adds support for geographic objects.

In the article, we use a new PostgreSQL image to build our database: the kartoza/postgis image. If you check out the tags, you’ll see some have TWO sets of numbers. These are versions for PostgreSQL and PostGIS. Ex: kartoza/postgis:11.5-2.5 uses PostgreSQL 11.5 and PostGIS 2.5.

We spin up this database with the following command in the terminal:

$ docker run --name=postgis -d -e POSTGRES_USER=user001 -e POSTGRES_PASS=123456789 -e POSTGRES_DBNAME=gis -p 5432:5432 kartoza/postgis:9.6-2.4

What’s all this mean?

• docker run means let’s run a Docker command
• --name is the container’s name
• -d means detach this container from the terminal so I don’t have to open another one
• -e are environment variables
• -p publishes a container’s port to the host
• and the last kartoza/postgis:9.6-2.4 is the name of the image from Docker Hub you want to use to build the container

Once that’s running, the rest of the stuff eventually works and it’s great! All is well in the world.

The tutorial, however, doesn’t show me how to use it with my docker-compose.yml file. So let’s dive into that.

Using docker-compose.yml

To get this part going, I started with the PostgreSQL image we made in Django for Professionals. The file we had before becomes…

version: "3.7"

services:
  web:
    build: .
    command: python /code/manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - 8000:8000
    depends_on:
      - db
  db:
    # NOTE the new image
    image: kartoza/postgis:11.5-2.5
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - "POSTGRES_HOST_AUTH_METHOD=trust"
      # NOTE the new environment variables
      - POSTGRES_USER=user001
      - POSTGRES_PASS=123456789
      - POSTGRES_DBNAME=gis

volumes:
  postgres_data:

Bring it all up with docker-compose up -d --build and things should work! Everything appears to be fine until you bring the containers down and back up again.

docker-compose down

docker-compose up -d --build

Our docker logs show us an error message Django is trying to tell us:

django.db.utils.OperationalError: could not connect to server: Connection refused

Is the server running on host "db" (192.168.96.2) and accepting TCP/IP connections on port 5432?

I had previously enabled python manage.py runsslserver so that I could have an SSL connection to work with some public APIs. When I ran the Django server with runsslserver, I get the beautiful Django DEBUG screen:

Note that the exception is in the “django/db/backends/utils.py” file. It’s a database problem. The database is gone! We just need to migrate it.

python manage.py migrate

or

docker-compose exec web python manage.py migrate (for our docker-compose Django service named “web”.

The terminal shows a ton of green “OK” notifications as it applies your database migrations. Check the site now and the error should be gone.

But now the main point of the article. Let’s bring the containers down and then up again.

docker-compose down

docker-compose up -d --build

And that error is still there!

Retain the Local kartoza/postgis Database

I started digging into why the database wouldn’t stick around. It worked when I used the official postgres image, but not with this new one.

If you look at the Docker Hub page for the postgres image, they say:

The default [database file location] is /var/lib/postgresql/data.

That’s already where we store it! But what does the kartoza/postgis image say?

Docker volumes can be used to persist your data.

mkdir -p ~/postgres_data

docker run -d -v $HOME/postgres_data:/var/lib/postgresql kartoza/postgis

Ahhhhh, different location! Let’s try that out. Updated docker-compose.yml file:

version: "3.7"

services:
  web:
    build: .
    command: python /code/manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    ports:
      - 8000:8000
    depends_on:
      - db
  db:
    image: kartoza/postgis:11.5-2.5
    volumes:
      # NOTE the new storage location
      - postgres_data:/var/lib/postgresql
    environment:
      - "POSTGRES_HOST_AUTH_METHOD=trust"
      # NOTE the new environment variables
      # NOTE use your correct credentials
      - POSTGRES_USER=user001
      - POSTGRES_PASS=123456789
      - POSTGRES_DBNAME=gis

volumes:
  postgres_data:

Migrate the database, bring the containers down, then back up.

If you’re lucky, it worked!

Conclusion

This whole scenario is a great example of how touchy coding can be. You need to specify the right folder. If you can’t quite appreciate that, start tinkering around with Linux and file permissions. HUGE PAIN. But valuable experience, too.