Caution About Faking Django Migrations

TypeError: expected string or bytes-like object

What is This TypeError?

Honestly, after my research, I’m not totally sure. I believe Django was trying to put a NULL value in that DateTime table column, but since the migration hadn’t been run, NULL values were still not allowed.

I found something that made my error message go away… but also introduced a bug.

Django’s Migrations Table

Django’s migrations actually create a table in your database where database updates are tracked. Here’s a look at that table from one of my newer projects:

postgres=# \d django_migrations
                                     Table "public.django_migrations"
 Column  |           Type           | Collation | Nullable |                    Default
---------+--------------------------+-----------+----------+-----------------------------------------------
 id      | integer                  |           | not null | nextval('django_migrations_id_seq'::regclass)
 app     | character varying(255)   |           | not null |
 name    | character varying(255)   |           | not null |
 applied | timestamp with time zone |           | not null |
Indexes:
    "django_migrations_pkey" PRIMARY KEY, btree (id)

At least in my simplified view of Django migrations, there are two things that need to happen:

1. Django needs to know the migration happened so it doesn’t try to repeat actions
2. The database actually needs to be updated with SQL commands

This django_migrations table tells Django what has been done.

Faking Migrations

django-admin and manage.py have --fake flag you can add to a migrate command. This argument tells Django that a migration has happened, but DOES NOT RUN IT. That is, it creates a row in the django_migrations table, but does not run the SQL to change the database structure. Here’s a link to the docs.

And here’s another link to an article on Django Migrations from Real Python.

So I ran the command:

python manage.py migrate app_name --fake

And I saw no errors! Yay!

But I had a sneaking suspicion I should do more digging…

Checking on the Table

Since this was the first time I’ve run the migrate --fake command, and the docs clearly state, “without actually running the SQL to change your database schema,” I figured there was a good chance my DateTime field would still not accept null values.

Run psql to see the command line interface:

python manage.py dbshell

or, if you’re using Docker Compose on a Linux VM like me…

sudo docker-compose exec db_service_name psql -U postgres_username

When in psql, your prompt should change to something like postgres=#. Let’s inspect the table.

Get a list of all tables:

postgres=# \d

Then inspect the appropriate table. Mine was a Mail models inside an app named “mail”, so my table was named mail_mail.

postgres=# \d mail_mail
                                        Table "public.mail_mail"
    Column    |           Type           | Collation | Nullable |                Default
--------------+--------------------------+-----------+----------+---------------------------------------
 id           | integer                  |           | not null | nextval('mail_mail_id_seq'::regclass)
 to_email     | character varying(254)   |           | not null |
 subject      | character varying(78)    |           | not null |
 body_text    | text                     |           | not null |
 body_html    | text                     |           | not null |
 created      | timestamp with time zone |           | not null |
 sent         | timestamp with time zone |           | not null |
 content_id   | integer                  |           | not null |
 recipient_id | integer                  |           | not null |

The DateTime field I was trying to allow NULL values in is called “sent”, describing when a mail was sent. The NULL value represents that it has not been sent, otherwise a DateTime value represents when it was sent.

If you notice up there, the “Nullable” column in our psql output tells us that “sent” cannot be null.

If we don’t change this, we’ll run into trouble.

Revert the Table

So again, I’m confused as to why this would happen considering I didn’t have any data in the table. But no data means we can start from scratch.

First, unapply your migrations:

python manage.py migrate mail zero

Then reapply migrations:

python manage.py migrate mail

If you have trouble, unapply back to “zero” and delete the migrations files in your app_name/migrations/ directory. These files usually look like “0001_initial.py”. Then you can makemigrations again, then re-try migrate.

Check Database to Confirm Success

Alright, let’s go back to psql and see if our migration was actually applied this time.

postgres=# \d mail_mail
                                        Table "public.mail_mail"
    Column    |           Type           | Collation | Nullable |                Default
--------------+--------------------------+-----------+----------+---------------------------------------
 id           | integer                  |           | not null | nextval('mail_mail_id_seq'::regclass)
 to_email     | character varying(254)   |           | not null |
 subject      | character varying(78)    |           | not null |
 body_text    | text                     |           | not null |
 body_html    | text                     |           | not null |
 created      | timestamp with time zone |           | not null |
 content_id   | integer                  |           | not null |
 recipient_id | integer                  |           | not null |
 sent         | timestamp with time zone |           |          |

Hey! The “sent” column is updated (as evidenced by the absence of “not null”).

Conclusion

Hopefully you’ve taken in the word of caution about using --fake in your Django migrations. And hey, maybe you even learned a few ways to use psql in your project!