django sqlite to postgresql ProgrammingError when migrate

Question

I've seen quite some posts about this but none helped me so far. I have a working django application and I want to go from sqlite to postgresql database. For that, I am following the djangogirls tutorial which is great, until some error occurs.

I have downloaded postgresql on my pc for development, then I downloaded and installed psycopg2 which is working fine, I changed my settings.py DB config, and I am at the point where I want to migrate to postgres.

All they say in the tutorial is to run "python manage.py migrate", which did not work for me, probably because they consider it is a blank application or something, and mine already has migrations and models associated to it.

I read that running

python manage.py makemigrations
python manage.py migrate

would do the trick, but I had a problem with my custom model. I then read that I could comment out all the code that used this problematic model, then run

python manage.py makemigrations  
python manage.py migrate --fake

uncomment, and run

python manage.py migrate

Which I tried, but now I'm getting the same error as my custom model but for the contenttypes app.

The traceback is this:
("la relation 'django_content_type' n'existe pas" = "the relation 'django_content_type' does not exist")

Operations to perform:
  Apply all migrations: auth, contenttypes, sessions, admin
Running migrations:
  No migrations to apply.
Traceback (most recent call last):
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\backends\utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: ERREUR:  la relation « django_content_type » n'existe pas
LINE 1: ..."."app_label", "django_content_type"."model" FROM "django_co...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\contrib\contenttypes\models.py", line 67, in get_for_model
    ct = self.get(app_label=opts.app_label, model=opts.model_name)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\models\manager.py", line 122, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\models\query.py", line 381, in get
    num = len(clone)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\models\query.py", line 240, in __len__
    self._fetch_all()
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\models\query.py", line 1074, in _fetch_all
    self._result_cache = list(self.iterator())
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\models\query.py", line 52, in __iter__
    results = compiler.execute_sql()
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\models\sql\compiler.py", line 848, in execute_sql
    cursor.execute(sql, params)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\backends\utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\backends\utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\utils\six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\db\backends\utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: ERREUR:  la relation « django_content_type » n'existe pas
LINE 1: ..."."app_label", "django_content_type"."model" FROM "django_co...
                                                             ^


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\core\management\__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\core\management\__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\core\management\base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\core\management\base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\core\management\commands\migrate.py", line 204, in handle
    emit_post_migrate_signal(self.verbosity, self.interactive, connection.alias)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\core\management\sql.py", line 50, in emit_post_migrate_signal
    using=db)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\dispatch\dispatcher.py", line 192, in send
    response = receiver(signal=self, sender=sender, **named)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\contrib\auth\management\__init__.py", line 85, in create_permissions
    ctype = ContentType.objects.db_manager(using).get_for_model(klass)
  File "C:\Users\gbastien1\Envs\django-carte\lib\site-packages\django\contrib\contenttypes\models.py", line 80, in get_for_model
    "Error creating new content types. Please make sure contenttypes "
RuntimeError: Error creating new content types. Please make sure contenttypes is migrated before trying to migrate apps individually.

My migrations look like this:

admin
 [X] 0001_initial
 [X] 0002_logentry_remove_auto_add
auth
 [X] 0001_initial
 [X] 0002_alter_permission_name_max_length
 [X] 0003_alter_user_email_max_length
 [X] 0004_alter_user_username_opts
 [X] 0005_alter_user_last_login_null
 [X] 0006_require_contenttypes_0002
 [X] 0007_alter_validators_add_error_messages
contenttypes
 [X] 0001_initial
 [X] 0002_remove_content_type_name
sessions
 [X] 0001_initial

I have no idea how to solve this problem since I tried so many things. And databases and deployment are not my strong points. Can somebody help me figure this out please?

EDIT I tried reverting all my changes to the part where my application worked and start over by deleting my sqlite file and all my migrations, I deleted all my pyc (python cache) files, commented out all the code related to the Ecole model, and then migrate again, and I still got the same error :(

Do you want to keep your data from sqlite? And what does `python manage.py migrate contenttypes` do? — Yash Tewari, Jul 06 '16 at 18:01
I don't mind losing my data from sqlite, and that command outputs the same error as mentioned above :/ — Gabrielle Bastien, Jul 07 '16 at 14:43
You can edit the Question instead of adding new info in answers. — marcanuy, Jul 08 '16 at 12:41

score 0 · Answer 1 · edited May 23 '17 at 12:33

0

You could dump your data into json, remove your sqlite file then migrate your project with postgres set up as a DB.

Example workflow might look like:

mkdir <app name>/fixtures

python manage.py dumpdata >> <app name>/fixtures/all_data.json

rm <sqlite file>.db

python manage.py migrate

python manage.py loaddata <app name>/fixtures/all_data.json

Where <app name> is the name of your main app (or any directory really) and <sqlite file> is the name of your sqlite file.

edited May 23 '17 at 12:33

Community

1
1

answered Jul 06 '16 at 18:17

marcusshep

1,916
2
18
31

This doesn't work any more then the other way sadly :( When I try to dumpdata, I get an error similar to the one I mentioned: `CommandError: Unable to serialize database: Error: relation "django_content_type" doesn't exist` – Gabrielle Bastien Jul 06 '16 at 19:39
Do you need to keep your data – marcusshep Jul 06 '16 at 19:40
Well, now that I think about it, not really, because I load my data from an Excel file on the load of the application. – Gabrielle Bastien Jul 07 '16 at 12:54
Then you are safe to delete your sqlite file. Once you delete it you should have no problem migrating with your postgres DB. If I have helped you at all you could help me out by accepting this as an answer and/or upvoting :) – marcusshep Jul 07 '16 at 12:55
I'll try that, but I tried something similar by deleting all migrations and sqlite file and I still got the same error, like I should undo all the applied migrations.. I'll try to revert to my latest commit and start over. I'll get back to you, but how can I upvote or accept a comment as an answer? :P – Gabrielle Bastien Jul 07 '16 at 13:00

Gabrielle Bastien · Answer 2 · 2016-07-07T20:08:35.927

Well, finally I found one small post on stackoverflow that said that psycopg2 should be in INSTALLED_APPS in my settings.py. I added it a ran again

python manage.py migrate

and it worked fine! Thanks to those who answered though, it's really appreciated!

Though now I get the error I got before commenting out all the code related to the "Ecole" model. It is very similar to the contentypes one:

programmingError: ERROR: relation "carte_interactive_ecole" does not exist

Where "carte_interactive" is my app name, and "Ecole" my model.

Do anyone have yet another idea for me? Why can't I migrate my model when there is code that references it ? If I comment out all the Ecole code, it's fine...

EDIT
I found that if I comment out the code in apps.py ready() I can migrate my application, but it probably does not migrate completely since when I run my application and I do something that uses the database, I get the same error as above. :/

Try deleted everything from your `migrations` folders, except the `__init__.py` files. Then run `python manage.py makemigrations` again, and then `python manage.py migrate`. — Yash Tewari, Jul 08 '16 at 04:35

Gabrielle Bastien · Answer 3 · 2016-07-08T12:42:12.380

So I post this as an answer because it is too long for a comment:

I commented out the code in apps.py that uses my model (it probably uses it before migrating so the model does not exist yet. Then I ran makemigrations, and I got:

Migrations for 'carte_interactive':  
    001_initial.py:  
        -Create model Ecole  
        -Create model ExcelFile

Which are my two models and seems fine.

Then I ran migrate, and got

apply all migrations: admin, carte_interactive, [...]  
Running Migrations: No migrations to apply.

And when I showmigrations, everything is checked so everything is done it seems, though I tried yesterday with --fake so it might be that it checked them but did not create some tables...

EDIT (SOLUTION)
OK so the whole problem was first that psycopg2 was not included in my INSTALLED_APPS in settings.py, so the tables could not be created in postgresql or something.

And also, I started over the migrations by deleting the database completely and recreated it (like dropping all the tables but easier). Then, the other problem was that the code in apps.py ran before the migrations, so I had to comment it before running migrations. After all that, running makemigrations and migrate with the commented code, and then uncomment and run the application worked like a charm! Thanks to everyone who answered, really appreciated!

score 0 · Answer 4 · answered Dec 11 '21 at 18:12

I am posting this in hopes it saves someone a bunch of time. I know this is an old post but I had encountered this problem just last night. while some of the above solutions work for some people I was finding that none of them worked for me. What I ended up doing was double-checking that the database I was using on my local machine was not in conflict with another database table.

turns out the naming conventions of another owners database was conflicting with my migrations. even though the databases were named different the tables shared the same convention. Django(2.2) had an issue with it resolving the migrations. I think this is because the migrations table is in the public schema so all databases in the server instance can reference the same table when running migrations. this is what caused me my issue.

So my solution was to set up another instance of postgres and stop the old instance of Postgres that was being used and configure a fresh database.

once this was done the migrations were applied without any of the above solutions needed. Very strange for two separate databases and projects to conflict like this because of similar names to their models. that is what i think happened.

django sqlite to postgresql ProgrammingError when migrate

4 Answers4