15

I am trying to bring a Django project from version 1.8 to 1.11. Pretty much everything seems to work fine except unit tests. We have a base test class inheriting from Django TestCase with a Tastypie mixin. The base class has some code in the setUp() like this

class BaseApiTest(ResourceTestCaseMixin, django.test.TestCase):
    def setUp(self):
        super().setUp()
        self.username = "secret_user"
        self.password = "sekret"
        self.email = "secret@mail.com"
        self.first_name = "FirstName"
        self.last_name = "LastName"
        self.user = User.objects.create_superuser(
            self.username,
            self.username,
            self.password
        )

And the app specific tests would inherit the base test and do something like

class TheAPITest(BaseApiTest):
    def setUp(self):
        super().setUp()
        # more setup goes here

So, under Django 1.8.x this works fine. But under 1.11.x all of these give me an error on the User.objects.create_superuser() line.

django.db.utils.InterfaceError: connection already closed

I have been going through the release notes, but there is just too much stuff that has happened between 1.8 and 1.11. Is there something simple that I am missing?

Jaffer Wilson
  • 7,029
  • 10
  • 62
  • 139
Mad Wombat
  • 14,490
  • 14
  • 73
  • 109
  • Few questions, which DB are you using? Python version? And sample project we can use to test? – Tarun Lalwani Aug 07 '17 at 17:38
  • 1
    Postgres 9.5 and python 3.5.x – Mad Wombat Aug 07 '17 at 17:41
  • This is a commercial project, so I am not at liberty to share the original source. Not sure I can cut out a sample project from the codebase. – Mad Wombat Aug 07 '17 at 17:42
  • Any sample template you can give I can use to reproduce the issue? – Tarun Lalwani Aug 07 '17 at 17:42
  • 1
    No need to put much code, smallest possible template which produce error at your end. Because otherwise it would be hard to look into. Could be because of something you use in your app config or a generic one. So without sample template would be hard to look into – Tarun Lalwani Aug 07 '17 at 17:44
  • I will try to setup a minimal example, but I cannot really promise. – Mad Wombat Aug 07 '17 at 17:46
  • No worries, even I can't promise a solution. Just trying to help – Tarun Lalwani Aug 07 '17 at 17:49
  • 1. Try https://docs.djangoproject.com/en/1.11/topics/testing/tools/#transactiontestcase instead of TestCase 2. Try adding tear down methods: https://groups.google.com/forum/#!topic/django-users/MDRcg4Fur98 – Michael Aug 07 '17 at 18:55
  • 1
    I tried it with the TransactionTestCase instead of test case. Same result. – Mad Wombat Aug 07 '17 at 18:58
  • And the base class has a tearDown() method, so that doesn't work either – Mad Wombat Aug 07 '17 at 20:22
  • 2
    It seems that I am finally getting to the bottom of this. Will post an answer when I get every test to work. TL;DR 1) if some test fails hard enough database gets closed and does not reopen. And every test after that gets the error I posted above 2) Django 1.11 is much stricter about creating model instances than 1.8. 1.8 allows you to pass arguments into create() that do not correspond to any fields and apparently we have had a bunch of that in our tests. – Mad Wombat Aug 07 '17 at 20:48
  • I am not sure whether this helps. Can you take a look at this https://docs.djangoproject.com/en/1.9/topics/testing/tools/#django.test.TransactionTestCase – Hardyanto Putra Antoni Aug 11 '17 at 06:42
  • Can you share your error log of PostGre and the Django, if any. This will help to diagnose the proper problem. – Jaffer Wilson Aug 11 '17 at 06:43

2 Answers2

8

I have uncovered multiple issues with my unit test coverage, but to answer the question I posted, two separate issues were causing the errors I described.

Django 1.11 is stricter about its model instance creation and we had some legacy test code that was not updated to new model structure. For example, if you take the default User model, in Django 1.8 you can do 

from django.contrib.auth.models import User    
User.objects.create(username="something", password="something", something="something")

But in Django 1.11 it will raise an exception "TypeError: 'something' is an invalid keyword argument for this function".

And the second issue is that Django TestCase wraps test cases in two separate atomic blocks. This helps with keeping class level setup separate from instance level setup, but it also introduces a subtle problem. If a test crashes with a database error you never see the error because it happens in the test level atomic and not in the class level atomic. The test level atomic fails and database connection gets dropped. After that every test is going to fail with the exact django.db.utils.InterfaceError: connection already closed error I was seeing. Switching from TestCase to TransactionTestCase caused a lot of these problems to be exposed directly in the test output.

Mad Wombat
  • 14,490
  • 14
  • 73
  • 109
3

It may be useful to have the error trace. But for the moment in the code I see this error:

   self.user = User.objects.create_superuser(
                self.username,
                self.username, **this should be self.email**
                self.password
            )
wololoooo
  • 134
  • 8