Python unit test with base and sub class

Question

I currently have a few unit tests which share a common set of tests. Here's an example:

import unittest

class BaseTest(unittest.TestCase):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

The output of the above is:

Calling BaseTest:testCommon
.Calling BaseTest:testCommon
.Calling SubTest1:testSub1
.Calling BaseTest:testCommon
.Calling SubTest2:testSub2
.
----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK

Is there a way to rewrite the above so that the very first testCommon is not called?

EDIT: Instead of running 5 tests above, I want it to run only 4 tests, 2 from the SubTest1 and another 2 from SubTest2. It seems that Python unittest is running the original BaseTest on its own and I need a mechanism to prevent that from happening.

I see noone has mentioned it but do you have the option to change main part and run a test suite that has all subclasses of BaseTest? — kon psych, Sep 27 '15 at 04:28
Is there still no great solution for this in 2022? Multiple inheritance is awkward and leads to linting issues. `setUpClass` with raising SkipTest is pretty good but the test runner shows skipped tests. Other frameworks solve these kinds of issues by adding an `__abstract__ = True`. Is there no clean way to do this still? — Matthew Moisen, Feb 28 '22 at 00:52

score 199 · Answer 1 · edited Jun 17 '20 at 14:06

199

Do not use multiple inheritance, it will bite you later.

Instead you can just move your base class into the separate module or wrap it with the blank class:

class BaseTestCases:

    class BaseTest(unittest.TestCase):

        def testCommon(self):
            print('Calling BaseTest:testCommon')
            value = 5
            self.assertEqual(value, 5)


class SubTest1(BaseTestCases.BaseTest):

    def testSub1(self):
        print('Calling SubTest1:testSub1')
        sub = 3
        self.assertEqual(sub, 3)


class SubTest2(BaseTestCases.BaseTest):

    def testSub2(self):
        print('Calling SubTest2:testSub2')
        sub = 4
        self.assertEqual(sub, 4)

if __name__ == '__main__':
    unittest.main()

The output:

Calling BaseTest:testCommon
.Calling SubTest1:testSub1
.Calling BaseTest:testCommon
.Calling SubTest2:testSub2
.
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

edited Jun 17 '20 at 14:06

Scott Skiles

3,647
6
40
64

answered Sep 05 '14 at 23:58

Vadim P.

2,255
1
16
14

9

This is my favorite. It is the least hacky means and doesn't interfere with overriding methods, doesn't alter the MRO and allows me to define setUp, setUpClass etc. in the base class. – Hannes May 27 '15 at 22:18
6

I seriously don't get it (where does the magic come from ?), but it's far the best solution according to me :) Coming from Java, I hate Multiple Inheritance... – Edouard Berthe Feb 24 '16 at 05:46
4

@Edouardb unittest only runs module-level classes that inherit from TestCase. But BaseTest is not module-level. – JoshB Aug 16 '16 at 12:21
As a very similar alternative, you could define the ABC inside a no-args function that returns the ABC when called – Anakhand May 01 '20 at 11:00
This is a nice solution as long as unittest behaves like this (only running module-level classes). A more future-proof solution is to override the `run(self, result=None)` method in BaseTest: `if self.__class__ is BaseTest: return result; else: return super().run(result)`. – interDist Aug 31 '23 at 06:50

score 168 · Accepted Answer · answered Aug 24 '09 at 17:00

168

Use multiple inheritance, so your class with common tests doesn't itself inherit from TestCase.

import unittest

class CommonTests(object):
    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(unittest.TestCase, CommonTests):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(unittest.TestCase, CommonTests):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

answered Aug 24 '09 at 17:00

Matthew Marshall

5,793
2
21
14

30

This method only works for setUp and tearDown methods if you reverse the order of the base classes. Because the methods are defined in unittest.TestCase, and they don't call super(), then any setUp and tearDown methods in CommonTests need to be first in the MRO, or they won't be called at all. – Ian Clelland Oct 11 '10 at 16:35
36

Just to clarify Ian Clelland's remark so that it will be clearer for people like me: if you add `setUp` and `tearDown` methods to `CommonTests` class, and you want them to be called for each test in derived classes, you have to reverse the order of the base classes, so that it will be: `class SubTest1(CommonTests, unittest.TestCase)`. – Dennis Golomazov Jul 17 '13 at 09:29
10

I'm not really a fan of this approach. This establishes a contract in the code that classes must inherit from both `unittest.TestCase` *and* `CommonTests`. I think the `setUpClass` method below is the best and is less prone to human error. Either that or wrapping the BaseTest class in a container class which is a bit more hacky but avoids the skip message in the test run printout. – David Sanders Oct 17 '14 at 16:23
14

The problem with this one is pylint has a fit because `CommonTests` is invoking methods which don't exist in that class. – MadScientist Sep 18 '15 at 03:07
1

I like this one the best. I argue CommonTests should not inherit from TestCase because it is not really a test case (i.e. you don't want it to run with your other tests). If you really don't like the "contract" that requires inheritance from both, consider a MixIn: `class CommonMixIn( CommonTests, unittest.TestCase): – CodeJockey Apr 10 '17 at 17:27
i had to switch the order of the multiple inheritance to get my `CommonTests.tearDown()` to run. not sure why. – Trevor Boyd Smith Sep 26 '17 at 18:49
Don't do this, use the 'blank class wrap' trick in https://stackoverflow.com/a/25695512/328384 – harmv Feb 01 '19 at 11:07

score 39 · Answer 3 · answered Apr 03 '14 at 11:20

39

You can solve this problem with a single command:

del(BaseTest)

So the code would look like this:

import unittest

class BaseTest(unittest.TestCase):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

del(BaseTest)

if __name__ == '__main__':
    unittest.main()

answered Apr 03 '14 at 11:20

Wojciech B.

415
4
2

4

BaseTest is a member of the module while it is being defined, so it is available for use as the base class of the SubTests. Just before the definition is complete, del() removes it as a member, so the unittest framework won't find it when it searches for TestCase subclasses in the module. – mhsmith May 05 '15 at 13:51
3

this is an awesome answer! I like it more than @MatthewMarshall 's because in his solution, you'll get syntax errors from pylint, because the `self.assert*` methods do not exist in a standard object. – SimplyKnownAsG May 07 '15 at 17:59
2

Doesn't work if BaseTest is referenced anywhere else in the base class or its subclasses, e.g. when calling super() in method overrides: `super( BaseTest, cls ).setUpClass( )` – Hannes May 27 '15 at 21:57
2

@Hannes At least in python 3, `BaseTest` can be referenced through `super(self.__class__, self)` or just `super()` in the subclasses, although [apparently not if you were to inherit constructors](http://stackoverflow.com/questions/576169/understanding-python-super-with-init-methods). Maybe there is also such an "anonymous" alternative when the base class needs to reference itself (not that I have any idea when a class needs to reference itself). – Stein Aug 08 '15 at 15:56

score 37 · Answer 4 · answered Jul 17 '13 at 10:01

37

Matthew Marshall's answer is great, but it requires that you inherit from two classes in each of your test cases, which is error-prone. Instead, I use this (python>=2.7):

class BaseTest(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        if cls is BaseTest:
            raise unittest.SkipTest("Skip BaseTest tests, it's a base class")
        super(BaseTest, cls).setUpClass()

answered Jul 17 '13 at 10:01

Dennis Golomazov

16,269
5
73
81

3

That's neat. Is there a way to get around having to use a skip? To me, skips are undesireable and are used to indicate a problem in the current test plan (either with the code or the test)? – Zach Young Feb 05 '14 at 19:29
@ZacharyYoung I don't know, maybe other answers can help. – Dennis Golomazov Feb 06 '14 at 08:49
@ZacharyYoung I've tried to fix this problem, see my answer. – simonzack Jul 24 '14 at 02:16
it's not immediately clear what is inherently error-prone about inheriting from two classes – jwg Apr 27 '15 at 17:34
@jwg see comments to the accepted answer :) You need to inherit each of your test classes from the two base classes; you need to preserve the correct order of them; should you like to add another base test class, you'd need to inherit from it too. There is nothing wrong with mixins, but in this case they can be replaced with a simple skip. – Dennis Golomazov Apr 28 '15 at 09:21
1

I like the approach of checking whether you're running in the BaseTest class. You can use some [other documented TestCase features](https://stackoverflow.com/a/63106148/647002) to avoid the skip result. – medmunds Jul 26 '20 at 21:59

score 12 · Answer 5 · edited Sep 02 '20 at 22:01

You can add __test__ = False in BaseTest class, but if you add it, be aware that you must add __test__ = True in derived classes to be able to run tests.

import unittest

class BaseTest(unittest.TestCase):
    __test__ = False

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(BaseTest):
    __test__ = True

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):
    __test__ = True

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

This solution does not work with unittest's own test discovery/test runner. (I believe it requires using an alternate test runner, like nose.) — medmunds, Jul 26 '20 at 20:37

score 8 · Answer 6 · answered Aug 24 '09 at 16:52

What are you trying to achieve? If you have common test code (assertions, template tests, etc), then place them in methods which aren't prefixed with test so unittest won't load them.

import unittest

class CommonTests(unittest.TestCase):
      def common_assertion(self, foo, bar, baz):
          # whatever common code
          self.assertEqual(foo(bar), baz)

class BaseTest(CommonTests):

    def testCommon(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)

class SubTest1(CommonTests):

    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)

class SubTest2(CommonTests):

    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

if __name__ == '__main__':
    unittest.main()

Under your suggestion, would common_assertion() still be run automatically when testing the subclasses? — Stewart, Apr 23 '15 at 12:12
@Stewart No it would not. The default setting is to only run methods starting with "test". — C S, Aug 27 '18 at 18:30

score 6 · Answer 7 · answered Jul 24 '14 at 02:15

A way I've thought of solving this is by hiding the test methods if the base class is used. This way the tests aren't skipped, so the test results can be green instead of yellow in many test reporting tools.

Compared to the mixin method, ide's like PyCharm won't complain that unit test methods are missing from the base class.

If a base class inherits from this class, it will need to override the setUpClass and tearDownClass methods.

class BaseTest(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        cls._test_methods = []
        if cls is BaseTest:
            for name in dir(cls):
                if name.startswith('test') and callable(getattr(cls, name)):
                    cls._test_methods.append((name, getattr(cls, name)))
                    setattr(cls, name, lambda self: None)

    @classmethod
    def tearDownClass(cls):
        if cls is BaseTest:
            for name, method in cls._test_methods:
                setattr(cls, name, method)
            cls._test_methods = []

score 6 · Answer 8 · edited Sep 17 '14 at 16:34

6

Another option is not to execute

unittest.main()

Instead of that you can use

suite = unittest.TestLoader().loadTestsFromTestCase(TestClass)
unittest.TextTestRunner(verbosity=2).run(suite)

So you only execute the tests in the class TestClass

edited Sep 17 '14 at 16:34

Egor Neliuba

14,784
7
59
77

answered Sep 17 '14 at 16:09

Angel

61
1
1

This is the least hacky solution. Instead of modifying what `unittest.main()` collects into default suite you form explicit suite and run its tests. – zgoda May 26 '17 at 09:55

score 6 · Answer 9 · answered Jun 23 '11 at 19:01

Matthew's answer is the one I needed to use since I'm on 2.5 still. But as of 2.7 you can use the @unittest.skip() decorator on any test methods you want to skip.

http://docs.python.org/library/unittest.html#skipping-tests-and-expected-failures

You'll need to implement your own skipping decorator to check for the base type. Haven't used this feature before, but off the top of my head you could use BaseTest as a marker type to condition the skip:

def skipBaseTest(obj):
    if type(obj) is BaseTest:
        return unittest.skip("BaseTest tests skipped")
    return lambda func: func

medmunds · Answer 10 · 2022-03-24T16:58:14.607

Here is a solution that uses only documented unittest features, and that avoids having a "skip" status in your test results:

class BaseTest(unittest.TestCase):

    def __init__(self, methodName='runTest'):
        if self.__class__ is BaseTest:
            # don't run these tests in the abstract base implementation
            methodName = 'runNoTestsInBaseClass'
        super().__init__(methodName)

    def runNoTestsInBaseClass(self):
        pass

    def testCommon(self):
        # everything else as in the original question

How it works: per the unittest.TestCase documentation, "Each instance of TestCase will run a single base method: the method named methodName." The default "runTests" runs all the test* methods on the class—that's how TestCase instances normally work. But when running in the abstract base class itself, you can simply override that behavior with a method that does nothing.

A side effect is your test count will increase by one: the runNoTestsInBaseClass "test" gets counted as a successful test when it's run on BaseClass.

(This also works in Python 2.7, if you're still on that. Just change super() to super(BaseTest, self).)

A better way is to override the `run(result=None)` method and just return the `result` in case of BaseTest. Then also the correct number of tests is reported. — interDist, Aug 31 '23 at 06:45

score 1 · Answer 11 · answered Sep 18 '15 at 00:45

Just rename the testCommon method to something else. Unittest (usually) skips anything that doesn't have 'test' in it.

Quick and simple

  import unittest

  class BaseTest(unittest.TestCase):

   def methodCommon(self):
       print 'Calling BaseTest:testCommon'
       value = 5
       self.assertEquals(value, 5)

  class SubTest1(BaseTest):

      def testSub1(self):
          print 'Calling SubTest1:testSub1'
          sub = 3
          self.assertEquals(sub, 3)


  class SubTest2(BaseTest):

      def testSub2(self):
          print 'Calling SubTest2:testSub2'
          sub = 4
          self.assertEquals(sub, 4)

  if __name__ == '__main__':
      unittest.main()`

This would have the result of not running the methodCommon test in either of the SubTests. — Pepper Lebeck-Jobe, Nov 17 '15 at 22:31

score 1 · Answer 12 · edited May 23 '17 at 11:47

I made about the same than @Vladim P. (https://stackoverflow.com/a/25695512/2451329) but slightly modified:

import unittest2


from some_module import func1, func2


def make_base_class(func):

    class Base(unittest2.TestCase):

        def test_common1(self):
            print("in test_common1")
            self.assertTrue(func())

        def test_common2(self):
            print("in test_common1")
            self.assertFalse(func(42))

    return Base



class A(make_base_class(func1)):
    pass


class B(make_base_class(func2)):

    def test_func2_with_no_arg_return_bar(self):
        self.assertEqual("bar", func2())

and there we go.

score 1 · Answer 13 · answered Jul 04 '18 at 17:40

As of Python 3.2, you can add a test_loader function to a module to control which tests (if any) are found by the test discovery mechanism.

For example, the following will only load the original poster's SubTest1 and SubTest2 Test Cases, ignoring Base:

def load_tests(loader, standard_tests, pattern):
    suite = TestSuite()
    suite.addTests([SubTest1, SubTest2])
    return suite

It ought to be possible to iterate over standard_tests (a TestSuite containing the tests the default loader found) and copy all but Base to suite instead, but the nested nature of TestSuite.__iter__ makes that a lot more complicated.

score 0 · Answer 14 · edited May 09 '18 at 18:30

So this is kind of an old thread but I came across this problem today and thought of my own hack for it. It uses a decorator that makes the values of the functions None when acessed through the base class. Don't need to worry about setup and setupclass because if the baseclass has no tests they won't run.

import types
import unittest


class FunctionValueOverride(object):
    def __init__(self, cls, default, override=None):
        self.cls = cls
        self.default = default
        self.override = override

    def __get__(self, obj, klass):
        if klass == self.cls:
            return self.override
        else:
            if obj:
                return types.MethodType(self.default, obj)
            else:
                return self.default


def fixture(cls):
    for t in vars(cls):
        if not callable(getattr(cls, t)) or t[:4] != "test":
            continue
        setattr(cls, t, FunctionValueOverride(cls, getattr(cls, t)))
    return cls


@fixture
class BaseTest(unittest.TestCase):
    def testCommon(self):
        print('Calling BaseTest:testCommon')
        value = 5
        self.assertEqual(value, 5)


class SubTest1(BaseTest):
    def testSub1(self):
        print('Calling SubTest1:testSub1')
        sub = 3
        self.assertEqual(sub, 3)


class SubTest2(BaseTest):

    def testSub2(self):
        print('Calling SubTest2:testSub2')
        sub = 4
        self.assertEqual(sub, 4)

if __name__ == '__main__':
    unittest.main()

score -2 · Answer 15 · answered Aug 24 '09 at 16:48

-2

Change the BaseTest method name to setUp:

class BaseTest(unittest.TestCase):
    def setUp(self):
        print 'Calling BaseTest:testCommon'
        value = 5
        self.assertEquals(value, 5)


class SubTest1(BaseTest):
    def testSub1(self):
        print 'Calling SubTest1:testSub1'
        sub = 3
        self.assertEquals(sub, 3)


class SubTest2(BaseTest):
    def testSub2(self):
        print 'Calling SubTest2:testSub2'
        sub = 4
        self.assertEquals(sub, 4)

Output:

Ran 2 tests in 0.000s

Calling BaseTest:testCommon Calling
SubTest1:testSub1 Calling
BaseTest:testCommon Calling
SubTest2:testSub2

From the documentation:

TestCase.setUp()
Method called to prepare the test fixture. This is called immediately before calling the test method; any exception raised by this method will be considered an error rather than a test failure. The default implementation does nothing.

answered Aug 24 '09 at 16:48

Brian R. Bondy

339,232
124
596
636

That would work, what if I have n testCommon, should I place them all under `setUp`? – Thierry Lam Aug 24 '09 at 16:50
1

Yes you should put all code that is not an actual test case under setUp. – Brian R. Bondy Aug 24 '09 at 16:56
But if a subclass has more than one `test...` method, `setUp` gets executed over and over and over again, once per such method; so it's NOT a good idea to put tests there! – Alex Martelli Aug 24 '09 at 18:14
Not really sure what OP wanted in terms of when executed in a more complex scenario. – Brian R. Bondy Aug 24 '09 at 18:36

Python unit test with base and sub class

15 Answers15

Linked