python unittest count tests

Question

Is it possible with unittest to have an option tou count the number of total test number of skipped tests and number of run. And after the run the number of test failed (I know that it can be seen in the output). I would like to dump it in a json if I wan take it programatically that would be great

Thx a lot

Answer 1

After many trials and errors, I finally got this working...

Based on scoffey's answer.

Hope it helps.

import unittest

class MyTest(unittest.TestCase):

    currentResult = None # holds last result object passed to run method

    @classmethod
    def setResult(cls, amount, errors, failures, skipped):
        cls.amount, cls.errors, cls.failures, cls.skipped = \
            amount, errors, failures, skipped

    def tearDown(self):
        amount = self.currentResult.testsRun
        errors = self.currentResult.errors
        failures = self.currentResult.failures
        skipped = self.currentResult.skipped
        self.setResult(amount, errors, failures, skipped)

    @classmethod
    def tearDownClass(cls):
        print("\ntests run: " + str(cls.amount))
        print("errors: " + str(len(cls.errors)))
        print("failures: " + str(len(cls.failures)))
        print("success: " + str(cls.amount - len(cls.errors) - len(cls.failures)))
        print("skipped: " + str(len(cls.skipped)))

    def run(self, result=None):
        self.currentResult = result # remember result for use in tearDown
        unittest.TestCase.run(self, result) # call superclass run method

    def testA(self):
        self.assertTrue(True) # succeeds

    def testB(self):
        self.assertTrue(False) # fails

    def testC(self):
        self.assertTrue(1 + None is None) # raises TypeError

    @unittest.skip("skip it") # skipped
    def testD(self):
        self.assertTrue("whatever")

if __name__ == '__main__':
    unittest.main() 

run script with

python test.py > result.txt

result.txt:

tests run: 3
errors: 1
failures: 1
success: 1
skipped: 1

I'm not sure this is the best way, but it's working. Unittest module is easy to use but hard to master, now I feel I know little about it.

Answer 2

I Uses the unittests TestSuite (Ref).

On after run it returns a TextTestResult, that contains a list with Failures, Errors and Skipped, a value with Test_runs, and more.

Here are a "minimum" working example, on how I would do it.

#!/usr/bin/python3
# -*- coding: utf-8 -*-
import unittest


class TestDummy(unittest.TestCase):
    """A Dummy UnitTesting class."""

    def test_failure(self):
        """Fails on test."""
        self.fail(msg="Need a failure")

    @unittest.skip("Need a Skipper")
    def test_skipping(self):
        """Skippes on test."""
        pass

    def test_error(self):
        """Gives a error on test."""
        self.not_a_thing()

    def test_pass(self):
        """Need a test that passes."""
        pass


def warp_test_suite(testcase_class):
    """Load tests from a specific set of TestCase classes."""
    suite = unittest.TestSuite()
    tests = unittest.defaultTestLoader.loadTestsFromTestCase(testcase_class)
    suite.addTest(tests)
    return suite


if __name__ == "__main__":
    import json  # For saving a JSON-file

    # The test results dictionary, for the JSON.
    result_value = {"Failures": 0, "Errors": 0, "Skipped": 0, "Test Runs": 0}

    # Setup and run the Test
    runner = unittest.TextTestRunner()
    TextTestResult = runner.run(warp_test_suite(TestDummy))

    # Passes the Result
    result_value["Failures"] += len(TextTestResult.failures)
    result_value["Errors"] += len(TextTestResult.errors)
    result_value["Skipped"] += len(TextTestResult.skipped)
    result_value["Test Runs"] += TextTestResult.testsRun

    # Save the result to a JSON-file.
    with open("result_data.json", 'w') as fp:
            json.dump(result_value, fp, indent=3)

Answer 3

I don't know any way for unittest to report in JSON. I am aware that nose is outputing result in XML format:

nosetests --with-xunit --xunit-file=mytests.xml mytests.py

Here is an excerpt from this XML file:

<testsuite name="nosetests" tests="3" errors="0" failures="1" skip="1">

If you don't mind XML format, then this is a solution to consider. I also heard that nose has a JSON plug-in, but have not played with it yet.

Answer 4

If you review to source code, you may found this line:

        self.result = testRunner.run(self.test)
    if self.exit:
        sys.exit(not self.result.wasSuccessful())

So what you need to do is write code like this:

    result = unittest.main(exit=False).result
    print(result.testsRun)

Remember to set exit args to be False. Then you can use result.result to output testing summary.