Unit testing tornado applications: How to improve the display of error messages

Question

I am using unittest to test a tornado app having several handlers, one of which raises an exception. If I run the following test code with python test.py:

# test.py

import unittest
import tornado.web
import tornado.testing

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write('Hello World') # handler works correctly

class HandlerWithError(tornado.web.RequestHandler):
    def get(self):
        raise Exception('Boom') # handler raises an exception
        self.write('Hello World')

def make_app():
    return tornado.web.Application([
        (r'/main/', MainHandler),
        (r'/error/', HandlerWithError),
    ])

class TornadoTestCase(tornado.testing.AsyncHTTPTestCase):

    def get_app(self):
        return make_app()

    def test_main_handler(self):
        response = self.fetch('/main/')
        self.assertEqual(response.code, 200) # test should pass

    def test_handler_with_error(self):
        response = self.fetch('/error/')
        self.assertEqual(response.code, 200) # test should fail with error

if __name__ == '__main__':
    unittest.main()

the test output looks like:

ERROR:tornado.application:Uncaught exception GET /error/ (127.0.0.1)                                                                                                                   
HTTPServerRequest(protocol='http', host='localhost:36590', method='GET', uri='/error/', version='HTTP/1.1', remote_ip='127.0.0.1', headers={'Connection': 'close', 'Host': 'localhost:3
6590', 'Accept-Encoding': 'gzip'})                                                                                                                                                     
Traceback (most recent call last):                                                                                                                                                     
  File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1332, in _execute                                                                                                 
    result = method(*self.path_args, **self.path_kwargs)                                                                                                                               
  File "test.py", line 13, in get                                                                                                                                                      
    raise Exception('Boom') # handler raises an exception                                                                                                                              
Exception: Boom                                                                                                                                                                        
ERROR:tornado.access:500 GET /error/ (127.0.0.1) 19.16ms                                                                                                                               
F.                                                                                                                                                                                     
======================================================================                                                                                                                 
FAIL: test_handler_with_error (__main__.TornadoTestCase)                                                                                                                               
----------------------------------------------------------------------                                                                                                                 
Traceback (most recent call last):                                                                                                                                                     
  File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in __call__                                                                                              
    result = self.orig_method(*args, **kwargs)                                                                                                                                         
  File "test.py", line 33, in test_handler_with_error                                                                                                                                  
    self.assertEqual(response.code, 200) # test should fail with error                                                                                                                 
AssertionError: 500 != 200                                                                                                                                                             

----------------------------------------------------------------------                                                                                                                 
Ran 2 tests in 0.034s                                                                                                                                                                  

FAILED (failures=1)

However, I would expect unittest to report an Error for the second test, instead of a failing assertion. Moreover, the fact that the traceback for the 'Boom' exception appears before the unittest test report and does not include a reference to the failing test function makes it difficult to find the source of the exception.

Any suggestions how to handle this situation?

Thanks in advance!

EDIT

What I find unexpected is the fact that test_handler_with_error actually arrives at making the assertEqual assertion, instead of throwing the error. For example, the following code does not execute the self.assertEqualstatement, and consequently reports an ERROR instead of a FAIL in the test output:

# simple_test.py
import unittest

def foo():
    raise Exception('Boom')
    return 'bar'

class SimpleTestCase(unittest.TestCase):
    def test_failing_function(self):
        result = foo()
        self.assertEqual(result, 'bar')

if __name__ == '__main__':
    unittest.main()

score 1 · Answer 1 · edited May 23 '17 at 12:26

You can disable logging and only the test reports will appear:

logging.disable(logging.CRITICAL)

You can put that for example in

created TestCase subclass
test runner

More info How can I disable logging while running unit tests in Python Django?

Keep in mind that CI/CD systems actually use normalized report e.g. junit and then present it in more readable/elegant way - more info:

A. Jesse Jiryu Davis · Accepted Answer · 2017-03-02T17:58:05.403

This is expected behavior. Your test itself asserts that the return code is HTTP 200, and since this is a formal assert that is false, the outcome is a "failure" instead of an "error". You can suppress logs as mentioned in kwaranuk's answer, but then you lose the information about what actually caused the HTTP 500 error.

Why does your code reach the assert, instead of throwing? It's because your test code does not call HandlerWithError.get. Your test code begins an asynchronous HTTP GET operation with an HTTP client provided by the AsyncHTTPTestCase class. (Check the source code of that class for details.) The event loop runs until HandlerWithError.get receives the request over a localhost socket, and responds on that socket with an HTTP 500. When HandlerWithError.get fails, it doesn't raise an exception into your test function, any more than a failure at Google.com would raise an exception: it merely results in an HTTP 500.

Welcome to the world of async! There's no easy way to neatly associate the assertion error and the traceback from HandlerWithError.get().

Thanks for the answer. I've edited my question to clarify the cause of my confusion. — Martin Bierbaum, Mar 02 '17 at 13:28
Editing my answer: your test function is not directly calling the function that raises an exception. — A. Jesse Jiryu Davis, Mar 02 '17 at 17:58

Unit testing tornado applications: How to improve the display of error messages

2 Answers2