what is the properly assert to fail test when exception occur?

Question

I would like testcase pass when no particular exception raise and make testcase only failed when ValueError occurs. get some information from another post feel like maybe there is a better way than try/except How to properly assert that an exception gets raised in pytest?

def test_exception():
    try:
        call some function:
    except ValueError:
        assert False

So what should happen when expection different from ValueError occurs? I mean test can only pass or fail, there is no third state, isn't it? — R2RT, Jul 19 '18 at 20:38
I'm pretty confused by this, are you saying if `some function` raises an `IndexError` for example, you want the test to pass? — Ryan Haining, Jul 19 '18 at 20:40
if some function raises exception different than ValueError..i though testcase will failed auto with exception — jacobcan118, Jul 19 '18 at 20:42
some testcase pass with no exception, testcase failed with ValueError testcase failed with exception when other than ValueError exception occur — jacobcan118, Jul 19 '18 at 20:44
@jacobcan118 so what's different about ValueError? The test would fail if a different exception its raised, but you're asking how to fail only when ValueError is raised... except also fail if another exception is raised too. The test is gonna pass, or it's gonna fail, if all exceptions should cause it to fail you don't have to do anything special for ValueError — Ryan Haining, Jul 19 '18 at 21:31
Possible duplicate of [How to use pytest to check that Error is NOT raised](https://stackoverflow.com/questions/20274987/how-to-use-pytest-to-check-that-error-is-not-raised) — hoefling, Jul 19 '18 at 21:45

Matt Messersmith · Answer 1 · 2018-07-19T23:53:54.010

4

You'd want this if you want to fail if and only if a ValueError is thrown (and pass on all other exceptions, and also pass if there is NO exception at all):

def test_exception():
    try:
        call_some_function()
    except ValueError:
        assert False
    except:
        pass

If you want to make sure an exception is thrown, but NOT a ValueError, then you'd want this:

def test_exception():
    try:
        call_some_function()
        assert False
    except ValueError:
        assert False
    except:
        pass

As a warning, this is backwards from what most people want to test. Not saying what you're doing is wrong, but usually you want to test that a certain exception IS being thrown: not that a certain exception is NOT being thrown. If that's what you want, you'd want this:

def test_exception():
    try:
        call_some_function()
        assert False
    except ValueError:
        pass
    except:
        assert False

Or you could get rid of that last except block and let it bubble up. The difference is just between an "error" and a "failure". Either way, your test suite won't pass.

~~In terms of style, there's nothing wrong with assert False or assert 0 when you want to explicitly fail.~~ I (and other professional python devs I work with) do it all the time.

EDIT: Better to use pytest.fail with a particular message instead of assert False, especially in cases where there is more than one assert False floating around. That way, you'll know which assert failed and why if you add a helpful error message.

edited Jul 19 '18 at 23:53

answered Jul 19 '18 at 20:49

Matt Messersmith

12,939
6
51
52

I'm not sure what you're asking. If you want to handle the exception, or assert something about a specific exception, then yes, you'll have to use `try` and `except` in some fashion. – Matt Messersmith Jul 19 '18 at 20:56
2

I would use `pytest.fail('some meaningful message about the failure context')` instead of `assert False`. – hoefling Jul 19 '18 at 21:38
@hoefling I agree that `pytest.fail` is good...but we strive to have extremely descriptive test names s.t. when they fail it isn't a mystery why. For example, `test_that_some_function_throws_a_value_error_when_called_with_2`. If it fails, we then have a pretty good idea of what went wrong. We're in the habit of never using `pytest.fail` for this reason (feels like we'd just be duplicating whatever our test function name is). – Matt Messersmith Jul 19 '18 at 22:22
It may work for most simple tests, but already with more than one assertion in a test (like in your example given) one is forced to read the stack trace in order to understand what particular `assert False` failed and why. A short explanation is always better than none. – hoefling Jul 19 '18 at 23:38
Ja, du hast recht. I've been bitten by the issue before. I'll add to my answer here – Matt Messersmith Jul 19 '18 at 23:50
Upvoting because you're a tactful person. – hoefling Jul 20 '18 at 19:29

score -1 · Answer 2 · answered Jul 19 '18 at 20:38

The TestCase class has a context manager to use for this case:

class TestSomething(TestCase):
    def test_something(*args, **kwargs):
        with self.assertRaises(ValueError) as e:
            raise ValueError
        # you can check status of e here for information

[Edit]

Just realized I answered the reverse of your question. Personally, I would just let the exception propagate up and let pytest fail that way, but your approach above is fine as well in my opinion.

what is the properly assert to fail test when exception occur?

2 Answers2