15

When intentionally failing a test case (for example when an exception is not thrown) I've seen people use both fail() and assertTrue(false). Are there any advantages to using one or the other?

try {
    //method call that should throw exception
    fail("oops");
} catch (Exception e) {}

vs.

try {
    //method call that should throw exception
    assertTrue("oops", false);
} catch (Exception e) {}
TrevorBliss
  • 371
  • 3
  • 13
  • Why do you want to fail a test? – Victor Oct 17 '12 at 16:54
  • 4
    When an exception should have been thrown but wasn't. Do you not agree with my examples? – TrevorBliss Oct 17 '12 at 16:59
  • 1
    Exception should be an exception. A good code should never expect an exception, your code should avoid exception before it was thrown. – Victor Oct 17 '12 at 17:05
  • 5
    @Victor normally I agree but if you have a complicated Exception stack (lots of custom exceptions, possible bubbling up) you may wish to make sure the correct exception is thrown in varying cases. – BeRecursive Oct 17 '12 at 17:09
  • 4
    @Victor I think you've been writing unit tests wrong, if you're never testing that exceptions get thrown in an error situation. You want to check _failure paths_ as well as _success paths_. This will often require checking whether or not an exception has been thrown. – Izkata Oct 17 '12 at 20:31
  • @Victor How do I unit test that my methods throw the correct exceptions then? – Edward Thomson Oct 17 '12 at 20:31
  • I believe the correct is make as @BeRecursive answer. Forcing the error. – Victor Oct 18 '12 at 13:47
  • @Victor I think BeRecursive's answer is valid, but I'm working on a relatively large project with a testing framework already well established and currently we only use assert/fail statements. Introducing the `@Test` annotation now will introduce inconsistencies, and in my opinion not worth the extra code and time to edit all existing test cases. Especially since I see no huge gain by doing so. – TrevorBliss Oct 18 '12 at 19:42

5 Answers5

21

Are there any advantages to using one or the other?

Functionally, no. However, fail() conveys the intention more clearly, and is therefore better.

Keppil
  • 45,603
  • 8
  • 97
  • 119
13

Use JUnit's expected parameter in the @Test annotation

@Test(expected = ArithmeticException.class)  
public void divisionWithException() {  
    int i = 1/0;
}

This will fail if an ArithmeticException is not thrown.

BeRecursive
  • 6,286
  • 1
  • 24
  • 41
  • 2
    The OP does say "for example, when an exception is not thrown" There could be other scenarios. – NullUserException Oct 17 '12 at 16:56
  • 3
    I have found that the `EpectedException` rule is a more robust mechanism for testing expected exceptions because it can be placed immediately before the method under test thereby ensuring it was not the test setup that threw an exception. – John B Oct 17 '12 at 16:57
  • 1
    But we are EXPECTING an exception to be thrown. So if it doesn't it should fail. Why bother with try/catch logic? We want the exception. – BeRecursive Oct 17 '12 at 16:57
  • Concur with NullUserException – John B Oct 17 '12 at 16:58
  • How do you add a failure message to be displayed to the user? Like in fail("failure message"). – TrevorBliss Oct 17 '12 at 17:03
  • @NullUserException I just don't understand why you would intentionally fail a test. The whole point in a test is to make sure something happened the way it was supposed. Not to make sure the infinite possible ways it could fail didn't happen. If it didn't do what you explicitly asserted then it should fail. You shouldn't explicitly state all the ways in which it shouldn't fail. – BeRecursive Oct 17 '12 at 17:03
  • @Trevor see an alternate ExpectedException usage for Junit4 at this answer: http://stackoverflow.com/questions/2469911/how-do-i-assert-my-exception-message-with-junit-test-annotation – BeRecursive Oct 17 '12 at 17:08
  • @BeRecursive You can't always let junit catch the exception. You might want to catch the exception yourself, so that you can test that it's correct. – Kenster Oct 17 '12 at 18:19
  • I have to agree with @JohnB - I've used the try/catch mechanism many times because this is testing the wrong thing in just about any test method larger than 3-4 lines. For example, I _don't_ want an exception thrown while setting up an object, but I _do_ want it thrown during the method being tested. If there's an error during setup, the test will pass even though it's supposed to be failing. – Izkata Oct 17 '12 at 20:35
  • @Izkata whilst this is possible, it is highly unlikely that the same exception will be thrown at any point during the test. If the wrong exception class is thrown the test still fails. Nonetheless if you want to test something specific about an exception you could use ExpectedException as I mentioned above. – BeRecursive Oct 18 '12 at 08:36
  • 1
    @BeRecursive Regarding your response to Izkata, I have seen a surprising number of occasions when I am expecting a NullPointerException and the test passes because it was thrown from the wrong place. Not to overkill this subject, but in my experience it is not "highly unlikely" when dealing with exceptions like NPE, IAE and Runtime. – John B Oct 18 '12 at 10:17
  • @JohnB I find it strange that you would ever expect a NullPointerException? Surely that is one of the exceptions that is always an error when thrown? Even in some niche case whereby you might want it to get thrown earlier on in the stack trace, I would expect you to catch it and wrap it in a custom Exception. Then you would be expecting your custom Exception instead. – BeRecursive Oct 18 '12 at 10:20
  • One of the first things I test in any public method, is that it throws an NPE if passed `null` for a `@Nonnull` parameter. Google has provided a helper for just that reason: `NullPoinerTester` and this is highly encouraged by `Effective Java` – John B Oct 18 '12 at 10:22
  • Another way I use `ExpectedException` is if I have a method that takes more than one parameter and does validation on the parameters (which all public methods should do) and thereby throws multiple different `IllegalArgumentExceptions`. `ExpectedException` allows me to test the message to ensure I am getting the correct exception. – John B Oct 18 '12 at 10:24
  • I am not saying that `@Test(expected)` does not have a place. But I only use it in a single line test where the method under test can only throw the expected exception from one location / in one condition. – John B Oct 18 '12 at 10:25
  • @JohnB I completely agree with you about ``ExpectedException`` and I also often follow that workflow. In regards to throwing ``NullPointerException``s yourself I guess that is a matter of convention. I, personally, severely dislike this practise and would prefer something more akin to an ``IllegalArgumentException`` or ``IllegalStateException``. – BeRecursive Oct 18 '12 at 10:32
  • I have been converted by Guava's `Preconditions` class which throws an NPE from `checkNotNull`. Prior to starting to use this I felt odd about throwing NPE as well. Again, `Effective Java` states it is appropriate to throw NPE, IAE and ISE from parameter validation. Good discussion, cheers! http://docs.guava-libraries.googlecode.com/git-history/release/javadoc/com/google/common/base/Preconditions.html#checkNotNull(T) – John B Oct 18 '12 at 10:36
5

assertTrue just calls fail.

IMHO fail() is simpler and clearer.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
3

Use fail, otherwise someone will look at the failure in the log and think "What's false that should be true".

John B
  • 32,493
  • 6
  • 77
  • 98
1

Both can be used (though fail is better because that's what is really intended), but since you're using JUnit, you may use @Test(expected=MyException.class) instead.

Alex
  • 25,147
  • 6
  • 59
  • 55