Cucumber JVM: Test if the correct exception is thrown

Question

How to test that the correct exception is thrown when using Cucumber JVM? When using JUnit, I would do something like this:

@Test(expected = NullPointerException.class)
public void testExceptionThrown(){
    taskCreater.createTask(null);
}

As you can see, this is very elegant. But how can I achieve the same elegance, when using cucumber JVM? My test looks like this right now:

@Then("the user gets a Null pointer exception$")
public void null_exception_thrown() {
    boolean result = false;
    try {
        taskCreater.createTask(null);
    } catch (NullPointerException e) {
        result = true;
    }
    assertTrue(result);
}

Note the need for a try..catch followed by an assertTrue on a flag.

Answer 1

Testing the not-happy-path can be hard. Here's a nice way that I've found to do it with cucumber.

Scenario: Doing something illegal should land you in jail
    Then a failure is expected
    When you attempt something illegal
    And it fails.

OK, don't shoot me because I put the Then before the When, I just think it reads better but you don't have to do that.

I store my excepions in a (cucumber-scoped) world object, but you could also do it in your step file, but this will limit you later.

public class MyWorld {
    private boolean expectException;
    private List<RuntimeException> exceptions = new ArrayList<>();

    public void expectException() {
        expectException = true;
    }

    public void add(RuntimeException e) {
        if (!expectException) {
            throw e;
        }
        exceptions.add(e);
    }

    public List<RuntimeException> getExceptions() {
        return exceptions;
    }
}

Your steps are then pretty simple:

@Then("a failure is expected")
public void a_failure_is_expected() {
    myWorld.expectException();
}

In a step where you are (at least sometimes) expecting an exception, catch it and add it to the world.

@When("you attempt something illegal")
public void you_attempt_something_illegal() {
    try {
        myService.doSomethingBad();
    } catch (RuntimeException e) {
        world.add(e);
    }
}

Now you can check whether the exception was recorded in the world.

@And("it fails")
public void it_fails() {
    assertThat(world.getExceptions(), is(not(empty()));
}

The most valuable thing about this approach is that it won't swallow an exception when you don't expect it.

Answer 2

Have you tried using the junit @Rule annotation with ExpectedException, like this:

@Rule
public ExpectedException expectedEx = ExpectedException.none();

@Then("the user gets a Null pointer exception$")
public void null_exception_thrown() {
    expectedEx.expect(NullPointerException.class);
    //expectedEx.expectMessage("the message");
    taskCreater.createTask(null);
}

Answer 3

Behaviour testing verifies if your project follows specifications, and I doubt the user expects a NullPointerException while using the system.

In my opinion (I'm not familiar with your project), exceptions should only be checked during Unit Tests, as they correspond to an unexpected error or user mistake.

It's very unusual to check for exceptions during behaviour tests. If an exception is thrown during a test, it should fail.

for instance:

test.feature

Given that I have a file "users.txt"
And I try to import users /* If an Exception is thrown here, the test fails */
Then I should see the following users: /* list of users */

In my unit test, I would have:

@Test(expected = MyException.class)
public void importUsersShouldThrowMyExceptionWhenTheFileIsNotFound() {
    // mock a call to a file and throw an exception
    Mockito.when(reader.readFile("file.txt").thenThrow(new FileNotFoundException());

    importer.importUsers();
}

Answer 4

I suggest you to use org.assertj.assertj-core.

Thanks to Assertions class, you could simplify your assertion like bellow:

@Then("the user gets a Null pointer exception$")
public void null_exception_thrown() {
    Assertions.assertThatThrownBy(() -> taskCreater.createTask(null)).
               isInstanceOf(NullPointerException.class);
}

Answer 5

Never used Cucumber, but would

    public void null_exception_thrown() {
            try {
                  taskCreater.createTask(null);
                  fail("Null Pointer Expected");
            } catch (NullPointerException e) {

                // Do Nothing
             }
        }

work for you?

Answer 6

I believe Cucumber is meant to test higher level acceptance tests than lower level unit tests as in that case we will be testing the structure opposed to the behavior which is not the desired usage of the Cucumber framework.

Answer 7

i used to have expected exception name in feature steps. for a calculator test example

Here is my feature file entry for division exception :

Scenario: Dividing a number with ZERO
Given I want to test calculator
When I insert 5.5 and 0.0 for division
Then I got Exception ArithmeticException

So, you may see, I have added the exception name. So, in step definition , i get the name of the exception.

Now, we need to get the exception when dividing by ZERO and put in a variable. And get this variable to compare its class name(exception name)

so, in step definition where i divide by zero

private Exception actualException;
 @When("^I insert (.+) and (.+)for division$")
public void iInsertAndForDivision(double arg0, double arg1) throws Throwable {
    try {
        result = calculator.div(arg0, arg1);
    } catch (Exception e) {
        actualException =e;
    }
}

And i need the step definition for validating the exception

  @Then("^I got Exception (.+)")
public void iGotArithmeticException(String aException) throws Throwable {        
Assert.assertEquals("Exception MissMatch",aException,actualException.getClass().getSimpleName());
}

The complete project can be seen here : Steps for cucumber