Specifying order of execution in JUnit test case

Question

I have a test case where I add an entity, update it and delete the same. Hence, the order of execution is important here. I want it to be :

1. Create
2. Update
3. Delete

Strangely, for just one test case ( out of 15) , JUnit executes it in the following order :

1. Delete
2. Update
3. Create .

How do I tell JUnit to execute them in a specific order ? In other cases, JUnit works totally fine ( executing serially ) . And why does JUnit behave weirdly in this one case ?

Relevant code snippet below :

    private static Date date;
    private static int entity;
    static Parking p;
    public ParkingTests(String name) {
       super(name);
    }
    public void testAdd() throws Exception {
           //Add code here
    }
    public void testUpdate() throws Exception {
            //update code here
    }
    public void testDelete() throws Exception {
            //delete code here
    }
  }

It gets weirder. I run a lot of test cases as part of a suite. If I run just the Parking case, the order is maintained. If I run it along with others, it is sometimes maintained, sometimes not !

Answer 1

Your kind of situation is awkward, as it feels bad to keep duplicating work in order to isolate the tests (see below) - but note that most of the duplication can be pulled out into setUp and tearDown (@Before, @After) methods, so you don't need much extra code. Provided that the tests are not running so slowly that you stop running them often, it's better to waste a bit of CPU in the name of clean testing.

public void testAdd() throws Exception {
      // wipe database
      // add something
      // assert that it was added
}
public void testUpdate() throws Exception {
      // wipe database
      // add something
      // update it
      // assert that it was updated
}
public void testDelete() throws Exception {
      // wipe database
      // add something
      // delete it
      // assert that it was deleted
}

The alternative is to stick everything into one test with multiple asserts, but this is harder to understand and maintain, and gives a bit less information when a test fails:

public void testCRUD() throws Exception {
      // wipe database
      // add something
      // assert that it was added
      // update it
      // assert that it was updated
      // delete it 
      // assert that it was deleted
}

Testing with databases or collections or storage of any kind is tricky because one test can always affect other tests by leaving junk behind in the database/collection. Even if your tests don't explicitly rely on one another, they may still interfere with one another, especially if one of them fails.

Where possible, use a fresh instance for each test, or wipe the data, ideally in as simple a way as possible - e.g. for a database, wiping an entire table is more likely to succeed than a very specific deletion that you might accidentally get wrong.

Update: It's usually better to wipe data at the start of the test, so one failed test run doesn't affect the next run.

Answer 2

Generally junit tests(test methods) should not depend on each other. Following is taken from junit FAQ

Each test runs in its own test fixture to isolate tests from the changes made by other tests. That is, tests don't share the state of objects in the test fixture. Because the tests are isolated, they can be run in any order...... The ordering of test-method invocations is not guaranteed.

So if you want to do some common initialization stuff then you could do that in the method annotated with @Before and cleanup in method annotated with @After. Or else if that initialization is not required for all tests methods in your test class then you could put that in private methods and call them appropriately from your tests.

On a side note, if you still want to do ordering of tests then you may have a look at TestNG.

Answer 3

If you're determined you would want to have order of execution for your tests, JUnit 4.11 now supports this through an annotation. See this thread for more discussion - basically, you would use

@FixMethodOrder

to guarantee some test order that way. It is discouraged though.

Answer 4

If you are using Java 7 then you should know that Junit gets the list of all tests using "Method[] getDeclaredMethods()" from java.lang.Class. You can read from the javadoc of this method or from junit docs that: "The elements in the array returned are not sorted and are not in any particular order.", but in previous jvm implementation methods list was ordered as they were in source code.

This was taken from this blog and he provides a work around.

Answer 5

In general, JUnit does not guarantee the ordering of test cases. It's not guaranteed to be alphabetical, nor the order in the file. If the ordering of tests were important, then one depends on the output of the previous. What if the first one failed? Should we even bother with the later (and dependent) tests? Probably not.

So if we had this:

@Test
public void first(){...}

@Test
public void second() {...}

@Test
public void third() {...}

We don't know what order they will run in. Since we are hoping they go in order, and we should probably not bother running second or third if the previous one(s) failed, we can do this instead:

@Test
public void firstThree(){
    first();
    second();
    third();
}

public void first(){...}
public void second() {...}
public void third() {...}

Notice that we only have one @Test this time, and it guarantees ordering.

Answer 6

If you want to run junit tests in order "just as they present in your source code", see my note about this here:

How to run junit tests in order as they present in your source code

But it is really not a good idea, tests must be independent.

Answer 7

What you can do :

• Cleanup the database before every test
• Start by testing the first logical operation first. When you have enough confidence, assume it is correct and move to the next, etc...
• Write white box tests first, but start with black box tests. For example if you have triggers or similar in your database, start with that.