How do I Dynamically create a Test Suite in JUnit 4?

Question

I would like to create a junit test suite using JUnit 4 where the names of the test classes to be included are not known until the test suite is run.

In JUnit 3 I could do this:

public final class MasterTester extends TestCase
{
  /**
   * Used by junit to specify what TestCases to run.
   * 
   * @return a suite containing what TestCases to run
   */
  public static TestSuite suite() {
    TestSuite suite = new TestSuite();

    for(Class<?> klass : gatherTestClasses()) {
      suite.addTestSuite(klass);
    }

    return suite;
  }
}

and let the gatherTestClasses() method deal with figuring out what test classes to run.

In JUnit 4, the documentation says to use an annotation: @SuiteClasses({TestClass1.class, TestClass2.class...}) to build up my test suite. There are numerous SO answers showing how to do this. Unfortunately the examples I see do not seem to allow for passing a dynamically generated list of TestClasses.

This SO answer suggested I would have to subclass BlockJUnit4ClassRunner which I do not want to do.

Dynamically specified test suites seem like something that must be in JUnit 4 somewhere. Does anyone know where?

score 41 · Answer 1 · edited Feb 14 '14 at 23:15

41

To create a dynamic test suite, you need to use the @RunWith annotation. There are two common ways to use it:

@RunWith(Suite.class)

This allows you to specify, which classes compose the test suite in question. This is equivalent to the JUnit 3 style:

import junit.framework.TestSuite;
import junit.framework.TestCase;

public final class MasterTester extends TestCase {

  public static TestSuite suite() {
    TestSuite suite = new TestSuite();
    suite.addTestSuite(TestClass1.class);        
    suite.addTestSuite(TestClass2.class);
    // etc...
    return suite;
  }
}

The equivalent JUnit 4 class will be:

import org.junit.runners.Suite;

@RunWith(Suite.class)
@SuiteClasses({TestClass1.class, TestClass2.class})
public final class MasterTester {

}

@RunWith(AllTests.class)

This allows you to dynamically specify the tests, which compose the test suite. If your tests are not known until runtime, you cannot specify them in the annotations. You can use this construction instead. So, if the JUnit 3 code is:

import junit.framework.TestCase;
import junit.framework.TestSuite;
import junit.framework.Test;

public final class MasterTester extends TestCase {

  public static TestSuite suite() {
    TestSuite suite = new TestSuite();
    for (Test test : findAllTestCasesRuntime()) {
      suite.addTest(test);
    }
    return suite;
  }
}

The equivalent JUnit 4 code will be:

import org.junit.runners.AllTests;
import junit.framework.TestSuite;
import junit.framework.Test;

@RunWith(AllTests.class)
public final class MasterTester {

  public static TestSuite suite() {
    TestSuite suite = new TestSuite();
    for (Test test : findAllTestCasesRuntime()) {
      suite.addTest(test);
    }
    return suite;
  }
}

edited Feb 14 '14 at 23:15

Don Kirkby

53,582
27
205
286

answered Sep 13 '11 at 15:47

Danail Nachev

19,231
3
21
17

The action inside this for loop seems wrong. Do you mean to do suite.addTest(test); ? – Ben McCann Sep 23 '12 at 23:21
Thanks for pointing out. I've updated the sources to use addTest() and addTestSuite() – Danail Nachev Sep 24 '12 at 11:10
why are you making a new TestCase()? shouldn't you add the test that you found? – Ben McCann Sep 24 '12 at 15:44
And what would a `Test` class be? What class should I extend? – Peterdk Sep 09 '13 at 13:39
@Peterdk The same you'll extend in JUnit 3 style test case: junit.framework.TestCase or some specification of it, which gives you additional features. – Danail Nachev Oct 21 '13 at 17:24
2

@Danail Nachev: Sorry, but this is all non-sense (not your fault). JUnit4 proudly replaced inheritance by annotations, so I don't have any `Test`, just plain old classes. So your solution doesn't work for me, unless I edit them all. – maaartinus Nov 18 '13 at 13:23
3

how to implement the findAllTestCasesRuntime() method? – 5YrsLaterDBA May 04 '15 at 18:49
@5YrsLaterDBA This depends on your specific case: you may use external files to determine the list of tests, you can scan classpaths for specifically annotated classes or something even crazier. The only requirement is that findAllTestCasesRuntime() returns the list of junit.framework.Test classes (or subclasses), which will be added to the suite. – Danail Nachev May 04 '15 at 19:35
I am using JUnit 4 and none of my test classes are a subclass of Junit Test interface. – 5YrsLaterDBA May 04 '15 at 19:54
1

You can use ```junit.framework.JUnit4TestAdapter``` to construct a suitable object to represent your test classes. – Danail Nachev May 27 '15 at 20:37
2

But where is the method `findAllTestCasesRuntime()`? Without that the listing is incomplete. – KrishPrabakar Dec 03 '15 at 05:46
@KrishPrabakar you are free to implement it however you feel like. The answer just shows how two different patterns in JUnit 3 are mapped to JUnit 4 constructs. – Danail Nachev Dec 15 '15 at 22:32
1

In other words, there IS no "free" solution to do this, we have to build a list of test classes manually? Well that's rubbish... – Manius Mar 16 '18 at 20:11

score 35 · Answer 2 · answered Aug 13 '12 at 10:08

35

I've tried this using JUnit 4.8 and it works:

@RunWith(AllTests.class)
public class SomeTests
{
    public static TestSuite suite()
    {
        TestSuite suite = new TestSuite();

        suite.addTest(new JUnit4TestAdapter(Test1.class));
        suite.addTest(new JUnit4TestAdapter(Test2.class));

        return suite;
     }
}

answered Aug 13 '12 at 10:08

Andrejs

26,885
12
107
96

1

What do I do with this method? I want the suite be become a child of my parent suite http://stackoverflow.com/questions/18834908 – Val Sep 17 '13 at 14:48
I tried that but when I add `@BeforeClass` annotation it, the code inside `@BeforeClass` doesn't work. Any idea? – Gokhan Arik Jul 02 '14 at 20:46
What does `JUnit4TestAdapter` do? – Alex Oct 14 '15 at 10:10
@Arik `AllTests` runner does not favor `@BeforeClass`. Use [Suite sublcass](http://stackoverflow.com/a/1070568/902859) – Alex Oct 22 '15 at 19:32

score 26 · Accepted Answer · edited Oct 14 '15 at 11:17

26

I found Classpath suite quite useful when used with a naming convention on my test classes.

https://github.com/takari/takari-cpsuite

Here is an example:

import org.junit.extensions.cpsuite.ClasspathSuite;
import org.junit.runner.RunWith;

@RunWith(ClasspathSuite.class)
@ClassnameFilters({".*UnitTest"})
public class MySuite {
}

edited Oct 14 '15 at 11:17

Alex

367
1
3
17

answered Jul 20 '10 at 12:18

JavaRocky

19,203
31
89
110

6

Andrejs's answer makes Classpath redundant, at least for the task of creating test suites dynamically. – Oleg Mikheev Sep 30 '12 at 20:44
Looks like it isn't maintained anymore, and has lot of troubles with native libraries and such – naab Jun 08 '16 at 13:51

Brad Cupit · Answer 4 · 2012-06-15T18:46:21.947

6

I'm not sure what gatherTestClasses() does, but let's say it returns some tests when the OS is Linux and different tests when the OS is Windows. You can replicate that in JUnit 4.4 with assumptions:

@Test
public void onlyOnLinux() {
    assumeThat(getOS(), is(OperatingSystem.LINUX));
    // rest of test
}

@Test
public void onlyOnWindows() {
    assumeThat(getOS(), is(OperatingSystem.WINDOWS));
    // rest of test
}

@Test
public void anyOperatingSystem() {
    // just don't call assumeThat(..)
}

The implementation of getOS() and OperatingSystem being your custom code.

edited Jun 15 '12 at 18:46

answered Jul 15 '10 at 16:35

Brad Cupit

6,530
8
55
60

2

Perfect, just what I was looking for. If you're using apache commons-lang, you can use "assumeTrue(SystemUtils.IS_OS_WINDOWS);" – Jason Day Jun 15 '12 at 17:05

score 0 · Answer 5 · answered Feb 23 '17 at 18:21

Here is a Complete example how to implement that. it combines of two testCase classes and one suite.

ExampleInstrumentedTest:

import android.support.test.rule.ActivityTestRule;

import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;

@RunWith(JUnit4.class)
public class ExampleInstrumentedTest {


    @Rule
    public ActivityTestRule<MainActivity> mActivityTestRule = new ActivityTestRule<>(MainActivity.class);

    @Test
    public void checkInputs() throws Exception {

    }
}

ExampleInstrumentedTest2:

import android.support.test.rule.ActivityTestRule;

import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;

@RunWith(JUnit4.class)
public class ExampleInstrumentedTest2 {


    @Rule
    public ActivityTestRule<MainActivity> mActivityTestRule = new ActivityTestRule<>(MainActivity.class);

    @Test
    public void checkInputs() throws Exception {

    }
}

ExampleInstrumentedSuite:

import junit.framework.TestSuite;

import org.junit.runner.RunWith;
import org.junit.runners.AllTests;

@RunWith(AllTests.class)
public class ExampleInstrumentedSuite {

    public static TestSuite suite() {
        TestSuite suite = new TestSuite();
        suite.addTest(new junit.framework.JUnit4TestAdapter(ExampleInstrumentedTest.class));
        suite.addTest(new junit.framework.JUnit4TestAdapter(ExampleInstrumentedTest2.class));
        return suite;
    }
}

Note that you should use @RunWith(JUnit4.class) instead of default @RunWith(AndroidJUnit4.class) in testCase Class

score 0 · Answer 6 · answered Mar 13 '21 at 11:18

public class MyTestCase extends TestCase {
    @Override
    public void runTest() {
        // define assertion here  <===
        assertEquals("yes", "yes");
    }
}

@RunWith(AllTests.class)
public class DynamicTestSuite {
    public static TestSuite suite() {
        TestSuite suite = new TestSuite();

        // dynamically create your test case here  <====
        suite.addTest(new MyTestCase());

        return suite;
    }
}

score 0 · Answer 7 · answered Dec 02 '22 at 16:33

Expanding on @kissLife's answer, here's a something you can paste and run that creates multiple tests on the fly:

import junit.framework.TestCase;
import junit.framework.TestSuite;

public final class TestJunit4DynamicConstruction {
    public static TestSuite suite() {
        TestSuite suite = new TestSuite();
        suite.addTest(new CompareInts(1, 1));
        suite.addTest(new CompareInts(2, 2));
        suite.addTest(new CompareInts(2, 1)); // huh, for some reason, 2 != 1
        suite.addTest(new CompareInts(1, 1));
        return suite;
    }

    static public class CompareInts extends TestCase {
        private final int got;
        private final int expected;

        CompareInts(int got, int expected) {
            super(Integer.toString(got) + ":" + Integer.toString(expected));
            this.got = got;
            this.expected = expected;
        }
        @Override
        public void runTest() {
            assertEquals(got, expected);
        }
    }
}

You'll run these tests:

TestJunit4DynamicConstruction$CompareInts.1:1
TestJunit4DynamicConstruction$CompareInts.2:2
TestJunit4DynamicConstruction$CompareInts.2:1
TestJunit4DynamicConstruction$CompareInts.1:1
TestJunit4DynamicConstruction$CompareInts

and get this error:

junit.framework.AssertionFailedError: 
Expected :2
Actual   :1


    ...
TestJunit4DynamicConstruction$CompareInts.runTest(TestJunit4DynamicConstruction.java:26)
   ...


Process finished with exit code 255

How do I Dynamically create a Test Suite in JUnit 4?

7 Answers7

Linked

Related