Surefire is not picking up Junit 4 tests

Question

For some reason I cannot get Maven 2 Surefire plugin to execute JUnit 4 test class.

public class SimpleTest {
  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

However if I change this class to be JUnit-3 like, such as

public class SimpleTest extends junit.framework.TestCase {
  public void testBar() {
     System.out.println("bar");
  }

  @org.junit.Test
  public void simple() {
     System.out.println("foo");
  }
}

then it gets executed. Here's what I've done:

verified Maven version: Apache Maven 2.2.1 (r801777; 2009-08-06 20:16:01+0100)
verified Surefire version: followed this advice
verified Surefire version: checked Surefire jars in my ~/.m2/repository/org/apache/maven/surefire -- all of them are either version 2.4.2 or 2.4.3
done a mvn dependency:tree | grep junit to ensure I only depend on junit version 4.7

The module I am having this problem at doesn't have JUnit 3 tests.

Is there anything else I am missing?

I can't post entire POM for both legal reasons and length - it's a big project, with hundreds of lines just in the POM. I could probably post relevant part(s), but not sure what exactly can be relevant. — mindas, Jan 07 '10 at 17:00
Well, at least the maven-surefire-plugin configuration (if you have one or if you are inheriting one). Can you say if this problem is specific to one module? Does it happen outside the project? Are you inheriting from a corporate POM? — Pascal Thivent, Jan 07 '10 at 17:16
Can you provide detail on what the symptoms of "can't execute" are? — Kaleb Pederson, Jan 07 '10 at 17:49

score 42 · Accepted Answer · answered Jan 08 '10 at 12:33

mvn -X helped me to reveal the following:

...
[INFO] [surefire:test {execution: default-test}]
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3:runtime (selected for runtime)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:runtime (selected for runtime)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-booter/2.4.3/surefire-booter-2.4.3.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/apache/maven/surefire/surefire-api/2.4.3/surefire-api-2.4.3.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG]   org.testng:testng:jar:jdk15:5.8:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/org/testng/testng/5.8/testng-5.8-jdk15.jar
[DEBUG] Adding to surefire booter test classpath: /home/mindas/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar
[DEBUG] dummy:dummy:jar:1.0 (selected for null)
[DEBUG] Retrieving parent-POM: org.apache.maven.surefire:surefire-providers:pom:2.4.3 for project: null:surefire-testng:jar:null from the repository.
[DEBUG] Adding managed dependencies for unknown:surefire-testng
[DEBUG]   org.apache.maven.surefire:surefire-api:jar:2.4.3
[DEBUG]   org.apache.maven.surefire:surefire-booter:jar:2.4.3
[DEBUG]   org.codehaus.plexus:plexus-utils:jar:1.5.1
[DEBUG]   org.apache.maven.surefire:surefire-testng:jar:2.4.3:test (selected for test)
[DEBUG]     org.apache.maven:maven-artifact:jar:2.0:test (selected for test)
[DEBUG]       org.codehaus.plexus:plexus-utils:jar:1.0.4:test (selected for test)
[DEBUG]     junit:junit:jar:3.8.1:test (selected for test)
[DEBUG]     org.testng:testng:jar:jdk15:5.7:test (selected for test)
[DEBUG]     org.apache.maven.surefire:surefire-api:jar:2.4.3:test (selected for test)
...
[DEBUG] Test Classpath :
...
[DEBUG]   /home/mindas/.m2/repository/junit/junit/4.7/junit-4.7.jar

So it seems that the problem was coming from testng jar requiring JUnit v3.8.1. Even though Test Classpath was set to depend on JUnit 4, it was too late.

testng dependency was located in my POM:

<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>5.8</version>
  <scope>test</scope>
  <classifier>jdk15</classifier>
</dependency>

Immediately after I have commented it out, tests started to execute.

Lessons learned:

mvn dependency:tree is not always enough, mvn -X is a friend.
surefire is not made for developer heaven (I have realized this while looking at project JIRA reports). This is especially true as there are no other alternatives if you use Maven.

Thanks everybody for your help. Unfortunately there is no way to split answer points between Pascal and Kaleb, but Kaleb's advice to use mvn -X helped me to get on the right track so correct answer points go to him.

Actually, you should accept your own answer as it is providing the whole solution to your problem:) — Pascal Thivent, Jan 08 '10 at 13:00

score 32 · Answer 2 · answered Apr 05 '11 at 21:24

The Surefire plugin figures out which JUnit provider should be used based upon the classpath. If there are multiple JUnit versions on the classpath, you can either correct the classpath to have only one JUnit version on the classpath (as discussed above), or you can explicitly specify which provider you want to use. For example, specifying the following in your (parent) POM forces using the newest provider (e.g., "surefire-junit47"):

[...]
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.8</version>
  <dependencies>
    <!-- Force using the latest JUnit 47 provider -->
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.8</version>
    </dependency>
  </dependencies>
[...]

Note however that Surefire 2.7 changed the way it's determining which unit test classes are run. The new behavior when using Surefire 2.7 (or later) with JUnit 4 is that any test without a @Test annotation will be skipped automatically. This may be great if you just have JUnit 4 unit tests, but if you have a combination of JUnit 3 and 4 unit tests, using the "surefire-junit47" provider will not work correctly. In such cases, its best to explicitly choose the "surefire-junit4" provider:

[...]
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.8</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <!-- Use the older JUnit 4 provider -->
      <artifactId>surefire-junit4</artifactId>
      <version>2.8</version>
    </dependency>
  </dependencies>
[...]

score 14 · Answer 3 · answered Jan 07 '10 at 17:56

I don't know what you mean by "can't execute," but does it help to explicitly set the includes used by the maven-surefire-plugin?

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
        </includes>
    </configuration>
</plugin>

Also, does running maven with the -X flag provide any useful information?

Thanks for your tip to use -X, this helped me to reveal the problem (see my own answer) — mindas, Jan 08 '10 at 12:34

Gábor Lipták · Answer 4 · 2016-10-05T11:47:08.843

13

One more possible cause can be this bug (closed with "Won't fix"): https://issues.apache.org/jira/browse/SUREFIRE-587

Short summary: Tests extending TestCase (but not using annotations) will not be picked up if their name does not end with "Test".

edited Oct 05 '16 at 11:47

answered Jul 04 '13 at 12:15

Gábor Lipták

9,646
2
59
113

1

No, this wasn't the case for me (see my own answer for the real cause). Anyway, thanks for the tip - might be helpful for other people. – mindas Jul 06 '13 at 20:00
Yeah, I saw your solution. I had the same question, and for me the solution was this bug. Thats why i shared it here. – Gábor Lipták Jul 07 '13 at 21:13
1

Thanks! I spent several hours debugging because of this "feature". – Gal Jul 29 '13 at 07:28
1

Updated link for SUREFIRE-587 : https://issues.apache.org/jira/browse/SUREFIRE-587 Thanks for the hint Gábor. – ocroquette Oct 05 '16 at 11:35

score 13 · Answer 5 · answered Apr 18 '21 at 11:55

13

In case you have JUnit 5 (import org.junit.jupiter.api.Test;) in your dependencies and you're trying to run some JUnit4 (import org.junit.Test;) tests, make sure you include following dependency

<dependency>
  <groupId>org.junit.vintage</groupId>
  <artifactId>junit-vintage-engine</artifactId>
  <scope>test</scope>
</dependency>

answered Apr 18 '21 at 11:55

anandbibek

1,659
16
20

Noted: I upgrade my service from 2.3.5.RELEASE to 2.6.7 - Add dependency junit 4.13.2 and junit-vintage-engine - Swagger 2 change to use springdoc-openapi-ui 1.6.8 - management.info.env.enabled=true for path /info – Visarut Sae-Pueng May 06 '22 at 04:54
This helped me. We have a confusing mix of junit4 and 5 tests. – TeaCoder Sep 12 '22 at 06:04

James Selvakumar · Answer 6 · 2016-09-13T00:49:38.677

For some poor soul out there who is wondering why Maven doesn't pick up JUnit tests.

I have both JUnit and TestNG as dependecies. But I want failsafe to run my functional tests using TestNG and surefire to run my unit tests using JUnit.

However, I found that surefire was attempting to run my unit tests using TestNG and found nothing to run. My JUnit tests were skipped.

Later I came across this Maven issue and configured surefire to run only "JUnit" tests like this:

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <properties>
      <property>
        <name>junit</name>
        <value>true</value>
      </property>
    </properties>
  </configuration>
</plugin>

Hope this helps someone.

This particular solution helped me !! – Kushal Paudyal Sep 20 '18 at 19:27 — Kushal Paudyal, Sep 20 '18 at 19:27

score 5 · Answer 7 · answered Aug 03 '11 at 10:32

For the benefit of Googlers, when I had this issue it was because I'd included a PowerMock dependency that pulled in TestNG, causing no [TestNG] tests to be detected by SureFire.

I used the m2eclipse "Dependency Hierarchy" tab of the POM editor to find the dependency and right-clicked to generate an exclusion (see XML below).

For completeness (and for those not using m2eclipse) here's the XML that excludes the dependency - I only came across this feature of Maven by seeing these tags generated automatically:

<dependency>
  <groupId>org.powermock</groupId>
  <artifactId>powermock-mockito-release-full</artifactId>
  <version>1.4.9</version>
  <classifier>full</classifier>
  <exclusions>
    <exclusion>
      <artifactId>powermock-module-testng</artifactId>
      <groupId>org.powermock</groupId>
    </exclusion>
  </exclusions>
</dependency>

(In my case, excluding "powermock-module-testng" was sufficient, but you could exclude TestNG directly if it's coming in from somewhere else.)

Thank you so much for this tip. I had the same issue and it was because of PowerMock. Again, big help as without this post I would have been looking in the wrong direction for some time. — AHungerArtist, Sep 22 '11 at 12:39

score 3 · Answer 8 · answered May 10 '17 at 10:09

1.) Include following surefire plugin in pom.xml

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
</configuration>
</plugin>

2.)By default surefire plugin picks Test class from package :- src/test/java/....

So go to Build Path and include test folder in classpath like below :

3.) Go to --> Run As -->Maven Test

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.081 
s - in com.bnym.dcm.api.controller.AccountControllerTest
[INFO] Running com.bnym.dcm.api.controller.DCMApiControllerTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - 
in com.bnym.dcm.api.controller.DCMApiControllerTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ----------------------------------------------------------------------
--
[INFO] BUILD SUCCESS

MadhuGujjalapudi · Answer 9 · 2014-02-21T11:12:54.243

2

One small change helped me Funnily!!!

I changed the class name from MyClassTest to TestMyClass, I got this idea after veryfing my parent POM.xml contains below line

<test.include.pattern> **/Test*.java <test.include.pattern/>

edited Feb 21 '14 at 11:12

answered Feb 21 '14 at 11:07

MadhuGujjalapudi

41
3

Pascal Thivent · Answer 10 · 2010-01-07T17:40:24.923

The verification that you've done are good, especially checking that you are using version 2.3+ of the surefire plugin (by default, you'll get version 2.4.3 with maven 2.1 super POM so this should be ok) and checking that you not pulling the junit-3.8.1.jar dependency transitively.

Now, just to validated that this is not a "global problem" (I don't think so TBH), could you create a project from scratch, for example by running:

mvn archetype:create -DgroupId=com.mycompany.app -DartifactId=maven-junit4-testcase

Then update the junit dependency:

<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.7</version>
  <scope>test</scope>
</dependency>

And configure the compiler level for 1.5+

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.5</source>
    <target>1.5</target>
  </configuration>
</plugin>

Finally put your SimpleTest.java next to AppTest.java and run mvn test.

If running mvn test works fine for that project (and I'm expecting it to run without problem), could you please update your question with the POM configuration you're using (from the project having troubles)?

I did what you suggested and indeed both tests (AppTest and my test were executed). Will try to dig this deeper tomorrow and/or post the relevant POM bits. Thanks a lot Pascal! Your method could benefit others while debugging same problem, too. Btw, can you modify your plugin mvn code - you forgot to close plugin tag at the end. — mindas, Jan 07 '10 at 17:37
Debugging shows it pulls junit-3.8.1.jar transitively. How do I override it. — Akshay Hazari, Mar 01 '21 at 05:33

score 0 · Answer 11 · answered May 19 '17 at 20:01

0

Had a similar issue when trying to run integration tests. I had an old version of the surefire plugin which was trying to run TestNG and not jUnit. I changed the version number in the pom to 2.20 and it worked.

answered May 19 '17 at 20:01

GeeMoo

1
1

Brent Bradburn · Answer 12 · 2020-07-28T21:01:37.603

0

If you're using Maven, the filename matters.

By default, test filenames need to either start or end with "Test" or "Tests", or they will be silently ignored when searching for tests.

https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html

edited Jul 28 '20 at 21:01

answered Jul 28 '20 at 02:02

Brent Bradburn

51,587
17
154
173

score -1 · Answer 13 · edited Mar 07 '18 at 13:35

-1

Have you configured your maven-compile-plugin for the correct compiler level, like:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <configuration>
    <source>1.5</source>
    <target>1.5</target>
  </configuration>
</plugin>

Otherwise maven will have trouble with annotations

edited Mar 07 '18 at 13:35

infojolt

5,244
3
40
82

answered Jan 07 '10 at 16:39

Andreas Dolk

113,398
19
180
268

Yes, all modules have these settings. Thanks for your input, though! – mindas Jan 07 '10 at 16:50
The test would fail without 1.5+ compiler level, but it would be picked up. – Pascal Thivent Jan 07 '10 at 16:53
I know that it's just wild guessing - I never had trouble with Junit4 tests in maven... – Andreas Dolk Jan 07 '10 at 17:30
Me neither. I'm really wondering what could cause this weird behavior. – Pascal Thivent Jan 07 '10 at 17:42

Surefire is not picking up Junit 4 tests

13 Answers13

Linked