Maven failsafe plugin doesn't run parallel Test

Question

I have a Maven POM file and when I provide parallel execution options, I dont see any signs of parallel execution in the logs. And XML debugging is driving me nuts. Any thoughts what is wrong here?

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <parent>
      <groupId>com.xxx.xxxx.testing.ranger</groupId>
      <artifactId>ranger-parent</artifactId>
      <relativePath>../parent/pom.xml</relativePath>
      <version>1.0.0</version>
   </parent>
   <artifactId>ranger-api-tests</artifactId>
   <name>Ranger API Tests</name>
   <description>Ranger API integration tests. Depends on the Ranger test framework project.</description>
   <properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      <test.groups />
      <surefire.and.failsafe.report.dir>C:/Users/xxx/Documents/Projects/LearnCubeAPIIT/tests/target</surefire.and.failsafe.report.dir>
   </properties>
   <dependencies>
      <!-- Granite Testing framework -->
      <dependency>
         <groupId>com.xxx.xxxx.testing.ranger</groupId>
         <artifactId>ranger-api-it-framework</artifactId>
         <version>1.0.0</version>
      </dependency>
      <dependency>
         <groupId>commons-lang</groupId>
         <artifactId>commons-lang</artifactId>
         <version>2.4</version>
      </dependency>
      <dependency>
         <groupId>commons-httpclient</groupId>
         <artifactId>commons-httpclient</artifactId>
         <version>3.1</version>
      </dependency>
      <dependency>
         <groupId>org.apache.maven.surefire</groupId>
         <artifactId>surefire-junit47</artifactId>
         <version>2.16</version>
      </dependency>
      <dependency>
         <groupId>junit</groupId>
         <artifactId>junit</artifactId>
         <version>4.11</version>
      </dependency>
   </dependencies>
   <build>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.16</version>
            <configuration>
               <parallel>classes</parallel>
               <threadCount>4</threadCount>
               <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
               <groups>${test.groups}</groups>
            </configuration>
            <executions>
               <execution>
                  <goals>
                     <goal>integration-test</goal>
                     <goal>verify</goal>
                  </goals>
               </execution>
            </executions>
         </plugin>
      </plugins>
   </build>
   <reporting>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-report-plugin</artifactId>
            <version>2.16</version>
            <configuration>
               <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
            </configuration>
            <reportSets>
               <reportSet>
                  <id>integration-tests</id>
                  <reports>
                     <report>failsafe-report-only</report>
                     <report>report-only</report>
                  </reports>
               </reportSet>
            </reportSets>
         </plugin>
      </plugins>
   </reporting>
</project>

Logs from run - http://pastebin.com/WkCDwYzJ

Question - When parallel execution options are passed to failsafe plugin, how are the logs supposed to look like to confirm that parallel execution of tests is taking place?

UPDATE

I noticed that when I set debug option for maven to execute, I noticed that parallelExecution is being set to false

[DEBUG] Populating class realm plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12.4--1057774753
[DEBUG]   Included: org.apache.maven.plugins:maven-surefire-plugin:jar:2.12.4
[DEBUG]   Included: org.apache.maven.surefire:surefire-booter:jar:2.12.4
[DEBUG]   Included: org.apache.maven.surefire:surefire-api:jar:2.12.4
[DEBUG]   Included: org.apache.maven.surefire:maven-surefire-common:jar:2.12.4
[DEBUG]   Included: org.apache.commons:commons-lang3:jar:3.1
[DEBUG]   Included: org.apache.maven.shared:maven-common-artifact-filters:jar:1.3
[DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.0.8
[DEBUG]   Included: org.apache.maven.reporting:maven-reporting-api:jar:2.0.9
[DEBUG]   Included: org.apache.maven.plugin-tools:maven-plugin-annotations:jar:3.1
[DEBUG]   Excluded: org.apache.maven:maven-plugin-api:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-artifact:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-project:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-settings:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-profile:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-model:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-artifact-manager:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-plugin-registry:jar:2.0.9
[DEBUG]   Excluded: org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1
[DEBUG]   Excluded: junit:junit:jar:3.8.1
[DEBUG]   Excluded: org.apache.maven:maven-core:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-plugin-parameter-documenter:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-repository-metadata:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-error-diagnostics:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-plugin-descriptor:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-monitor:jar:2.0.9
[DEBUG]   Excluded: classworlds:classworlds:jar:1.1
[DEBUG]   Excluded: org.apache.maven:maven-toolchain:jar:2.0.9
[DEBUG] Configuring mojo org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test from plugin realm ClassRealm[plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12.4--1057774753, parent: sun.misc.Launcher$AppClassLoader@7987aeca]
[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test' with basic configurator -->
[DEBUG]   (s) basedir = /Users/nallagun/Git/testing/Ranger/tests
[DEBUG]   (s) childDelegation = false
[DEBUG]   (s) classesDirectory = /Users/nallagun/Git/testing/Ranger/tests/target/classes
[DEBUG]   (s) disableXmlReport = false
[DEBUG]   (s) enableAssertions = true
[DEBUG]   (s) forkMode = once
[DEBUG]   (s) junitArtifactName = junit:junit
[DEBUG]   (s) localRepository =        id: local
      url: file:///Users/nallagun/.m2/repository/
   layout: none

[DEBUG]   (s) parallel = classes
[DEBUG]   (f) parallelMavenExecution = false
[DEBUG]   (s) perCoreThreadCount = true

Any Idea why this might be happening?

Thanks

I am having issues with XML formatting, so didnt want to mess up the post :( — SuperMan, Oct 31 '13 at 21:55
That log you're seeing is related to the maven-surefire-plugin. I can't see configuration of this plugin. Do you have surefire configuration somewhere in your parent pom? — Boj, Nov 04 '13 at 08:38

score 7 · Answer 1 · edited May 23 '17 at 12:24

Firstly, make sure you're forcing a specific jUnit provider, since older versions (<4.7) are known to have issues with concurrent execution.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.16</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.16</version>
    </dependency>
  </dependencies>
</plugin>

You can use the forkCount for concurrent execution option instead of parallel. I find it more flexible and since each test run in its own JVM, you would have less concurrency issues that might make your test results non-deterministic. These configurations apply to maven-surefire-plugin and maven-failsafe-plugin.

<forkCount>8</forkCount>
<reuseForks>true</reuseForks>

You can combine forkCount and parallel, but the different possible combinations of forkCount, reuseForks, etc make things more complicated. Note this from the docs:

When using reuseForks=true and a forkCount value larger than one, test classes are handed over to the forked process one-by-one. Thus, parallel=classes would not change anything. However, you can use parallel=methods: classes are executed in forkCount concurrent processes, each of the processes can then use threadCount threads to execute the methods of one class in parallel.

I'm not sure the reports would tell you if tests ran in concurrent mode or not, but the way I know they are running is by using top or htop (on Linux or OS X). Here's a screenshot of one my test runs. You can see multiple JVMs running tests and consuming CPU. Also my tests run much faster in this fashion.

htop

Thanks, another bonus point question, how are you handling test logs? Do you let failsafe handle them or you use your own wrapper? — SuperMan, Nov 01 '13 at 02:42
I did try your updated answer by adding the dependency plugin, but doesn't help — SuperMan, Nov 04 '13 at 19:01
Add that to pom. Also add similar configuration for surefire. — Boj, Nov 05 '13 at 00:53
maven-failsafe-plugin-v2.18.1. Doesn't work. Still single process :( — AlikElzin-kilaka, Aug 20 '15 at 09:55
@Boj you solution didn't work in my case. I've multiple test cases in single class but they don't run in parallel. http://stackoverflow.com/questions/42115143/maven-failsafe-plugin-does-not-execute-test-cases-from-single-class-in-parallel — vikramvi, Feb 08 '17 at 14:30

Maven failsafe plugin doesn't run parallel Test

1 Answers1

Linked