How do I run my cucumber-jvm features in parallel? (with maven using cucumber-jvm-parallel-plugin)

maven cucumber cucumber-jvm cucumber-junit cucumber-java
Katie picture Katie · Nov 11, 2016 · Viewed 14.2k times · Source

I am trying to incorporate a maven plugin cucumber-jvm-parallel-plugin into my Cucumber-JVM code and am running into some problems... I think I have configured my pom.xml properly, but my cucumber features are still getting ran one by one instead of in parallel.

How do I configure my pom.xml to make my cucumber features run in parallel? I have some feature files that block execution and I don't want to have to wait for them to finish in order to start testing the other features.

I followed two tutorials, and can't find where I am going wrong, or if this is expected:

Structure

Here's my program's structure:

├── features/
│   └── api/
│       ├── 006-Email.feature
│       └── 999-Login.feature
├── results/
│   └── api-json.json
└── src/
    └── java/
        ├── pom.xml
        ├── src/
        │   └── test/
        │       └── java/
        │           └── com/
        │               └── mycompany/
        │                   └── commonapps/
        │                       └── queuemanager/
        │                           ├── Globals.java
        │                           ├── GmailHelper.java
        │                           ├── HttpDeleteWithBody.java
        │                           ├── JsonHelper.java
        │                           ├── RequestHelper.java
        │                           ├── RunApiTest.java
        │                           ├── ThreadedSteps.java
        │                           └── UserHelper.java
        └── steps/
            └── com/
                └── mycompany/
                    └── commonapps/
                        └── queuemanager/
                            ├── GivenSteps.java
                            ├── Setup.java
                            ├── ThenSteps.java
                            └── WhenSteps.java

Maven Pom Logic

The way I configured my pom is as follows:

  1. I copy ../../features/api/ to my build path using Maven Resources Plugin 

    <build>
  <resources>
     <resource>
        <directory>../../features/api/</directory> 
     </resource>
  </resources>
</build>

  2. I tell cucumber-jvm-parallel-plugin that my features are now located at target/classes/active/, because I think ../../features/api/ was copied to target/, so target/active/ should contain my .feature files.. I think.

     <plugin>
      <groupId>com.github.temyers</groupId>
      <artifactId>cucumber-jvm-parallel-plugin</artifactId>
      ...
      <configuration>
          ...
          <featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory>
          ...
      </configuration>
 </plugin>

  3. Then, after I run maven test, all .feature files are then compiled into target/test-classes/Parallel**IT.class. One per feature file.

  4. So I tell maven-surefire-plugin to run all <include>**/Parallel*IT.class</include> files in parallel, which are the feature files created by cucumber-jvm-parallel-plugin.

     <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <executions>
       <execution>
          <id>acceptance-test</id>
          <phase>integration-test</phase>
          <goals>
             <goal>test</goal>
          </goals>
          <configuration>
            <forkCount>5</forkCount>
            <reuseForks>true</reuseForks>
            <includes>
              <include>**/Parallel*IT.class</include>
            </includes>
          </configuration>
       </execution>
    </executions>
 </plugin>

Full pom.xml

And my pom.xml is:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.mycompany.commonapps</groupId>
   <artifactId>e2e-server</artifactId>
   <version>0.0.1-SNAPSHOT</version>
   <dependencies>
     ...
   </dependencies>
   <build>
<!-- COPY RESOURCES TO TARGET/ -->
      <resources>
    <!-- copies features to target: "${project.build.directory}/classes/active/" -->
         <resource>
            <directory>../../features/api/</directory> 
         </resource>
      </resources>
<!-- RUN EACH CUCUMBER FEATURE FILE AS A FORK -->
      <plugins>
    <plugin>
      <groupId>com.github.temyers</groupId>
      <artifactId>cucumber-jvm-parallel-plugin</artifactId>
      <version>1.2.1</version>
      <executions>
        <execution>
          <id>generateRunners</id>
          <phase>generate-test-sources</phase>
          <goals>
        <goal>generateRunners</goal>
          </goals>
          <configuration>
          <glue>com.mycompany.commonapps.queuemanager</glue>
           <outputDirectory>${project.build.directory}/generated-test-sources</outputDirectory>
           <featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory>
           <cucumberOutputDir>${project.build.directory}</cucumberOutputDir>
           <format>json</format>
           <!--<strict>true</strict>
           <monochrome>true</monochrome> 
           <useTestNG>false</useTestNG>
           <namingScheme>simple</namingScheme>-->
           <parallelScheme>FEATURE</parallelScheme>
          </configuration>
        </execution>
      </executions>
    </plugin>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
<!-- RUN ALL CLASSES CREATED BY cucumber-jvm-parallel-plugin IN PARALLEL -->
        <executions>
         <execution>
             <id>acceptance-test</id>
             <phase>integration-test</phase>
             <goals>
                 <goal>test</goal>
             </goals>
             <configuration>
                 <forkCount>5</forkCount>
                 <reuseForks>true</reuseForks>
                 <includes>
                    <include>**/Parallel*IT.class</include>
                 </includes>
             </configuration>
        </execution>
        </executions>
         </plugin>
<!-- ADD ../../STEPS/ TO THE SOURCE -->
         <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.7</version>
            <executions>
               <execution>
                  <id>add-source</id>
                  <phase>generate-sources</phase>
                  <goals>
                     <goal>add-source</goal>
                  </goals>
                  <configuration>
                     <sources>
                        <source>steps/</source>
                        <source>src/test/java/</source>
                     </sources>
                  </configuration>
               </execution>
            </executions>
         </plugin>
      </plugins>
   </build>
</project>

Cucumber Runner

And my cucumber running class looks like this:

@RunWith(Cucumber.class)
@CucumberOptions(plugin={"pretty","json:../../results/api-json.json"}, glue={"com.mycompany.commonapps.queuemanager"}, features={"../../features/api/active"})
public class RunApiTest {
}

Output

When I run mvn test, I get the following:

...
[INFO] ------------------------------------------------------------------------
[INFO] Building e2e  Server Tests 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2
[INFO] 
[INFO] --- build-helper-maven-plugin:1.7:add-source (add-source) @ e2e--server ---
[INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/steps added.
[INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java added.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ e2e--server ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 7 resources
[INFO] Copying 2 resources
[INFO] Copying 7 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ e2e--server ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 12 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/classes
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations.
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- cucumber-jvm-parallel-plugin:1.2.1:generateRunners (generateRunners) @ e2e--server ---
[INFO] Adding /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/generated-test-sources to test-compile source root
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ e2e--server ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ e2e--server ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 10 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/test-classes
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations.
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ e2e--server ---
[INFO] Surefire report directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.mycompany.commonapps.queuemanager.RunApiTest
....
*Tests run one by one properly*
...
Tests run: 31, Failures: 2, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:40 min
[INFO] Finished at: 2016-11-10T15:49:11-08:00
[INFO] Final Memory: 20M/261M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project e2e--server: There are test failures.
[ERROR] 
[ERROR] Please refer to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Does anyone see what I did wrong?

More Debugging Info

  • cucumber-jvm-parallel-plugin's outputDirectory is getting created correctly and contains a bunch of Parallel**IT.class files
  • cucumber-jvm-parallel-plugin's featuresDirectory is getting accessed correctly.
  • cucumber-jvm-parallel-plugin's cucumberOutputDir is NOT getting created at all after a build.

Answer

Katie picture Katie · Nov 15, 2016

I finally got it to work! Here's what I had to do to make my tests run in parallel:

  1. I move some fields in the maven-surefire-plugin execution tag to the configuration tag. I moved this section:

         <forkCount>5</forkCount>
     <reuseForks>true</reuseForks>
     <includes>
        <include>**/*IT.class</include>
     </includes>
    • So that my maven-surefire-plugin now looks like this:

        <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.19.1</version>

      <configuration>
         <forkCount>5</forkCount>
         <reuseForks>true</reuseForks>
         <includes>
             <include>**/*IT.class</include>
         </includes>
      </configuration>

    </plugin>

  2. Since cucumber-jvm-parallel-plugin is now running the show, I had to remove my cucumber runner class. I commented it out like this:

    /*
    @RunWith(Cucumber.class)
    @CucumberOptions(plugin={"pretty","json:../../results/api-json.json"}, glue={"com.mycompany.commonapps.queuemanager"}, features={"../../features/api/active"})
    */
    public class RunApiTest {
    }

And that's it. Since Parallel**IT.class where getting created, it was maven-surefire-plugin that was not working right.

Related questions

Running Cucumber tests directly from executable jar

I have a project with cucumber and maven also I am using the JUnit. I am able to run and build my project successfully from Eclipse. Now I want to run the test from command line in another system which …

Read More
cucumber junit runner java.lang.NoSuchMethodError:

Trying to implement cucumber to do some automated tests. jUnit tests. I've created 2 files and edited pom.xml that comes with maven project to add dependencies. Content is shown below. The first of two files are cucumber .feature files, which …

Read More
Maven running cucumber specific feature files or folders

How is it possible to use the cucumber feature option from maven command line. Of a tree full of features (tests), I'm trying to run only individual features. In my case - all feature files in a selected folder. I've …

Read More