maven-failsafe-plugin Errors and BUILD SUCCESS?

Question

my question is very similar to this one: maven-failsafe-plugin Failures and BUILD SUCCESS?

and I manage to set up failsafe plugin to fail if tests fail.

But if test goes into error state, failsafe plugin still does not break the build.

.................
-------------------------------------------------------
   T E S T S
-------------------------------------------------------
Running xxxxx.IntegrationTierFunctionalTestCase
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.054 sec <<< FAILURE!

Results :

Tests in error:
  testException(xxxxx.IntegrationTierFunctionalTestCas

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is
[INFO] [failsafe:verify {execution: functional-test-1024}]
[INFO] Failsafe report directory: C:\projects\oec-integration-server\trunk\oec-integrati
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is
[INFO] [failsafe:integration-test {execution: functional-test-24}]
[INFO] Failsafe report directory: C:\projects\oec-integration-server\trunk\oec-integrati
.............
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 58 seconds
[INFO] Finished at: Tue May 28 17:48:13 BST 2013
[INFO] Final Memory: 114M/781M
[INFO] ------------------------------------------------------------------------

for simplicy IntegrationTierFunctionalTestCase contains only this code

import org.junit.Test;
import static org.junit.Assert.fail;
public class IntegrationTierFunctionalTestCase 
{

    @Test
    public void testException(){
        //fail();
        throw new RuntimeException("super error");
    }
}

if I uncomment fail() whole build fails correctly, with build failed. but if I just throw an exception, it fails as on shown above.

oour plugin configuration looks like this

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.7</version>
    <configuration>
        <redirectTestOutputToFile>true</redirectTestOutputToFile>
        <systemPropertyVariables>
            <oec.env>TEST</oec.env>
            <mule.test.timeoutSecs>2400</mule.test.timeoutSecs>
        </systemPropertyVariables>
        <additionalClasspathElements>
            <additionalClasspathElement>${basedir}/src/main/resources/config</additionalClasspathElement>
            </additionalClasspathElement>
        </additionalClasspathElements>
    </configuration>
    <executions>
        <execution>
            <id>functional-test-1024</id>
            <phase>test</phase>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <includes>
                    <include>**/IntegrationTierFunctionalTestCase.java</include>
                </includes>
                <forkMode>once</forkMode>
                <argLine>-XX:MaxPermSize=256M -Xmx1024M</argLine>
            </configuration>
        </execution>
    </executions>
</plugin>

What am I missing? And no I do not want to wrap it in try-catch blocks and fail tests manually.

Answer 1

You need having two executions blocks, cause the verify goal of the maven-failsafe-plugin is intended to check the results of the integration tests.

  <executions>
    <execution>
        <id>functional-test-1024</id>
        <phase>test</phase>
        <goals>
            <goal>integration-test</goal>
        </goals>
        <configuration>
            <includes>
                <include>**/IntegrationTierFunctionalTestCase.java</include>
            </includes>
            <forkMode>once</forkMode>
            <argLine>-XX:MaxPermSize=256M -Xmx1024M</argLine>
        </configuration>
    </execution>
    <execution>
        <id>verify</id>
        <phase>verify</phase>
        <goals>
            <goal>verify</goal>
        </goals>
    </execution>
  </executions>

Furthermore you should update the version of the maven-failsafe-plugin to 2.14.1 instead of 2.7. Update: In the meantime update to 2.17.