How to get Cobertura to fail M2 build for low code coverage

java maven-2 build-automation code-coverage cobertura
Andrew Swan picture Andrew Swan · Oct 13, 2008 · Viewed 9.5k times · Source

I'm trying to configure my WAR project build to fail if the line or branch coverage is below given thresholds. I've been using the configuration provided on page 455 of the excellent book Java Power Tools, but with no success. Here's the relevant snippet of my project's Maven 2 POM:

<build>
...
<plugins>
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>cobertura-maven-plugin</artifactId>
    <version>2.2</version>
    <configuration>
      <check>
        <!-- Per-class thresholds -->
        <lineRate>80</lineRate>
        <branchRate>80</branchRate>
        <!-- Project-wide thresholds -->
        <totalLineRate>90</totalLineRate>
        <totalBranchRate>90</totalBranchRate>
      </check>
      <executions>
        <execution>
          <goals>
            <goal>clean</goal>
            <goal>check</goal>
          </goals>
        </execution>
        <execution>
          <id>coverage-tests</id>
          <!-- The "verify" phase occurs just before "install" -->
          <phase>verify</phase>
          <goals>
            <goal>clean</goal>
            <goal>check</goal>
          </goals>
        </execution>
      </executions>
      <instrumentation>
        <excludes>
      <exclude>au/**/*Constants.*</exclude>
        </excludes>
        <ignores>
      <ignore>au/**/*Constants.*</ignore>
        </ignores>
      </instrumentation>
    </configuration>
  </plugin>
  ...
</plugins>
...
</build>

As I say, the coverage report works fine, the problem is that the "install" goal isn't failing as it should if the line or branch coverage is below my specified thresholds. Does anyone have this working, and if so, what does your POM look like and which version of Cobertura and Maven are you using? I'm using Maven 2.0.9 and Cobertura 2.2.

I've tried Googling and reading the Cobertura docs, but no luck (the latter are sparse to say the least).

Answer

Pascal Thivent picture Pascal Thivent · Oct 24, 2009

To my knowledge, if the <haltOnFailure> element is set to true and any of the specified checks fails, then Cobertura will cause the build to fail which is what you're asking for. But actually, this element defaults to true if you do not specify it so you don't have to add it to your configuration checks. Failing the build below any coverage threshold is (or at least should be) the default behavior.

EDIT: I did some further testing and haltOnFailure seems to be working as expected on my environment (Maven 2.2.1. and versions 2.3, 2.2, 2.1 of the plugin i.e. versions 1.9.2, 1.9, 1.8 of cobertura on Linux). I'm updating this answer with the result below.

Actually, I've added an <execution> element to my pom. I may be misinterpreting the part of cobertura:check's documentation that says it "Binds by default to the lifecycle phase: verify" but, without the <execution> element, cobertura:check wasn't triggered during the verify phase of my build. Below the setup I've use for the cobertura-maven-plugin:

<project>
  ...
  <build>
    ...
    <plugins>
      ...
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>cobertura-maven-plugin</artifactId>
        <version>2.3</version>
        <configuration>
          <check>
            <!--<haltOnFailure>true</haltOnFailure>--><!-- optional -->
            <!-- Per-class thresholds -->
            <lineRate>80</lineRate>
            <branchRate>80</branchRate>
            <!-- Project-wide thresholds -->
            <totalLineRate>90</totalLineRate>
            <totalBranchRate>90</totalBranchRate>
          </check>
        </configuration>
        <executions>
          <execution>
            <phase>verify</phase>
            <goals>
              <!--<goal>clean</goal>--><!-- works if uncommented -->
              <goal>check</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

I get the following result when running mvn clean install on a freshly generated maven project (with mvn archetype:create) patched with the plugin configuration mentioned above: 

$ mvn archetype:create -DgroupId=com.mycompany.samples -DartifactId=cobertura-haltonfailure-testcase
...
$ mvn clean install
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building cobertura-haltonfailure-testcase
[INFO]    task-segment: [clean, install]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting directory /home/pascal/Projects/cobertura-haltonfailure-testcase/target
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/main/resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Compiling 1 source file to /home/pascal/Projects/cobertura-haltonfailure-testcase/target/classes
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/test/resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Compiling 1 source file to /home/pascal/Projects/cobertura-haltonfailure-testcase/target/test-classes
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /home/pascal/Projects/cobertura-haltonfailure-testcase/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.mycompany.samples.AppTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.09 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [jar:jar {execution: default-jar}]
[INFO] Building jar: /home/pascal/Projects/cobertura-haltonfailure-testcase/target/cobertura-haltonfailure-testcase-1.0-SNAPSHOT.jar
[INFO] Preparing cobertura:check
[WARNING] Removing: check from forked lifecycle, to prevent recursive invocation.
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/main/resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [cobertura:instrument {execution: default}]
[INFO] Cobertura 1.9.2 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Instrumenting 1 file to /home/pascal/Projects/cobertura-haltonfailure-testcase/target/generated-classes/cobertura
Cobertura: Saved information on 1 classes.
Instrument time: 337ms

[INFO] Instrumentation was successful.
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/pascal/Projects/cobertura-haltonfailure-testcase/src/test/resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /home/pascal/Projects/cobertura-haltonfailure-testcase/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.mycompany.samples.AppTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.098 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] [cobertura:check {execution: default}]
[INFO] Cobertura 1.9.2 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
Cobertura: Loaded information on 1 classes.

[ERROR] com.mycompany.samples.App failed check. Line coverage rate of 0.0% is below 80.0%
Project failed check. Total line coverage rate of 0.0% is below 90.0%

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Coverage check failed. See messages above.
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18 seconds
[INFO] Finished at: Sat Oct 24 21:00:39 CEST 2009
[INFO] Final Memory: 17M/70M
[INFO] ------------------------------------------------------------------------
$

I didn't test with maven 2.0.9, but on my machine, haltOnFailure generates a BUILD ERROR and halt the build. I don't see any differences with your plugin configuration, I can't reproduce the behavior you describe.

Related questions

How can I create an executable JAR with dependencies using Maven?

I want to package my project in a single executable JAR for distribution. How can I make a Maven project package all dependency JARs into my output JAR?

Read More
Controlling maven final name of jar artifact

I'm trying to define a property in our super pom which will be used by all child projects as the destination of the generated artifact. For this I was thinking about using project/build/finalName yet this does not seem …

Read More
Why does Maven have such a bad rep?

There is a lot of talk on the internet about how Maven is bad. I have been using some features of Maven for a few years now and the most important benefit in my view is the dependency management. Maven …

Read More