How do I configure maven to print JUnit assertion failure message to console

Question

I'm using the surefire maven plugin to run unit tests. My test class looks like this:

public class Test1 {
  @org.junit.Test
  public void testThatFails(){
    Assert.assertTrue("false is never true", false);
  }
}

When a test fails I expect to see the message "false is never true" that was associated with the AssertionError, but it's not printed to the console. Instead I'm instructed to dig around in the surefire report directory and find it (which sucks).

Running com.example.Test1
Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.799 sec <<< FAILURE!
Running com.example.Test2
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
Running com.example.Test3
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.122 sec

Results :

Failed tests: 
testThatFails(com.example.Test1)

Tests run: 12, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

Please refer to /<path-to-pom.xml>/target/surefire-reports for the individual test results.

After browsing to above directory I find the .txt file which contains the error message from my JUnit test, including the stack trace. I would like to see that same message and stack trace printed to the console during build (if it also gets added to a report file that would be great). Any ideas?

Answer 1

There is a surefire configuration which will specify whether to write the test reports to a file or to the console: useFile. This defaults to true.

http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#useFile

With Maven, you can configure it to be false by including the following section in your pom.xml:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.9</version>
            <configuration>
                <useFile>false</useFile>
            </configuration>
        </plugin>
    </plugins>
</build>

This will generate output like:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.example.MyTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.055 sec <<< FA
ILURE!
foo(com.example.MyTest)  Time elapsed: 0.017 sec  <<< FAILURE!
java.lang.AssertionError: false is never true
        at org.junit.Assert.fail(Assert.java:91)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at com.example.MyTest.foo(MyTest.java:10)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
Method.java:44)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCal
lable.java:15)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMe
thod.java:41)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMet
hod.java:20)
        at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4Cla
ssRunner.java:79)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRun
ner.java:71)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRun
ner.java:49)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.
java:53)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4
Provider.java:123)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider
.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(
ReflectionUtils.java:164)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke
(ProviderFactory.java:110)
        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(Suref
ireStarter.java:172)
        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWh
enForked(SurefireStarter.java:104)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:
70)

Results :

Failed tests:   foo(com.example.MyTest): false is never true

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.675s
[INFO] Finished at: Wed Aug 10 13:15:46 CST 2011
[INFO] Final Memory: 7M/153M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.
9:test (default-test) on project test-output: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Code\experiments\test-output\target\surefire-reports
for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExc
eption

NOTE:

Using this configuration setting may break the reporting mechanisms of CI servers which show test failures. CI servers like TeamCity and Jenkins use the surefire-reports.txt files to show detailed test failure information and it looks like this setting makes these reports not get generated. Unfortunately, I couldn't find a way to make the report show in both console and get written to a file.