JMH not working in Eclipse (as Maven project) - No benchmarks to run

Question

I want to start have a look at JMH and I'm failing to run benchmarks due some reasons. Let me explain what I tried:

1. Setup a maven project in Eclipse

2. Define pom.xml like:

3. Downloaded some official JMH examples. As example I choosed which is pretty simple and a good place to start: http://hg.openjdk.java.net/code-tools/jmh/file/0c58dc4fcf17/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_01_HelloWorld.java

4. Right mouse click > Run As > Java Application

However this produces the output:

Exception in thread "main" No benchmarks to run; check the include/exclude regexps.
at org.openjdk.jmh.runner.Runner.run(Runner.java:155)
at org.openjdk.jmh.samples.JMHSample_01_HelloWorld.main(JMHSample_01_HelloWorld.java:90)

I have googled and it seems the above example should work, but it is not the case for me. I also tried to solve the issue by reading but this seems not working for me:

• "No matching benchmarks" when running JMH from main in eclipse

• I tried to move the generated class manually to /META-INF/MicroBenchmarks however this produces following error:

  Exception in thread "main" java.lang.IllegalStateException: Mismatched format for the line: JMHSample_01_HelloWorld.class
  at org.openjdk.jmh.runner.BenchmarkRecord.<init>(BenchmarkRecord.java:92)
  at org.openjdk.jmh.runner.MicroBenchmarkList.find(MicroBenchmarkList.java:133)
  at org.openjdk.jmh.runner.Runner.run(Runner.java:150)
  at JMHSample_01_HelloWorld.main(JMHSample_01_HelloWorld.java:80)
  

It seems like the JMH should produce some valid lines in /META-INF/MicroBenchmarks - meaning in this folder there should no generated java class files, correct?

Can anyone help me to find the mistake? Thank you.

Answer 1

The benchmark "source code", in addition to be compiled as usual, needs to be also processed by the JMH annotation processor. Prior to JMH 0.5 the annotation processor was part of the main JMH artifact, so a single dependency to jmh-core was sufficient. In JMH 0.5 the annotation processing functionality was extracted into separate artifact to facilitate support for other languages.

In order the get the annotation processor back in "old" Java-based JMH projects an additional dependency to the processor artifact should be declared:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>${jmh.version}</version>
    <!-- 
    the processor artifact is required only during compilation and 
    does not need to be transitive, hence provided scope
    -->
    <scope>provided</scope> 
</dependency>

For new JMH projects the easiest way is to generate a new project using the proper language-specific archetype (besides Java, there are Scala, Groovy and Kotlin archetypes), as described on the JMH home page.

mvn archetype:generate \
      -DinteractiveMode=false \
      -DarchetypeGroupId=org.openjdk.jmh \
      -DarchetypeArtifactId=jmh-java-benchmark-archetype \
      -DgroupId=my.benchmark.group \
      -DartifactId=MyBenchMarkArtifact \
      -Dversion=1.0