java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;

Question

I've been trying to connect kafka to elasticsearch using kafka-connect api.Kafka version is 0.11.0.0.These are the steps I followed:

1.Buiding Elasticsearch Connector:

https://github.com/confluentinc/kafka-connect-elasticsearch.git

2.Build the connector

$ cd kafka-connect-elasticsearch
$ mvn clean package

3.Finally running the script:

$ bin/connect-standalone.sh config/connect-standalone.properties config/elasticsearch-connect.properties

It throws the following exception:

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;
    at org.reflections.Reflections.expandSuperTypes(Reflections.java:380)
    at org.reflections.Reflections.<init>(Reflections.java:126)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:221)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:198)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:159)
    at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
    at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:68)

Can't understand what's going wrong.

Answer 1

From experience, this error means that you are bringing an older version of guava earlier in your classpath. Connect worker requires guava >= 20 for org.reflections to work correctly.

kafka-connect-elasticsearch or any other connector that brings with it guava 18.0 or older will prohibit the worker from starting up. This error message means that the older guava jar was encountered first in the classpath.

Two solutions:

1. Indeed, as Hans Jespersen mentions, using classloading isolation by setting your plugin.path in Connect worker's configuration, will allow the connector to work as-is without interfering with the Connect framework.
2. If adding the connector to the CLASSPATH is your only option, make sure it's added after Kafka Connect's dependencies, so that the most recent guava will be picked up.