I am working on Lettuce cluster Java client. It is set up inside a bolt
topology (Apache Strom). spout
is reading data from kafka
and passing it to bolt
. However, when I am starting my topology, I am getting below error message and program terminated. Am I missing something? what is causing this?
29502 [Thread-17-RecommendationLettuceBolt-executor[2 2]] ERROR o.a.s.util - Async loop died! io.lettuce.core.RedisException: Cannot retrieve initial cluster partitions from initial URIs [RedisURI [host='127.0.0.1', port=7001]] at io.lettuce.core.cluster.RedisClusterClient.loadPartitions(RedisClusterClient.java:865) ~[lettuce-core-5.1.7.RELEASE.jar:?] at io.lettuce.core.cluster.RedisClusterClient.initializePartitions(RedisClusterClient.java:819) ~[lettuce-core-5.1.7.RELEASE.jar:?] at io.lettuce.core.cluster.RedisClusterClient.connect(RedisClusterClient.java:345) ~[lettuce-core-5.1.7.RELEASE.jar:?] at com.projectName.indexer.lettuce.LettuceClusterClientProvider.getConnection(LettuceClusterClientProvider.java:72) ~[classes/:?] at com.projectName.indexer.lettuce.LettuceCacheRepopulationHandler.openLettuceConnection(LettuceCacheRepopulationHandler.java:42) ~[classes/:?] at com.projectName.indexer.bolts.RecommendationLettuceBolt.prepare(RecommendationLettuceBolt.java:35) ~[classes/:?] at org.apache.storm.daemon.executor$fn__8058$fn__8071.invoke(executor.clj:795) ~[storm-core-1.0.2.jar:1.0.2] at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:482) [storm-core-1.0.2.jar:1.0.2] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.base/java.lang.Thread.run(Thread.java:844) [?:?] Caused by: io.lettuce.core.RedisConnectionException: Unable to establish a connection to Redis Cluster at [RedisURI [host='127.0.0.1', port=7001]] at io.lettuce.core.cluster.topology.AsyncConnections.get(AsyncConnections.java:89) ~[lettuce-core-5.1.7.RELEASE.jar:?] at io.lettuce.core.cluster.topology.ClusterTopologyRefresh.loadViews(ClusterTopologyRefresh.java:73) ~[lettuce-core-5.1.7.RELEASE.jar:?] at io.lettuce.core.cluster.RedisClusterClient.doLoadPartitions(RedisClusterClient.java:871) ~[lettuce-core-5.1.7.RELEASE.jar:?] at io.lettuce.core.cluster.RedisClusterClient.loadPartitions(RedisClusterClient.java:844) ~[lettuce-core-5.1.7.RELEASE.jar:?] ... 9 more
private void init() {
redisUri = RedisURI.Builder
.redis(lettuceConfig.getLettuceClusterHost())
.withPort(lettuceConfig.getLettuceClusterPort())
.withTimeout(Duration.ofMillis(lettuceConfig.getLettuceClusterTimeout()))
.build();
}
public StatefulRedisClusterConnection getConnection() {
if (connection == null || !connection.isOpen()) {
redisClusterClient = RedisClusterClient.create(redisUri);
final ClusterTopologyRefreshOptions topologyRefreshOptions = ClusterTopologyRefreshOptions.builder()
.enablePeriodicRefresh(Duration.ofMinutes(BoltConstants.Lettuce.PERIODIC_REFRESH_TIME_IN_MIN))
.enableAdaptiveRefreshTrigger()
.build();
final ClusterClientOptions clusterClientOptions = ClusterClientOptions.builder()
.autoReconnect(true)
.topologyRefreshOptions(topologyRefreshOptions)
.build();
redisClusterClient.setOptions(clusterClientOptions);
connection = redisClusterClient.connect(SnappyCompressor.wrap(new StringCodec()));
log.info("Connected to Redis client lettuce. lettuce connection is up and running.");
}
return connection;
}
compile 'io.lettuce:lettuce-core:5.1.7.RELEASE'
The error was coming because I have not installed redis cluster locally on my machine.
I solved using these steps
https://redis.io/download and run below commands Installation
$ wget http://download.redis.io/releases/redis-5.0.5.tar.gz
$ tar xzf redis-5.0.5.tar.gz
$ cd redis-5.0.5
$ make //The binaries that are now compiled are available in the src directory
$ src/redis-server
Creating a Redis Cluster using the create-cluster script
Now check redis-5.0.5/utils/create-cluster
directory in the Redis distribution. There is a script called create-cluster
inside, it's a simple bash script. In order to start a 6 nodes cluster with 3 masters and 3 slaves just type the following commands:
$ create-cluster start // start
$ create-cluster create // create cluster
$ create-cluster stop // for stoping
$ create-cluster clean // clean all the cluster
You will see the following output in the terminal
$ ps aux |grep redis
300067846 28222 0.6 0.0 4379932 2612 ?? Ss 7:41PM 0:00.73 ../../src/redis-server *:30004 [cluster]
300067846 28227 0.5 0.0 4380956 2604 ?? Ss 7:41PM 0:00.71 ../../src/redis-server *:30006 [cluster]
300067846 28218 0.4 0.0 4381980 2676 ?? Ss 7:41PM 0:00.73 ../../src/redis-server *:30002 [cluster]
300067846 28216 0.4 0.0 4374812 2576 ?? Ss 7:41PM 0:00.50 ../../src/redis-server *:30001 [cluster]
300067846 28225 0.4 0.0 4380956 2632 ?? Ss 7:41PM 0:00.68 ../../src/redis-server *:30005 [cluster]
300067846 28220 0.3 0.0 4379932 2596 ?? Ss 7:41PM 0:00.52 ../../src/redis-server *:30003 [cluster]
300067846 85550 0.0 0.0 4309420 1128 ?? S 3:08PM 0:07.76 redis-server *:6379
see all master slaves nodes
../redis-5.0.5/utils/create-cluster$ cat nodes-3000*.conf
You can update the Redis cluster config. open vi create-cluster
and update the following config as you needed.
// default config are these
CLUSTER_HOST=127.0.0.1
PORT=30000
TIMEOUT=2000
NODES=6
REPLICAS=1