Practical Limits of ElasticSearch + Cassandra

cassandra elasticsearch limits
Henry picture Henry · Jun 15, 2011 · Viewed 10.9k times · Source

I am planning on using ElasticSearch to index my Cassandra database. I am wondering if anyone has seen the practical limits of ElasticSearch. Do things get slow in the petabyte range? Also, has anyone has any problems using ElasticSearch to index Cassandra?

Answer

DNA picture DNA · Jun 21, 2011

See this thread from 2011, which mentions ElasticSearch configurations with 1700 shards each of 200GB, which would be in the 1/3 petabyte range. I would expect that the architecture of ElasticSearch would support almost limitless horizontal scalability, because each shard index works separately from all other shards.

The practical limits (which would apply to any other solution as well) include the time needed to actually load that much data in the first place. Managing a Cassandra cluster (or any other distributed datastore) of that size will also involve significant workload just for maintenance, load balancing etc.

Related questions

Elasticsearch vs Cassandra vs Elasticsearch with Cassandra

I am learning NoSQL and looking at different options for one of my client's requirements. I have gone through various resources before putting up this question (a person with little knowledge in NoSQL) I need to store data at faster …

Read More
MongoDB vs. Cassandra

I am evaluating what might be the best migration option. Currently, I am on a sharded MySQL (horizontal partition), with most of my data stored in JSON blobs. I do not have any complex SQL queries (already migrated away after …

Read More
How to list all the available keyspaces in Cassandra?

I am newbie in Cassandra and trying to implement one toy application using Cassandra. I had created one keyspace and few column families in my Cassandra DB but I forgot the name of my cluster. I am trying to find …

Read More