Clustering, Sharding or simple Partition / Replication

mysql database-design partitioning cluster-computing sharding
albertosh picture albertosh · Jan 4, 2011 · Viewed 8.7k times · Source

I need some advice from you experts on this subject. The thing is that we have created a Facebook application some days ago and it got a lot of virality thankfully, the problem is that our database started getting REALLY FULL (some tables have more than 25 million rows now) and it got to the point that the app just stopped working because there was a queue of thousands and thousands of writes to be made.

I need to implement a solution for scaling this app QUICKLY but I'm not sure if I should pursue Sharding or Clustering since I'm not sure what are the pro's and con's of each of them and I was thinking on doing a Partition / Replication approach but I think that doesn't help if the load is on the writes?

Answer

Abdel Hegazi picture Abdel Hegazi · Jun 5, 2017

Clustering/Sharding/Partitioning comes when single node has reached to the point where its hardware cannot bear the load. But your hardware has still room to expand. This is the first lesson I learnt when I started being hit by such issues

Related questions

Table with 80 million records and adding an index takes more than 18 hours (or forever)! Now what?

A short recap of what happened. I am working with 71 million records (not much compared to billions of records processed by others). On a different thread, someone suggested that the current setup of my cluster is not suitable for my …

Read More
How to see table partition size in MySQL ( is it even possible? )

I've partitioned my table horizontally and I'd like to see how the rows are currently distributed. Searching the web didn't bring any relevant results. Could anyone tell me if this is possible?

Read More
Database - Designing an "Events" Table

After reading the tips from this great Nettuts+ article I've come up with a table schema that would separate highly volatile data from other tables subjected to heavy reads and at the same time lower the number of tables needed …

Read More