How many records can you insert in a redis instance?

optimization amazon-web-services redis benchmarking amazon-elasticache
user244333 picture user244333 · Mar 29, 2014 · Viewed 7.9k times · Source

I have a list of 100 million strings one on each line. The file size is 1.66 Gb. Each string is about 20 chars long.

I started inserting the data into small instance and got max memory error. There were only 1million records inserted by then.

Micro
Micro Cache Node (cache.t1.micro): 213 MB memory, Up to 2 ECU (for short periodic bursts), 64-bit platform, Low I/O Capacity
Standard
Small Cache Node (cache.m1.small): 1.3 GB memory, 1 ECU (1 virtual core with 1 ECU), 64-bit platform, Moderate I/O Capacity
Medium Cache Node (cache.m1.medium): 3.35 GB memory, 2 ECU (1 virtual core with 2 ECUs), 64-bit platform, Moderate I/O Capacity
Large Cache Node (cache.m1.large): 7.1 GB memory, 4 ECUs (2 virtual cores with 2 ECUs each), 64-bit platform, High I/O Capacity
Extra Large Cache Node (cache.m1.xlarge): 14.6 GB of memory, 8 ECUs (4 virtual cores with 2 ECUs each), 64-bit platform, High I/O Capacity

Will a small cache node be able to store the data or will I run out of space? How to calculate the number of records that an instance can handle?

Answer

datasage picture datasage · Mar 30, 2014

According to this: http://redis.io/topics/faq

When you have a lot of small keys, it will use 5-6 times as much memory as the size of the data stored in redis.

You will probably need somewhere around 8-10 GB of memory to store your data set. Which will limit you to cache.m1.xlarge.

Related questions

Which "href" value should I use for JavaScript links, "#" or "javascript:void(0)"?

The following are two methods of building a link that has the sole purpose of running JavaScript code. Which is better, in terms of functionality, page load speed, validation purposes, etc.? or function myJsFunc() { alert("myJsFunc"); } <a href="javascript:…

Read More
Why is processing a sorted array faster than processing an unsorted array?

Here is a piece of C++ code that shows some very peculiar behavior. For some strange reason, sorting the data miraculously makes the code almost six times faster: #include <algorithm> #include <ctime> #include <iostream> …

Read More
How do I position one image on top of another in HTML?

I'm a beginner at rails programming, attempting to show many images on a page. Some images are to lay on top of others. To make it simple, say I want a blue square, with a red square in the upper …

Read More