HDFS vs GridFS: When to use which?

mongodb hadoop hdfs gridfs
iCode picture iCode · Jan 31, 2012 · Viewed 7.9k times · Source

HDFS and GridFS are two great technologies for distributed file saving but what are their differences? What type of problems fit better to each?

Answer

om-nom-nom picture om-nom-nom · May 22, 2012

HDFS intended for batch processing (you're know, when you running a query that will read many of your files one-by-one), but really suck when you doing random access operations and it is pain in the neck to maintain it or even deploy (you're know, all of these Zookepers, Namenodes and so on). On the other hand GridFS is slower at batches, but not in the case when you do a lot of random accesses, but have a bigger storage overhead compared to HDFS.

I would say that you should use HDFS for analitycs and GridFS for backing web-site.

Related questions

How to choose between Cassandra, Membase, Hadoop, MongoDB, RDBMS etc.?

Is there a paper/blog-post on when to use Cassandra or Membase or Hadoop or plain old relational databases ? Is there a paper discussing the strengths/weaknesses of each, and on what scenarios either of these technologies should be chosen ? …

Read More
Getting Spark, Python, and MongoDB to work together

I'm having difficulty getting these components to knit together properly. I have Spark installed and working succesfully, I can run jobs locally, standalone, and also via YARN. I have followed the steps advised (to the best of my knowledge) here …

Read More
How to import from sql dump to MongoDB?

I am trying to import data from MySQL dump .sql file to get imported into MongoDB. But I could not see any mechanism for RDBMS to NoSQL data migration. I have tried to convert the data into JSON and CSV …

Read More