Is Mongodb Aggregation framework faster than map/reduce?

performance mongodb mapreduce aggregation-framework
Taha Jahangir picture Taha Jahangir · Dec 17, 2012 · Viewed 18k times · Source

Is the aggregation framework introduced in mongodb 2.2, has any special performance improvements over map/reduce?

If yes, why and how and how much?

(Already I have done a test for myself, and the performance was nearly same)

Answer

Asya Kamsky picture Asya Kamsky · Dec 17, 2012

Every test I have personally run (including using your own data) shows aggregation framework being a multiple faster than map reduce, and usually being an order of magnitude faster.

Just taking 1/10th of the data you posted (but rather than clearing OS cache, warming the cache first - because I want to measure performance of the aggregation, and not how long it takes to page in the data) I got this:

MapReduce: 1,058ms
Aggregation Framework: 133ms

Removing the $match from aggregation framework and {query:} from mapReduce (because both would just use an index and that's not what we want to measure) and grouping the entire dataset by key2 I got:

MapReduce: 18,803ms
Aggregation Framework: 1,535ms

Those are very much in line with my previous experiments.

Related questions

MySQL vs MongoDB 1000 reads

I have been very excited about MongoDb and have been testing it lately. I had a table called posts in MySQL with about 20 million records indexed only on a field called 'id'. I wanted to compare speed with MongoDB and …

Read More
When to use CouchDB over MongoDB and vice versa

I am stuck between these two NoSQL databases. In my project I will be creating a database within a database. For example, I need a solution to create dynamic tables. So users can create tables with columns and rows. I …

Read More
Fastest way to remove duplicate documents in mongodb

I have approximately 1.7M documents in mongodb (in future 10m+). Some of them represent duplicate entry which I do not want. Structure of document is something like this: { _id: 14124412, nodes: [ 12345, 54321 ], name: "Some beauty" } Document is duplicate if it has at …

Read More