How to stream MongoDB Query Results with nodejs?

mongodb node.js streaming
Jan Algermissen picture Jan Algermissen · Sep 10, 2011 · Viewed 40.7k times · Source

I have been searching for an example of how I can stream the result of a MongoDB query to a nodejs client. All solutions I have found so far seem to read the query result at once and then send the result back to the server.

Instead, I would (obviously) like to supply a callback to the query method and have MongoDB call that when the next chunk of the result set is available.

I have been looking at mongoose - should I probably use a different driver?

Jan

Answer

Dan Milon picture Dan Milon · Jun 11, 2012

node-mongodb-driver (the underlying layer that every mongoDB client uses in nodejs) except the cursor API that others mentioned has a nice stream API (#458). Unfortunately i did not find it documented elsewhere.

Update: there are docs also here.

It can be used like this:

var stream = collection.find().stream()
stream.on('error', function (err) {
  console.error(err)
})
stream.on('data', function (doc) {
  console.log(doc)
})

It actually implements the ReadableStream interface, so it has all the goodies (pause/resume etc)

Related questions

How do I update/upsert a document in Mongoose?

Perhaps it's the time, perhaps it's me drowning in sparse documentation and not being able to wrap my head around the concept of updating in Mongoose :) Here's the deal: I have a contact schema and model (shortened properties): var mongoose = …

Read More
Render basic HTML view?

I have a basic node.js app that I am trying to get off the ground using Express framework. I have a views folder where I have an index.html file. But I receive the following error when loading the …

Read More
How do I remove documents using Node.js Mongoose?

FBFriendModel.find({ id: 333 }, function (err, docs) { docs.remove(); //Remove all the documents that match! }); The above doesn't seem to work. The records are still there. Can someone fix?

Read More