What is the best way to do ajax pagination with MongoDb and Nodejs?

Question

I have mongodb and NodeJs. The connection is done via mongoosejs.
What is the best way to develop ajax infinity scroll ? Should I use limit and offset ?

Answer 1

"skip and limit" approach is not very efficient when you are paging far into dataset. It is effectively a Shlemiel the Painter's algorithm.

Range queries are much more efficient (when supported by indexes). For example, let's imagine that you're displaying tweets. Your page size is 20 and you're on page 1000 and want to load page 1001.

This query

db.tweets.find().sort({created_at: -1}).skip(1001*20).limit(20)

is much less efficient than

db.tweets.find({created_at: {$lt: last_displayed_date}}).
          sort({created_at: -1}).limit(20);

(provided that you have index on created_at).

You get the idea: when you load a page, note the timestamp of the last tweet and use it to query the next page.