Proper Way to Retrieve More than 128 Documents with RavenDB

Question

I know variants of this question have been asked before (even by me), but I still don't understand a thing or two about this...

It was my understanding that one could retrieve more documents than the 128 default setting by doing this:

session.Advanced.MaxNumberOfRequestsPerSession = int.MaxValue;

And I've learned that a WHERE clause should be an ExpressionTree instead of a Func, so that it's treated as Queryable instead of Enumerable. So I thought this should work:

public static List<T> GetObjectList<T>(Expression<Func<T, bool>> whereClause)
{
    using (IDocumentSession session = GetRavenSession())
    {
        return session.Query<T>().Where(whereClause).ToList();                
    }
}

However, that only returns 128 documents. Why?

Note, here is the code that calls the above method:

RavenDataAccessComponent.GetObjectList<Ccm>(x => x.TimeStamp > lastReadTime);

If I add Take(n), then I can get as many documents as I like. For example, this returns 200 documents:

return session.Query<T>().Where(whereClause).Take(200).ToList();

Based on all of this, it would seem that the appropriate way to retrieve thousands of documents is to set MaxNumberOfRequestsPerSession and use Take() in the query. Is that right? If not, how should it be done?

For my app, I need to retrieve thousands of documents (that have very little data in them). We keep these documents in memory and used as the data source for charts.

** EDIT **

I tried using int.MaxValue in my Take():

return session.Query<T>().Where(whereClause).Take(int.MaxValue).ToList();

And that returns 1024. Argh. How do I get more than 1024?

** EDIT 2 - Sample document showing data **

{
  "Header_ID": 3525880,
  "Sub_ID": "120403261139",
  "TimeStamp": "2012-04-05T15:14:13.9870000",
  "Equipment_ID": "PBG11A-CCM",
  "AverageAbsorber1": "284.451",
  "AverageAbsorber2": "108.442",
  "AverageAbsorber3": "886.523",
  "AverageAbsorber4": "176.773"
}

Answer 1

It is worth noting that since version 2.5, RavenDB has an "unbounded results API" to allow streaming. The example from the docs shows how to use this:

var query = session.Query<User>("Users/ByActive").Where(x => x.Active);
using (var enumerator = session.Advanced.Stream(query))
{
    while (enumerator.MoveNext())
    {
        User activeUser = enumerator.Current.Document;
    }
}

There is support for standard RavenDB queries, Lucence queries and there is also async support.

The documentation can be found here. Ayende's introductory blog article can be found here.