How to fetch more than 5000 entities from CRM

Question

I'm querying MS Dynamics CRM Online from my console app:

public EntityCollection GetEntities(string entityName)
{
    IOrganizationService proxy = ServerConnection.GetOrganizationProxy();

    string request = string.Format("<fetch mapping ='logical'><entity name = '{0}'></entity></fetch>", entityName);
    FetchExpression expression = new FetchExpression(request);
    var mult = proxy.RetrieveMultiple(expression);

    return mult;
}

This code only returns maximum of 5000 elements in mult.Entities. I know there are more entities in CRM. How to retrieve all entites?

Answer 1

You can only get back 5000 records at a time using fetch XML.

To get more records, you have to use a paging cookie, see here:

Sample: Use FetchXML with a paging cookie

The relevant bits of code:

// Define the fetch attributes.
// Set the number of records per page to retrieve.
int fetchCount = 3;
// Initialize the page number.
int pageNumber = 1;
// Specify the current paging cookie. For retrieving the first page, 
// pagingCookie should be null.
string pagingCookie = null;

The main loop modified, since the sample doesn't seem to do update the paging cookie:

while (true)
{
    // Build fetchXml string with the placeholders.
    string xml = CreateXml(fetchXml, pagingCookie, pageNumber, fetchCount);

     FetchExpression expression = new FetchExpression(xml);
     var results = proxy.RetrieveMultiple(expression);

    // * Build up results here *

    // Check for morerecords, if it returns 1.
    if (results.MoreRecords)
    {
        // Increment the page number to retrieve the next page.
        pageNumber++;
        pagingCookie = results.PagingCookie;
    }
    else
    {
        // If no more records in the result nodes, exit the loop.
         break;
    }
}

I personally tend to use LINQ rather than FetchXML, but it's worth noting what Lasse V. Karlsen said, if you are presenting this information to the user, you probably want to be doing some kind of paging (either in FetchXML or LINQ)