How to avoid unnecessary Firestore reads with Cache

Question

I have a list of trips data (Large Sets) ordered by date.

Existing Behaviour

I store all the trips data into SqlLite Db. For every new data added, I used to get a fcm notification and I synced only the new added data using last updated time concept. This way when ever an cx opens the app, he will always read the data from my DB, Hence saving read and network operations.

How could i achieve the same with Firestore?

Few problems to consider:

• Firestore get() always tries to get the data first from SERVER, even if i add CACHE to my query, how can i synced only the updated document
• In my Recyler View I want the data to be order by a field date not by lut(last-modified-time)
• With FireStore Snapshot Listener i can't query such a large record set.

Answer 1

Firestore get() always tries to get the data first from SERVER, even if I add CACHE to my query, how can I sync only the updated document?

According to the official documentation regarding enable offline data:

For Android and iOS, offline persistence is enabled by default.

So to use offline persistence, you don't need to make ("add") any changes to your code in order to be able to use and access Cloud Firestore data.

Also according to the official documentation regarding Query's get() method:

By default, get() attempts to provide up-to-date data when possible by waiting for data from the server, but it may return cached data or fail if you are offline and the server cannot be reached. This behavior can be altered via the Source parameter.

So this is normal behavior. Starting with 2018-06-13 (16.0.0 SDK) is now possible to specify the source from which we want to get data. We can achieve this with the help of the DocumentReference.get(Source source) and Query.get(Source source) methods.

As you can see, we can now pass as an argument to the DocumentReference or to the Query the source so we can force the retrieval of data from the server only, cache only or attempt server and fall back to the cache.

So something like this is now possible:

yourDocRef.get(Source.SERVER).addOnSuccessListener(new OnSuccessListener<DocumentSnapshot>() {
    @Override
    public void onSuccess(DocumentSnapshot documentSnapshot) {
        //Get data from the documentSnapshot object
    }
});

In this case, we force the data to be retrieved from the server only. If you want to force the data to be retrieved from the cache only, you should pass as an argument to the get() method, Source.SERVER. More informations here. I also wrote an article that can help understand more clear the concept:

• How to drastically reduce the number of reads when no documents are changed in Firestore?

If you want to get only updated documents, you can view changes between snapshots. An example from the official documentation would be:

db.collection("cities")
    .whereEqualTo("state", "CA")
    .addSnapshotListener(new EventListener<QuerySnapshot>() {
        @Override
        public void onEvent(@Nullable QuerySnapshot snapshots,
                @Nullable FirebaseFirestoreException e) {
        if (e != null) {
            Log.w(TAG, "listen:error", e);
            return;
        }

        for (DocumentChange dc : snapshots.getDocumentChanges()) {
            switch (dc.getType()) {
            case ADDED:
                Log.d(TAG, "New city: " + dc.getDocument().getData());
                break;
            case MODIFIED:
                Log.d(TAG, "Modified city: " + dc.getDocument().getData());
                break;
            case REMOVED:
                Log.d(TAG, "Removed city: " + dc.getDocument().getData());
                break;
            }
        }

        }
    });

See the switch statement for every particular case? The second case will help you get only the updated data.

In my Recyler View I want the data to be order by a field date not by lut(last-modified-time)

In this case you should create a query that allow you order the results by a specific date property. Assuming you have a database structure that looks like this:

Firestore-root
   |
   --- items (collection)
        |
        --- itemId (document)
             |
             --- date: Oct 08, 2018 at 6:16:58 PM UTC+3
             |
             --- //other properties

The query like will help you achieve this:

FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionRefference itemsRef = rootRef.collection("items");
Query query = itemsRef.orderBy("date", Query.Direction.ASCENDING);

This is also a recommended way on which you can add a timestamp to a Cloud Firestore database.

With FireStore Snapshot Listener i can't query such a large record set.

Oh, yes you can. According this answer, Firebase can effectively loop through billions items. This answer is for Firebase realtime database but same principles apply to Cloud Firestore.