Efficient pagination and database querying in django

Question

There were some code examples for django pagination which I used a while back. I may be wrong but when looking over the code it looks like it wastes tons of memory. I was looking for a better solution, here is the code:

# in views.py
from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger

... 
...    

def someView():
    models = Model.objects.order_by('-timestamp')
    paginator = Paginator(models, 7)
    pageNumber = request.GET.get('page')

    try: 
        paginatedPage = paginator.page(pageNumber)
    except PageNotAnInteger: 
        pageNumber = 1
    except EmptyPage: 
        pageNumber = paginator.num_pages
    models = paginator.page(pageNumber)

    return render_to_resp ( ..... models ....)

I'm not sure of the subtlties of this code but from what it looks like, the first line of code retrieves every single model from the database and pushes it into. Then it is passed into Paginator which chunks it up based on which page the user is on from a html GET. Is paginator somehow making this acceptable, or is this totally memory inefficient? If it is inefficient, how can it be improved?

Also, a related topic. If someone does:

   Model.objects.all()[:40]

Does this code mean that all models are pushed into memory, and we splice out 40 of them? Which is bad. Or does it mean that we query and push only 40 objects into memory period?

Thank you for your help!

Answer 1

mymodel.objects.all() yields a queryset, not a list. Querysets are lazy - no request is issued and nothing done until you actually try to use them. Also slicing a query set does not load the whole damn thing in memory only to get a subset but adds limit and offset to the SQL query before hitting the database.