Can Django's .annotate() return objects?

Daniel Quinn picture Daniel Quinn · Apr 19, 2011 · Viewed 8k times · Source

I have a record set that looks like this (extraneous data omitted to protect the easily bored):

 id | user_id |            created            | units
----+---------+-------------------------------+-------
  1 |       1 | 2011-04-18 15:43:02.737063+00 |    20
  2 |       1 | 2011-04-18 15:43:02.737063+00 |     4
  3 |       1 | 2011-04-18 15:46:48.592999+00 |    -1
  4 |       1 | 2011-04-19 12:02:10.687587+00 |    -1
  5 |       1 | 2011-04-19 12:09:20.039543+00 |    -1
  6 |       1 | 2011-04-19 12:11:21.948494+00 |    -1
  7 |       1 | 2011-04-19 12:15:51.544394+00 |    -1
  8 |       1 | 2011-04-19 12:16:44.623655+00 |    -1

And I'd like to get a result that looks like this:

 id | user_id |            created            | units
----+---------+-------------------------------+-------
  8 |       1 | 2011-04-19 12:16:44.623655+00 |    14
  2 |       1 | 2011-04-18 15:43:02.737063+00 |     4

So I naturally look to .annotate():

u = User.objects.get(pk=1)
(MyModel.objects.filter(user=u)
        .values("id","user","created")
        .annotate(stuff=Sum("units"))
)

The problem is that I want objects, not a single list of dictionaries. I need the methods attached to those objects to be available. Any ideas as to how to do this?

Edit: I should have pointed out that I tried using .values() because without it I would get a bunch of annotated objects alright, but there would be 8 of them (as in the first query above), and not 2 (as in the second result above). My guess is that it's not combining the rows because there's a timestamp in there making the rows different:

MyModel.objects.filter(user=u).annotate(Sum("units"))
# Returns 8 records, not 2 as expected

Answer

Daniel Quinn picture Daniel Quinn · Apr 20, 2011

It looks like there was some confusion when I initially asked the question, and as a result I was directed to not use .values() when that wasn't actually the problem. The problem is that Django's .annotate() doesn't allow for the overwriting of column properties with calculated values, and only annotates objects with additional data. This makes sense really, since if you want an object returned, you want to be able to assume that the object in question actually represents a row in the database, and not a mishmash of calculated values overwriting column values.

However, this didn't work for me, because the above functionality means that with columns like created (a timestamp) you can't get the calculated values I wanted without using .values()... and that doesn't give me objects.

So, I opted for the next best thing: .raw():

query = """
    SELECT
        ep.product_id              AS product_id,
        ep.user_id                 AS user_id,
        MAX(ep.created)            AS created,
        SUM(eup.units)             AS units
    FROM
        ecom_unitpermission AS eup
    WHERE
        ep.user_id = 1
    GROUP BY
        ep.product_id,
        ep.user_id
"""

for perm in MyModel.objects.raw(query):
    print "%15s %3s %s" % (perm.product.name, perm.units, perm.product.somemethod())

Using .defer() might have done this for me, though there appears to be a bug in Django 1.3 when combining that with .annotate()