python dict: get vs setdefault

Question

The following two expressions seem equivalent to me. Which one is preferable?

data = [('a', 1), ('b', 1), ('b', 2)]

d1 = {}
d2 = {}

for key, val in data:
    # variant 1)
    d1[key] = d1.get(key, []) + [val]
    # variant 2)
    d2.setdefault(key, []).append(val)

The results are the same but which version is better or rather more pythonic?

Personally I find version 2 harder to understand, as to me setdefault is very tricky to grasp. If I understand correctly, it looks for the value of "key" in the dictionary, if not available, enters "[]" into the dict, returns a reference to either the value or "[]" and appends "val" to that reference. While certainly smooth it is not intuitive in the least (at least to me).

To my mind, version 1 is easier to understand (if available, get the value for "key", if not, get "[]", then join with a list made up from [val] and place the result in "key"). But while more intuitive to understand, I fear this version is less performant, with all this list creating. Another disadvantage is that "d1" occurs twice in the expression which is rather error-prone. Probably there is a better implementation using get, but presently it eludes me.

My guess is that version 2, although more difficult to grasp for the inexperienced, is faster and therefore preferable. Opinions?

Answer 1

Your two examples do the same thing, but that doesn't mean get and setdefault do.

The difference between the two is basically manually setting d[key] to point to the list every time, versus setdefault automatically setting d[key] to the list only when it's unset.

Making the two methods as similar as possible, I ran

from timeit import timeit

print timeit("c = d.get(0, []); c.extend([1]); d[0] = c", "d = {1: []}", number = 1000000)
print timeit("c = d.get(1, []); c.extend([1]); d[0] = c", "d = {1: []}", number = 1000000)
print timeit("d.setdefault(0, []).extend([1])", "d = {1: []}", number = 1000000)
print timeit("d.setdefault(1, []).extend([1])", "d = {1: []}", number = 1000000)

and got

0.794723378711
0.811882272256
0.724429205999
0.722129751973

So setdefault is around 10% faster than get for this purpose.

The get method allows you to do less than you can with setdefault. You can use it to avoid getting a KeyError when the key doesn't exist (if that's something that's going to happen frequently) even if you don't want to set the key.

See Use cases for the 'setdefault' dict method and dict.get() method returns a pointer for some more info about the two methods.

The thread about setdefault concludes that most of the time, you want to use a defaultdict. The thread about get concludes that it is slow, and often you're better off (speed wise) doing a double lookup, using a defaultdict, or handling the error (depending on the size of the dictionary and your use case).