Fast way to copy dictionary in Python

python performance dictionary copy
Joern picture Joern · May 2, 2011 · Viewed 83.1k times · Source

I have a Python program that works with dictionaries a lot. I have to make copies of dictionaries thousands of times. I need a copy of both the keys and the associated contents. The copy will be edited and must not be linked to the original (e.g. changes in the copy must not affect the original.)

Keys are Strings, Values are Integers (0/1).

I currently use a simple way:

newDict = oldDict.copy()

Profiling my Code shows that the copy operation takes most of the time.

Are there faster alternatives to the dict.copy() method? What would be fastest?

Answer

Daniel DiPaolo picture Daniel DiPaolo · May 2, 2011

Looking at the C source for the Python dict operations, you can see that they do a pretty naive (but efficient) copy. It essentially boils down to a call to PyDict_Merge:

PyDict_Merge(PyObject *a, PyObject *b, int override)

This does the quick checks for things like if they're the same object and if they've got objects in them. After that it does a generous one-time resize/alloc to the target dict and then copies the elements one by one. I don't see you getting much faster than the built-in copy().

Related questions

[] and {} vs list() and dict(), which is better?

I understand that they are both essentially the same thing, but in terms of style, which is the better (more Pythonic) one to use to create an empty list or dict?

Read More
Dictionary vs Object - which is more efficient and why?

What is more efficient in Python in terms of memory usage and CPU consumption - Dictionary or Object? Background: I have to load huge amount of data into Python. I created an object that is just a field container. Creating 4…

Read More
Improving performance of very large dictionary in Python

I find that if I initialize an empty dictionary at the beginning, and then adding elements to the dictionary in a for loop (about 110,000 keys, the value for each key is a list, also increasing in the loop), the speed …

Read More