Memory-usage of dictionary in Python?

Question

I am slightly confused when I use the getsizeof method in the sys module for dictionaries. Below I have created a simple dictionary of two strings. The two strings' sizes are clearly larger than the one of the dictionary. The dictionary size is probably the dictionary overhead only, i.e., it doesn't take the actual data into account. What is the best way to figure out the memory-usage of the whole dictionary (keys, values, dictionary overhead)?

>>> first = 'abc'*1000
>>> second = 'def'*1000
>>> my_dictionary = {'first': first, 'second': second}
>>> getsizeof(first)
3021
>>> getsizeof(second)
3021
>>> getsizeof(my_dictionary)
140

score 13 · Accepted Answer · answered Jul 05 '11 at 08:30

13

From the PythonDocs

See recursive sizeof recipe for an example of using getsizeof() recursively to find the size of containers and all their contents.

So it only counts the overhead, but you can use the function in this link to calculate it for containers like dicts.

answered Jul 05 '11 at 08:30

Jacob

39,723
6
76
81

it is not working for simple dict like `d = dict(a="4", b="4", c="4", d="4")` it is skipping values corresponding to `b,c,d` – rishi007bansod Oct 12 '18 at 11:40
sorry i guess your code is correct, in above case python is reusing object `"4"` – rishi007bansod Oct 12 '18 at 11:58

score 5 · Answer 2 · answered Jun 07 '16 at 19:35

The recursive getsizeof would get the actual size, but if you have multiple layers of dictionaries and only want to get a rough estimate. The json comes handy.

>>> first = 'abc'*1000
>>> second = 'def'*1000
>>> my_dictionary = {'first': first, 'second': second}
>>> getsizeof(first)
3049
>>> getsizeof(second)
3049
>>> getsizeof(my_dictionary)
288
>>> getsizeof(json.dumps(my_dictionary))
6076
>>> size = getsizeof(my_dictionary)
>>> size += sum(map(getsizeof, my_dictionary.values())) + sum(map(getsizeof, my_dictionary.keys()))
>>> size
6495

Definitely points for creativity, but it needs everything to be serializable, it is slower, and as you say it's an approximation... — Mark, Apr 30 '17 at 18:33

score 4 · Answer 3 · answered Jul 05 '11 at 08:28

4

Well, dictionaries don't store the actual string inside them, it works a bit like C/C++ pointers, so you only get a constant overhead in the dictionary for every element.

The total size is

size = getsizeof(d)
size += sum(map(getsizeof, d.itervalues())) + sum(map(getsizeof, d.iterkeys()))

answered Jul 05 '11 at 08:28

orlp

106,415
33
201
300

6

To be pedantic, if any of the values is a container (rather than a scalar) it needs to drill down that container as well. – Maxim Egorushkin Jul 05 '11 at 08:34

Memory-usage of dictionary in Python?

3 Answers3

Linked