Is there a recursive version of the dict.get() built-in?

Question

I have a nested dictionary object and I want to be able to retrieve values of keys with an arbitrary depth. I'm able to do this by subclassing dict:

>>> class MyDict(dict):
...     def recursive_get(self, *args, **kwargs):
...         default = kwargs.get('default')
...         cursor = self
...         for a in args:
...             if cursor is default: break
...             cursor = cursor.get(a, default)
...         return cursor
... 
>>> d = MyDict(foo={'bar': 'baz'})
>>> d
{'foo': {'bar': 'baz'}}
>>> d.get('foo')
{'bar': 'baz'}
>>> d.recursive_get('foo')
{'bar': 'baz'}
>>> d.recursive_get('foo', 'bar')
'baz'
>>> d.recursive_get('bogus key', default='nonexistent key')
'nonexistent key'

However, I don't want to have to subclass dict to get this behavior. Is there some built-in method that has equivalent or similar behavior? If not, are there any standard or external modules that provide this behavior?

I'm using Python 2.7 at the moment, though I would be curious to hear about 3.x solutions as well.

Answer 1

A very common pattern to do this is to use an empty dict as your default:

d.get('foo', {}).get('bar')

If you have more than a couple of keys, you could use reduce (note that in Python 3 reduce must be imported: from functools import reduce) to apply the operation multiple times

reduce(lambda c, k: c.get(k, {}), ['foo', 'bar'], d)

Of course, you should consider wrapping this into a function (or a method):

def recursive_get(d, *keys):
    return reduce(lambda c, k: c.get(k, {}), keys, d)

Answer 2

@ThomasOrozco's solution is correct, but resorts to a lambda function, which is only necessary to avoid TypeError if an intermediary key does not exist. If this isn't a concern, you can use dict.get directly:

from functools import reduce

def get_from_dict(dataDict, mapList):
    """Iterate nested dictionary"""
    return reduce(dict.get, mapList, dataDict)

Here's a demo:

a = {'Alice': {'Car': {'Color': 'Blue'}}}  
path = ['Alice', 'Car', 'Color']
get_from_dict(a, path)  # 'Blue'

---

If you wish to be more explicit than using lambda while still avoiding TypeError, you can wrap in a try / except clause:

def get_from_dict(dataDict, mapList):
    """Iterate nested dictionary"""
    try:
        return reduce(dict.get, mapList, dataDict)
    except TypeError:
        return None  # or some other default value

---

Finally, if you wish to raise KeyError when a key does not exist at any level, use operator.getitem or dict.__getitem__:

from functools import reduce
from operator import getitem

def getitem_from_dict(dataDict, mapList):
    """Iterate nested dictionary"""
    return reduce(getitem, mapList, dataDict)
    # or reduce(dict.__getitem__, mapList, dataDict)

Note that [] is syntactic sugar for the __getitem__ method. So this relates precisely how you would ordinarily access a dictionary value. The operator module just provides a more readable means of accessing this method.

Answer 3

You can actually achieve this really neatly in Python 3, given its handling of default keyword arguments and tuple decomposition:

In [1]: def recursive_get(d, *args, default=None):
   ...:     if not args:
   ...:         return d
   ...:     key, *args = args
   ...:     return recursive_get(d.get(key, default), *args, default=default)
   ...: 

Similar code will also work in python 2, but you'd need to revert to using **kwargs, as you did in your example. You'd also need to use indexing to decompose *args.

In any case, there's no need for a loop if you're going to make the function recursive anyway.

You can see that the above code demonstrates the same functionality as your existing method:

In [2]: d = {'foo': {'bar': 'baz'}}

In [3]: recursive_get(d, 'foo')
Out[3]: {'bar': 'baz'}

In [4]: recursive_get(d, 'foo', 'bar')
Out[4]: 'baz'

In [5]: recursive_get(d, 'bogus key', default='nonexistent key')
Out[5]: 'nonexistent key'

Answer 4

You can use a defaultdict to give you an empty dict on missing keys:

from collections import defaultdict
mydict = defaultdict(dict)

This only goes one level deep - mydict[missingkey] is an empty dict, mydict[missingkey][missing key] is a KeyError. You can add as many levels as needed by wrapping it in more defaultdicts, eg defaultdict(defaultdict(dict)). You could also have the innermost one as another defaultdict with a sensible factory function for your use case, eg

mydict = defaultdict(defaultdict(lambda: 'big summer blowout'))

If you need it to go to arbitrary depth, you can do that like so:

def insanity():
    return defaultdict(insanity)

print(insanity()[0][0][0][0])

Answer 5

There is none that I am aware of. However, you don't need to subclass dict at all, you can just write a function that takes a dictionary, args and kwargs and does the same thing:

 def recursive_get(d, *args, **kwargs):
     default = kwargs.get('default')
     cursor = d
     for a in args:
         if cursor is default: break
         cursor = recursive_get(cursor, a, default)
     return cursor 

use it like this

recursive_get(d, 'foo', 'bar')

Answer 6

collections.default_dict will handle the providing of default values for nonexistent keys at least.

Answer 7

The Iterative Solution

def deep_get(d:dict, keys, default=None, create=True):
    if not keys:
        return default
    
    for key in keys[:-1]:
        if key in d:
            d = d[key]
        elif create:
            d[key] = {}
            d = d[key]
        else:
            return default
    
    key = keys[-1]
    
    if key in d:
        return d[key]
    elif create:
        d[key] = default
    
    return default


def deep_set(d:dict, keys, value, create=True):
    assert(keys)
    
    for key in keys[:-1]:
        if key in d:
            d = d[key]
        elif create:
            d[key] = {}
            d = d[key]
    
    d[keys[-1]] = value 
    return value

I am about to test it inside of a Django project with a line such as:

keys = ('options', 'style', 'body', 'name')

val = deep_set(d, keys, deep_get(s, keys, 'dotted'))