381

Is it legitimate to delete items from a dictionary in Python while iterating over it?

For example:

for k, v in mydict.iteritems():
   if k == val:
     del mydict[k]

The idea is to remove elements that don't meet a certain condition from the dictionary, instead of creating a new dictionary that's a subset of the one being iterated over.

Is this a good solution? Are there more elegant/efficient ways?

Trilarion
  • 9,942
  • 9
  • 61
  • 98
  • 1
    A related question with very interesting answers: http://stackoverflow.com/questions/9023078/custom-dict-that-allows-delete-during-iteration. – max Aug 17 '12 at 08:19
  • 52
    @Trilarion One could have tried easily... **and easily learned nothing of value.** If it succeeds, it's _not_ necessarily legitimate. Edge cases and unexpected caveats abound. This question is of non-trivial interest to all would-be Pythonistas. Hand-waving dismissal on the order of "One could have tried easily!" is unhelpful and contrary to the inquisitive spirit of stackoverflow enquiry. – Cecil Curry Feb 22 '16 at 05:30
  • After perusing [max](https://stackoverflow.com/users/336527/max)'s [related question](https://stackoverflow.com/questions/9023078/custom-dict-that-allows-delete-during-iteration), **I must concur.** You probably just want to peruse that disturbingly in-depth question and its well-written answers instead. Your Pythonic mind will be blown. – Cecil Curry Feb 22 '16 at 05:54
  • 1
    @CecilCurry Testing an idea for yourself before presenting it here is kind of in the spirit of stackoverflow if I'm not mistaken. That was all I wanted to convey. Sorry if there has been any disturbance because of that. Also I think it is a good question and did not downvote it. I like the answer of [Jochen Ritzel](http://stackoverflow.com/a/5385196/1536976) most. I don't think one needs to do all that stuff to delete on the fly when deleting in a second step is much more simple. That should be the preferred way in my view. – Trilarion Feb 22 '16 at 08:20
  • Dumb observation: Your whole loop is kind of pointless if you're just looking for a specific key. You could replace it with `try: del mydict[val]` `except KeyError: pass` or as a one-liner, with `mydict.pop(val, None)`, both of which would be `O(1)` operations, not `O(n)`. The question is still valid if the condition for deletion is more than just "equal to some value" though. – ShadowRanger Mar 02 '17 at 21:00
  • Many of the comments & answers here presume that this is impossible by definition - but it's not, it's just that Python doesn't provide an implementation. For example in Java, it's possible to delete the most-recently iterated value of a `HashMap` by using the iterator's `remove()` method. This is highly desirable, because as all the examples here show, working around it is annoying, error-prone, and a waste of memory and time. – Ken Williams May 08 '20 at 16:44

10 Answers10

382

EDIT:

For Python3 (or greater):

>>> mydict
{'four': 4, 'three': 3, 'one': 1}

>>> for k in list(mydict.keys()):
...     if mydict[k] == 3:
...         del mydict[k]
...
>>> mydict
{'four': 4, 'one': 1}

The rest of the answers works fine with Python2 but do not work for Python3 and raises RuntimeError.

RuntimeError: dictionary changed size during iteration.

This happens because mydict.keys() returns an iterator not a list. As pointed out in comments simply convert mydict.keys() to a list by list(mydict.keys()) and it should work.


For python2:

A simple test in the console shows you cannot modify a dictionary while iterating over it:

>>> mydict = {'one': 1, 'two': 2, 'three': 3, 'four': 4}
>>> for k, v in mydict.iteritems():
...    if k == 'two':
...        del mydict[k]
...
------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython console>", line 1, in <module>
RuntimeError: dictionary changed size during iteration

As stated in delnan's answer, deleting entries causes problems when the iterator tries to move onto the next entry. Instead, use the keys() method to get a list of the keys and work with that:

>>> for k in mydict.keys():
...    if k == 'two':
...        del mydict[k]
...
>>> mydict
{'four': 4, 'three': 3, 'one': 1}

If you need to delete based on the items value, use the items() method instead:

>>> for k, v in mydict.items():
...     if v == 3:
...         del mydict[k]
...
>>> mydict
{'four': 4, 'one': 1}
rakeb.mazharul
  • 5,513
  • 3
  • 19
  • 40
Blair
  • 14,573
  • 7
  • 44
  • 56
  • 57
    Note that in Python 3, dict.items() returns an iterator (and dict.iteritems() is gone). – Tim Lesher Sep 27 '11 at 19:01
  • 95
    To elaborate on @TimLesher comment... This will NOT work in Python 3. – max Jan 26 '12 at 16:55
  • 114
    To elaborate on @max's elaboration, it will work if you convert the above code with 2to3. One of the default fixers will make the loop look like `for k, v in list(mydict.items()):` which works fine in Python 3. Same for `keys()` becoming `list(keys())`. – Walter Mundt Aug 15 '12 at 17:59
  • 8
    This doesn't work. I get an error: `RuntimeError: dictionary changed size during iteration` – Tomáš Zato - Reinstate Monica Dec 18 '16 at 02:56
  • 19
    @TomášZato as Walter pointed out, for python3 you need to use `for k in list(mydict.keys()):` as python3 makes the keys() method an iterator, and also disallows deleting dict items during iteration. By adding a list() call you turn the keys() iterator into a list. So when you are in the body of the for loop you are no longer iterating over the dictionary itself. – Geoff Crompton Mar 22 '17 at 23:12
  • Will `list` make a copy of the keys, or only point at each of them? – matanster Sep 12 '18 at 06:11
  • @matanster `list` basically converts an `iterator` into a list of `keys`. – CodeIt Oct 05 '19 at 14:28
  • 2
    I would opt for `for key, value in my_dict.copy().items():` so you are iterating through a duplicate dictionary and deleting from the original. It maintains the cleanliness of the data typing – xle Jan 27 '21 at 15:18
  • https://stackoverflow.com/questions/6777485/modifying-a-python-dict-while-iterating-over-it If the runtime error is removed there will a chance that you won't be able to iterate over all items – Ebrahim Karam Feb 27 '21 at 22:31
  • @xle Thanks; this solution worked for me, and is more semantically intuitive to grasp than transforming into a list, to my mind anyway. – cjstevens Jun 06 '21 at 13:05
110

You could also do it in two steps:

remove = [k for k in mydict if k == val]
for k in remove: del mydict[k]

My favorite approach is usually to just make a new dict:

# Python 2.7 and 3.x
mydict = { k:v for k,v in mydict.items() if k!=val }
# before Python 2.7
mydict = dict((k,v) for k,v in mydict.iteritems() if k!=val)
Nils Lindemann
  • 748
  • 1
  • 13
  • 22
Jochen Ritzel
  • 99,912
  • 29
  • 194
  • 188
  • 11
    @senderle: Since 2.7 actually. – Jochen Ritzel Mar 22 '11 at 00:17
  • 7
    The dict comprehension approach makes a copy of the dictionary; luckily the values at least don't get deep-copied, just linked. Still if you have a lot of keys, it could be bad. For that reason, I like the `remove` loop approach more. – max Jan 26 '12 at 17:28
  • 2
    You can also combine the steps: `for k in [k for k in mydict if k == val]: del mydict[k]` – AXO Jan 16 '17 at 11:14
  • the first solution is the only efficient one on big dicts in this thread so far - as it doesn't make a full length copy. – kxr Apr 28 '17 at 21:05
29

Iterate over a copy instead, such as the one returned by items():

for k, v in list(mydict.items()):
0 _
  • 9,294
  • 10
  • 71
  • 105
Ignacio Vazquez-Abrams
  • 740,318
  • 145
  • 1,296
  • 1,325
  • 1
    That doesn't make much sense -- then you can't `del v` directly, so you've made a copy of each v which you're never going to use and you have to access the items by key anyways. `dict.keys()` is a better choice. – jscs Mar 22 '11 at 02:21
  • 2
    @Josh: It all depends on how much you're going to need to use `v` as a criterion for deletion. – Ignacio Vazquez-Abrams Mar 22 '11 at 07:00
  • 3
    Under Python 3, `dict.items()` returns an iterator rather than a copy. See commentary for [Blair](https://stackoverflow.com/users/668807/blair)'s [answer](https://stackoverflow.com/a/5385075/2809027), which (sadly) also assumes Python 2 semantics. – Cecil Curry Feb 22 '16 at 05:37
23

You can't modify a collection while iterating it. That way lies madness - most notably, if you were allowed to delete and deleted the current item, the iterator would have to move on (+1) and the next call to next would take you beyond that (+2), so you'd end up skipping one element (the one right behind the one you deleted). You have two options:

  • Copy all keys (or values, or both, depending on what you need), then iterate over those. You can use .keys() et al for this (in Python 3, pass the resulting iterator to list). Could be highly wasteful space-wise though.
  • Iterate over mydict as usual, saving the keys to delete in a seperate collection to_delete. When you're done iterating mydict, delete all items in to_delete from mydict. Saves some (depending on how many keys are deleted and how many stay) space over the first approach, but also requires a few more lines.
  • `You can't modify a collection while iterating it.` this is just correct for dicts and friends, but you can modify lists during iteration: `L = [1,2,None,4,5] for n,x in enumerate(L): if x is None: del L[n]` – Nils Lindemann Feb 29 '16 at 16:42
  • 3
    @Nils It doesn't throw an exception but it's still incorrect. Observe: http://codepad.org/Yz7rjDVT -- see e.g. http://stackoverflow.com/q/6260089/395760 for an explanation –  Feb 29 '16 at 17:19
  • Got me here. Still `can't` is correct only for dict and friends, while it should be `shouldn't` for lists. – Nils Lindemann Feb 29 '16 at 19:20
12

With python3, iterate on dic.keys() will raise the dictionary size error. You can use this alternative way:

Tested with python3, it works fine and the Error "dictionary changed size during iteration" is not raised:

my_dic = { 1:10, 2:20, 3:30 }
# Is important here to cast because ".keys()" method returns a dict_keys object.
key_list = list( my_dic.keys() )

# Iterate on the list:
for k in key_list:
    print(key_list)
    print(my_dic)
    del( my_dic[k] )


print( my_dic )
# {}
glihm
  • 887
  • 10
  • 26
12

It's cleanest to use list(mydict):

>>> mydict = {'one': 1, 'two': 2, 'three': 3, 'four': 4}
>>> for k in list(mydict):
...     if k == 'three':
...         del mydict[k]
... 
>>> mydict
{'four': 4, 'two': 2, 'one': 1}

This corresponds to a parallel structure for lists:

>>> mylist = ['one', 'two', 'three', 'four']
>>> for k in list(mylist):                            # or mylist[:]
...     if k == 'three':
...         mylist.remove(k)
... 
>>> mylist
['one', 'two', 'four']

Both work in python2 and python3.

rsanden
  • 1,180
  • 14
  • 20
  • This isn't good in case your dataset is large. This is copying all the objects in memory, right? – AFP_555 Feb 09 '20 at 03:46
  • 1
    @AFP_555 Yes - my goal here is for clean, parallel, pythonic code. If you need memory efficiency, the best approach I know of is to iterate and build either a list of keys to delete or a new dict of items to save. Beauty is my priority with Python; for large datasets I am using Go or Rust. – rsanden Feb 10 '20 at 04:27
10

You can use a dictionary comprehension.

d = {k:d[k] for k in d if d[k] != val}

Aaron
  • 101
  • 1
  • 3
4

You could first build a list of keys to delete, and then iterate over that list deleting them.

dict = {'one' : 1, 'two' : 2, 'three' : 3, 'four' : 4}
delete = []
for k,v in dict.items():
    if v%2 == 1:
        delete.append(k)
for i in delete:
    del dict[i]
Pob
  • 41
  • 2
  • Its rather a duplicate of @Ritzel's 1st solution (efficient on big dicts w/o full copy). Though a "long read" w/o list comprehension. Yet is it possibly faster nevertheless ? – kxr Apr 28 '17 at 21:08
3

There is a way that may be suitable if the items you want to delete are always at the "beginning" of the dict iteration

while mydict:
    key, value = next(iter(mydict.items()))
    if should_delete(key, value):
       del mydict[key]
    else:
       break

The "beginning" is only guaranteed to be consistent for certain Python versions/implementations. For example from What’s New In Python 3.7

the insertion-order preservation nature of dict objects has been declared to be an official part of the Python language spec.

This way avoids a copy of the dict that a lot of the other answers suggest, at least in Python 3.

Michal Charemza
  • 24,475
  • 11
  • 89
  • 143
2

I tried the above solutions in Python3 but this one seems to be the only one working for me when storing objects in a dict. Basically you make a copy of your dict() and iterate over that while deleting the entries in your original dictionary.

        tmpDict = realDict.copy()
        for key, value in tmpDict.items():
            if value:
                del(realDict[key])
Jason Landbridge
  • 589
  • 6
  • 14
  • Can also use `for key, value in my_dict.copy().items():` to be more succinct and avoid adding more variables to your scope than necessary – xle Jan 27 '21 at 15:21