Does python garbage-collect at the end of an iteration in a loop?

Question

Please observe this simple code:

    import random
    while True:
        L = list( str(random.random()))

Question: if I let this run, will python run out of memory?
reason I am asking:
First iteration of this loop, a list is created, and 'L' is assigned to represent that list. The next iteration of this loop, another list is created, 'L' is yanked from the previous list and and assigned to the new list. The previous list has lost it reference. Is the previous list going to be garbage collected? if not at the end of each iteration, but eventually I hope?

Having said that, just expand the scenario a bit further into multiprocessing:

    import random
    while True:
        l1 = list( str(random.random()))            
        pseudo: multiprocessing.Queue.put(l1)
        # how is l1 handled here?
        # is l1 .copy()-ed to the queue or referenced by the queue?
        # is l1 destoryed in this process (this while loop) at the end of iteration?

kindall · Accepted Answer · 2016-04-20T00:40:34.107

The primary means of garbage collection is reference counting in CPython (the reference implementation of the language). When there are no longer any references to an object, the memory it occupies is freed immediately and can be reused by other Python objects. (It may or may not ever be released back to the operating system.) There are a few exceptions of objects that are never freed: smallish integers, interned strings (including literals), the empty tuple, None.

So to answer your initial question, L is going to be reassigned to a new list on each iteration. At that point, the previous list has no references and its memory will be released immediately.

With regard to your second example, putting something into a multiprocessing queue is, of necessity, a copy operation. The object must be serialized ("pickled" in Python parlance) to be sent to the new process, which has its own memory space and can't see anything from the original process's memory. When, in your loop, you reassign li to the next list, the previous list has no references and, again, will be released.

At the end of your loop, the L or l1 variable still refers to a list: the one you created in the last iteration of the loop. If you want to release this object, just del L or del l1 respectively.

PS -- When objects contain references to themselves (either directly, or indirectly through a chain of other objects), this is referred to a cyclic reference. These aren't collected automatically by reference counting and Python has a separate garbage collector which runs periodically to clean them up.

It may come to mind in more complex cases, whether there is an idiomatic way for running a loop without variables defined in the loop persisting beyond the scope of each iteration. In particular in scenarios where one has several large objects being created in each iteration. — matanster, May 07 '19 at 13:56
An idiomatic way to do that is to put it in a function. Then all those variables are local and are automatically freed when the function returns. — kindall, Apr 15 '22 at 18:19

score 10 · Answer 2 · answered Sep 13 '19 at 01:13

We can easily test this by adding a custom __del__ command to a class as watch what happens:

class WithDestructor(object):
   def __del__(self):
       print(f"Exploding {self}")

Q=None
for i in range(5):
    Q = WithDestructor()
    print(f"In loop {i}")

If cleanup only happened at the end of the loop, we'd get the loop output followed by the destructor output. Instead I get it interlaced, so the object in Q is getting immediately cleaned up when Q is reassigned.

In loop 0
Exploding <__main__.WithDestructor object at 0x7f93141176d8>
In loop 1
Exploding <__main__.WithDestructor object at 0x7f93141172b0>
In loop 2
Exploding <__main__.WithDestructor object at 0x7f93141176d8>
In loop 3
Exploding <__main__.WithDestructor object at 0x7f93141172b0>
In loop 4

Does python garbage-collect at the end of an iteration in a loop?

2 Answers2

Linked