Is double-checked locking thread-safe in Python?

Question

The double-checked locking idiom is not reliable in some languages, and I want to know whether Python is one of them. More concretely, is the following code...

# Objects shared by threads:
obj = None
lock_for_obj = threading.Lock()

def get_obj():
    """Function called concurrently by threads."""
    if obj is None:
        with lock_for_obj:
            if obj is None:
                obj = factory()  # Never returns `None`
    return obj

...thread-safe in Python? Are there scenarios/implementations where it is/isn't? Why?

You're assuming it's safe to perform lookups on the cache while another thread is modifying it. Even with the GIL (and not all Python implementations have a GIL), that's not completely safe. — user2357112, May 02 '19 at 18:04
@user2357112: What cache? The processor cache? I’m willing to believe that a GIL-less implementation might have different safety rules, but if the GIL didn’t provide cache coherence, wouldn’t *everything* be broken in CPython? — Davis Herring, May 04 '19 at 17:22
@DavisHerring ah, they were responding to a previous version of the question which I've edited away. — Brian Rodriguez, May 04 '19 at 18:02
@BrianRodriguez: Hm. The original version of your question had a very interesting answer for CPython, at least as long as the call to `factory` has no side effects (outside the object being created). — Davis Herring, May 04 '19 at 18:18
`obj is None` is a read of `obj` and `return obj` is another one, so there’s a possible code path doing two reads of `obj` without any synchronization actions. This could fail even with languages where double checked locking works; they usually require reading `obj` into a local variable to be used for the test and return, so that the code path without synchronization action bears only one read operation. — Holger, May 06 '19 at 15:18
That pattern is unsafe (any serious analysis shows why) unless additional mechanisms add some additional guarantees about read/write ordering at the low level. I haven't seen anything like that in official documentation. — VPfB, May 10 '19 at 06:13
@user2357112supportsMonica: With the GIL, that's completely safe; the GIL doesn't just protect reference count modifications, it also prevents the normal issues with simple reads; there are no torn or invalid values to be read. Not going to make any claims about non-GIL-ed interpreters though. — ShadowRanger, Sep 23 '20 at 22:09

score 2 · Accepted Answer · edited Sep 23 '20 at 22:06

Have a look at PEP 583 - Concurrency memory model for Python, which was withdrawn.

I guess the reason it was withdrawn (though I can't find enough information on that) is since there are different implementations of Python, and it's difficult to enforce a standard like this on every one of them.

Conclusion: this code might be safe when using CPython implementation on a single processor, or it might be safe to use it in general using Jython implementation, but there's no guarantee.

Is double-checked locking thread-safe in Python?

1 Answers1

Linked