17

As far as I know, the following code will be blocked if lock is already acquired by another thread.

It seems that non-blocking can be implemented by lock.acquire(0), but instead I have to use try-finally block instead with block. 

lock = threading.Lock()

def func():
 with lock:
  # do something...

Is there any method to implement non-blocking lock acquisition?

JaeJun LEE
  • 1,154
  • 2
  • 10
  • 26

5 Answers5

12
@contextmanager
def nonblocking(lock):
    locked = lock.acquire(False)
    try:
        yield locked
    finally:
        if locked:
            lock.release()

lock = threading.Lock()
with nonblocking(lock) as locked:
    if locked:
        do_stuff()
Николай Коновалов
  • 121
  • 1
  • 5
  • 1
    Welcome to Stack Overflow! While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value. Code-only answers are discouraged. – Ajean Oct 26 '16 at 18:58
10

Is there any method to implement non-blocking lock acquisition?

Yes. Just raise an exception if the lock can't be acquired immediately. Something like:

@contextlib.contextmanager
def non_blocking_lock(lock=threading.Lock()):
    if not lock.acquire(blocking=False):
        raise WouldBlockError
    try:
        yield lock
    finally:
        lock.release()

Usage:

with non_blocking_lock():
    # run with the lock acquired
jfs
  • 374,366
  • 172
  • 933
  • 1,594
  • 1
    Thanks for answering. Anyway I have to use `lock.acquire` function. – JaeJun LEE Jul 19 '15 at 17:17
  • I don't understand the point of the default argument. The lock would be created inside the context manager, and will be provided back to the user already locked - but of course no other thread knows about this lock. In what scenario could this be useful? – max May 10 '17 at 21:26
  • @max: It is not how default arguments work in Python. Understanding ["Least Astonishment" and the Mutable Default Argument](http://stackoverflow.com/q/1132941/4279) might help. – jfs May 10 '17 at 21:28
  • 1
    @J.F.Sebastian Ahh right! I know how mutable default arguments work, but I didn't realize that feature can be used for good rather than for evil :) Cool, so you get this one effectively global lock instance, "hidden" by the context manager definition, and you can implicitly rely on it everywhere by just using this context manager without arguments. That said, it seems a bit dangerous, since an accidental omission of the argument would cause a very subtle bug that will be super hard to detect. – max May 10 '17 at 21:51
  • 1
    @max if you want your own lock; you pass your own lock. Otherwise, you shouldn't pass any arguments to the context manager. Yes, it is very easy to introduce bugs in a multithreaded code. – jfs May 10 '17 at 22:00
  • One problem with this solution - the `finally` block would be called even if the lock was not acquired, so the lock would be immediately released by the thread which failed to acquire it, and not by the tread which owns it - usually not what you want. – bavaza May 31 '22 at 10:12
  • 1
    @bavaza: false. (`raise` is outside try/finally) – jfs Jun 01 '22 at 06:09
1

If you need a context manager that acquires a lock in a non-blocking manner, but still retries until the lock can finally be acquired, you could do like this:

@contextlib.contextmanager
def non_blocking_lock(lock : threading.Lock):
    # Waits as long as the lock can not be acquired, but releases the GIL in the meanwhile
    while not lock.acquire(blocking=False):
        pass

    try:
        yield   # Lock has been successfully acquired
    finally:
        lock.release()

It can be used exactly like the normal lock context manager:

class TestClass:
    def __init__(self):
         self._lock = threading.Lock()
    
    def method(self):
         with non_blocking_lock(self._lock):
         # do something that should be only done from one thread at once

... with the difference, that the lock is non-blocking and doesn't hold the GIL until the lock is released. I used it to fix some deadlocks.

The difference to the other solutions is, that the code eventually gets executed and the context manager does not simply return a false or throw an exception when the lock couldn't be acquired.

Correct me if you see any caveats with this solution.

Mayor Mayer
  • 101
  • 1
  • 11
0

The whole point of a lock is to ensure that certain sections of your program will only be executed by one thread or process at a time. This is achieved by blocking any threads/processes trying to acquire the lock while something else holds it.

If you don't want acquiring the lock to block, why are you using a lock in the first place? Presumably so that you can do something else while you wait?

To attempt to acquire the a lock l without blocking, call l.acquire(blocking=False). This will immediately return False if the lock was not acquired. If the lock was acquired it returns True, and the you will continue to hold the lock until you call its release() method.

This form, however, isn't particularly useful with the with statement. Usually you want the controlled code (the indented suite after the with) to run only when the lock has been acquired. not to query whether it has or not and take two alternative actions.

holdenweb
  • 27,899
  • 7
  • 50
  • 73
  • Thanks. I have to use `lock.acquire()`. :D. – JaeJun LEE Jul 19 '15 at 17:18
  • 2
    It is exactly the unusual cases that require a solution as requested in the question. The `with` construction allows to cleanly handle those cases where lock acquisition succeeds. – quazgar Feb 17 '21 at 09:01
-1

If you want to use the with statement with a non-blocking lock, you could also first check if it is locked. If it is, then you don't go into the with block. E.g:

lock = threading.Lock()

def func():
    if not lock.locked():
        with lock:
            # do something...
boudewijn21
  • 318
  • 1
  • 9
  • Nothing prevents this from blocking, as a context switch may occur between calling `lock.locked()` and 'with lock`. – bavaza May 30 '22 at 13:54