pomponchik / locklib Goto Github PK

It contains several useful additions to the standard thread synchronization tools, such as lock protocols and locks with advanced functionality.

• Installation
• Lock protocols
• SmartLock - deadlock is impossible with it

Get the locklib from the pypi:

pip install locklib

... or directly from git:

pip install git+https://github.com/pomponchik/locklib.git

You can also quickly try out this and other packages without having to install using instld.

Protocols are needed so that you can write typed code without being bound to specific classes. Protocols from this library allow you to "equalize" locks from the standard library and third-party locks, including those provided by this library.

We consider the basic characteristic of the lock protocol to be the presence of two methods for an object:

def acquire() -> None: pass
def release() -> None: pass

All the locks from the standard library correspond to this, as well as the locks presented in this one.

To check for compliance with this minimum standard, locklib contains the LockProtocol. You can check for yourself that all the locks match it:

from multiprocessing import Lock as MLock
from threading import Lock as TLock, RLock as TRLock
from asyncio import Lock as ALock

from locklib import SmartLock, LockProtocol

print(isinstance(MLock(), LockProtocol)) # True
print(isinstance(TLock(), LockProtocol)) # True
print(isinstance(TRLock(), LockProtocol)) # True
print(isinstance(ALock(), LockProtocol)) # True
print(isinstance(SmartLock(), LockProtocol)) # True

However! Most idiomatic python code using locks uses them as context managers. If your code is like that too, you can use one of the two inheritors of the regular LockProtocol: ContextLockProtocol or AsyncContextLockProtocol. Thus, the protocol inheritance hierarchy looks like this:

LockProtocol
 ├── ContextLockProtocol
 └── AsyncContextLockProtocol

ContextLockProtocol describes the objects described by LockProtocol, which are also context managers. AsyncContextLockProtocol, by analogy, describes objects that are instances of LockProtocol, as well as asynchronous context managers.

Almost all the locks from the standard library are instances of ContextLockProtocol, as well as SmartLock.

from multiprocessing import Lock as MLock
from threading import Lock as TLock, RLock as TRLock

from locklib import SmartLock, ContextLockProtocol

print(isinstance(MLock(), ContextLockProtocol)) # True
print(isinstance(TLock(), ContextLockProtocol)) # True
print(isinstance(TRLock(), ContextLockProtocol)) # True
print(isinstance(SmartLock(), ContextLockProtocol)) # True

However, the Lock from asyncio belongs to a separate category and AsyncContextLockProtocol is needed to describe it:

from asyncio import Lock
from locklib import AsyncContextLockProtocol

print(isinstance(Lock(), AsyncContextLockProtocol)) # True

If you use type hints and static verification tools like mypy, we highly recommend using the narrowest of the presented categories for lock protocols, which describe the requirements for your locales.

locklib contains a lock that cannot get into the deadlock - SmartLock, based on Wait-for Graph. You can use it as a usual Lock from the standard library. Let's check that it can protect us from the race condition in the same way:

from threading import Thread
from locklib import SmartLock


lock = SmartLock()
counter = 0

def function():
  global counter

  for _ in range(1000):
      with lock:
          counter += 1

thread_1 = Thread(target=function)
thread_2 = Thread(target=function)
thread_1.start()
thread_2.start()

assert counter == 2000

Yeah, in this case the lock helps us not to get a race condition, as the standard Lock does. But! Let's trigger a deadlock and look what happens:

from threading import Thread
from locklib import SmartLock


lock_1 = SmartLock()
lock_2 = SmartLock()

def function_1():
  while True:
    with lock_1:
      with lock_2:
        pass

def function_2():
  while True:
    with lock_2:
      with lock_1:
        pass

thread_1 = Thread(target=function_1)
thread_2 = Thread(target=function_2)
thread_1.start()
thread_2.start()

And... We have an exception like this:

...
locklib.errors.DeadLockError: A cycle between 1970256th and 1970257th threads has been detected.

Deadlocks are impossible for this lock!

If you want to catch the exception, import this from the locklib too:

from locklib import DeadLockError