scoder / fastrlock Goto Github PK

View Code? Open in Web Editor NEW

23.0 23.0 10.0 84 KB

A fast RLock implementation for CPython

License: MIT License

Makefile 7.31% Python 57.42% C 0.38% Batchfile 8.46% PowerShell 10.11% Cython 16.32%

fastrlock's Introduction

Hi there, I'm Stefan 👋

🧑🏽‍💻 I am the maintainer of Cython, lxml and a few other Python data processing tools that you can find below.
🧑🏽‍💻 I am also a core developer of CPython and contributor to many other open source software projects.
🤝 In my city there's an old house from the 1600s with an inscription: "May God give to all those who know me, twice what they feel that they owe me" (*). With a donation, YOU can help me live the life that you think I deserve for the work that I do on Cython and on lxml, or my other projects.
🤔 You can contact me at [email protected]
😄 My pronouns are he/him
🗣️ I speak and 🖋️ write in Deutsch, English, Français

(*) original German inscription: "Gott gebe allen, die mich kennen, doppelt so viel, als sie mir gönnen"

fastrlock's People

Contributors

Stargazers

Watchers

Forkers

sonots hmaarrfk premprabhat odidev prakriti07 kmaehashi aosingh python-repository-hub arpitjain799

fastrlock's Issues

pip install fails on Windows 10

Windows 10
CUDA 10.0
python 3.7.4
pip 19.0.3

CUDA and cuDNN are installed and are working correctly - I've tested with tensorflow-gpu and it works fine.

Visual Studio is installed.

Trying to install fastrlock (really - trying to install cupy, for which fastrlock is a dependency) with pip and it fails. It's probably something simple but I'm a total newbie to Visual Studio (I'm more at home on Linux).

C:\Users\florin>pip3 install --user fastrlock
Collecting fastrlock
  Using cached https://files.pythonhosted.org/packages/8f/ff/6777b85fdbc6efd572a2b713e044c29f5f01ca8b1c74f8d94b427bac60c3/fastrlock-0.4.tar.gz
Building wheels for collected packages: fastrlock
  Building wheel for fastrlock (setup.py) ... error
  Complete output from command "c:\program files\python37\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\florin\\AppData\\Local\\Temp\\pip-install-479f2_e3\\fastrlock\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\florin\AppData\Local\Temp\pip-wheel-csg06n1t --python-tag cp37:
  building without Cython
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.7
  creating build\lib.win-amd64-3.7\fastrlock
  copying fastrlock\__init__.py -> build\lib.win-amd64-3.7\fastrlock
  copying fastrlock\rlock.pxd -> build\lib.win-amd64-3.7\fastrlock
  copying fastrlock\__init__.pxd -> build\lib.win-amd64-3.7\fastrlock
  copying fastrlock\_lock.pxi -> build\lib.win-amd64-3.7\fastrlock
  running build_ext
  building 'fastrlock.rlock' extension
  error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

  ----------------------------------------
  Failed building wheel for fastrlock
  Running setup.py clean for fastrlock
Failed to build fastrlock
Installing collected packages: fastrlock
  Running setup.py install for fastrlock ... error
    Complete output from command "c:\program files\python37\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\florin\\AppData\\Local\\Temp\\pip-install-479f2_e3\\fastrlock\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\florin\AppData\Local\Temp\pip-record-pqknhnzg\install-record.txt --single-version-externally-managed --compile --user --prefix=:
    building without Cython
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.7
    creating build\lib.win-amd64-3.7\fastrlock
    copying fastrlock\__init__.py -> build\lib.win-amd64-3.7\fastrlock
    copying fastrlock\rlock.pxd -> build\lib.win-amd64-3.7\fastrlock
    copying fastrlock\__init__.pxd -> build\lib.win-amd64-3.7\fastrlock
    copying fastrlock\_lock.pxi -> build\lib.win-amd64-3.7\fastrlock
    running build_ext
    building 'fastrlock.rlock' extension
    error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

    ----------------------------------------
Command ""c:\program files\python37\python.exe" -u -c "import setuptools, tokenize;__file__='C:\\Users\\florin\\AppData\\Local\\Temp\\pip-install-479f2_e3\\fastrlock\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\florin\AppData\Local\Temp\pip-record-pqknhnzg\install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in C:\Users\florin\AppData\Local\Temp\pip-install-479f2_e3\fastrlock\
You are using pip version 19.0.3, however version 19.2.2 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Wheels for Python 3.10

Hi, I am contacting you on behalf of @kmaehashi #9 #11, and can I ask you to upload wheels for Python 3.10 for Linux / Windows? CuPy still relies on fastrlock in the Cython layer.

unittest.findTestCases is deprecated in Python 3.11 and will be removed in Python 3.13

Ref : python/cpython#28382

fastrlock/fastrlock/tests/__init__.py

Line 11 in f2ae506

tests.append(unittest.findTestCases('tests.'+ filename[:-3]))

Add Wheels for Python 3.10 Windows

Hi!
given "pip install fastrlock" didnt work out on Windows, and i dont see any Python 3.10 Wheels for Windows listed in this repo, would you be able to add such a wheel?

Thank you in advance!

No such file or directory: '/tmp/easy_install-6cvm28rx/fastrlock-0.1/README.rst'

With setup.py

install_requires = [
    'fastrlock>=0.1',
]

I got following error:

$ python setup.py develop
Searching for fastrlock>=0.1
Reading https://pypi.python.org/simple/fastrlock/
Downloading https://pypi.python.org/packages/25/07/20e36384c749af052b57f6179745aa006b44b28657b884af8542f4f10572/fastrlock-0.1.tar.gz#md5=6076db7b1561ad6c404592ba0fc8cbc5
Best match: fastrlock 0.1
Processing fastrlock-0.1.tar.gz
Writing /tmp/easy_install-6cvm28rx/fastrlock-0.1/setup.cfg
Running fastrlock-0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-6cvm28rx/fastrlock-0.1/egg-dist-tmp-h3nufkcm
building without Cython
error: [Errno 2] No such file or directory: '/tmp/easy_install-6cvm28rx/fastrlock-0.1/README.rst'

It seems that downloaded fastrlock-0.1.tar.gz does not have README.rst

$ tar zxvf fastrlock-0.1.tar.gz
fastrlock-0.1/
fastrlock-0.1/setup.py
fastrlock-0.1/PKG-INFO
fastrlock-0.1/MANIFEST.in
fastrlock-0.1/fastrlock/
fastrlock-0.1/fastrlock/rlock.pyx
fastrlock-0.1/fastrlock/rlock.c
fastrlock-0.1/fastrlock/tests/
fastrlock-0.1/fastrlock/tests/test_rlock.py
fastrlock-0.1/fastrlock/tests/__init__.py
fastrlock-0.1/fastrlock/__init__.py
fastrlock-0.1/fastrlock/rlock.pxd

although fastrlock's setup.py refers it

fastrlock/setup.py

Line 82 in 1009af5

for text_file in ['README.rst', 'CHANGES.rst']])

Wheels for Python 3.8

It would be great if you provide wheels for Python 3.8.
Thanks!

Add support to release Linux aarch64 wheels

Problem

On aarch64, ‘pip install fastrlock’ builds the wheels from source code and then installs it. It requires the user to have a development environment installed on his system. Also, it takes some time to build the wheels than downloading and extracting the wheels from pypi.

Resolution

On aarch64, ‘pip install fastrlock’ should download the wheels from pypi

@scoder Please let me know your interest in releasing aarch64 wheels. I can help in this.

Not fast?

I wrote benchmark scripts as follows to compare:

lock_bench.py

import time
import threading

lock = threading.RLock()

start_time = time.time()
for i in range(100000):
    lock.acquire()
    lock.release()
    # with lock:
    #     pass
elapsed_time = time.time() - start_time
print(elapsed_time)

fastrlock_bench.py

import time
from fastrlock import rlock

lock = rlock.FastRLock()

start_time = time.time()
for i in range(100000):
    lock.acquire()
    lock.release()
    # with lock:
    #     pass
elapsed_time = time.time() - start_time
print(elapsed_time)

I obtained following results which show fastrlock is not fast.

$ python ~/lock_bench.py
0.03992772102355957
$ python ~/fastrlock_bench.py
0.04404258728027344

My python and cython version is as follows:

$ python --version
Python 3.6.1
$ cython --version
Cython version 0.25.2

Could you tell me if I am doing something wrong?

test fail with error "ImportError: cannot import name 'rlock' from 'fastrlock'"

step to reproduce:

python setup.py build
PYTHONPATH="${PWD}/build/lib.linux-${CARCH}-3.8" pytest -v

The complete log:

==> Starting check()...
============================= test session starts ==============================
platform linux -- Python 3.8.0, pytest-5.2.4, py-1.8.0, pluggy-0.13.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /build/python-fastrlock/src/fastrlock-0.4
collecting ... collected 0 items / 1 errors

==================================== ERRORS ====================================
________________ ERROR collecting fastrlock/tests/test_rlock.py ________________
ImportError while importing test module '/build/python-fastrlock/src/fastrlock-0.4/fastrlock/tests/test_rlock.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
fastrlock/tests/test_rlock.py:14: in <module>
    from fastrlock import rlock
E   ImportError: cannot import name 'rlock' from 'fastrlock' (/build/python-fastrlock/src/fastrlock-0.4/fastrlock/__init__.py)
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.25s ===============================
==> ERROR: A failure occurred in check().
    Aborting...
==> ERROR: Build failed, check /var/lib/archbuild/extra-x86_64/hubutui/build

Wheels for Python 3.9

Hello, sorry to bother you again (#9).
It would be great if you have time to upload wheels for Python 3.9 for Linux / Windows.
Thanks in advance!

Free-threaded Python support

Hi,

Recently PEP 703 has been approved, which removes the GIL in Python. The timeline of the change is far, however I'd like to ask, is there a particular plan of fastrlock working without the GIL?

I'm particularly asking, since from the How does it work section I understand that fastrlock is inherently coupled with GIL.

Ideally fastrlock API might stay unchanged, so that any project using fastrlock does not require code changes.

from fastrlock cimport rlock failed with setup.py

With setup.py

install_requires = [
    'fastrlock>=0.2',
]

and

from fastrlock cimport rlock

, I got a following error:

from fastrlock cimport rlock
^
------------------------------------------------------------

cupy/cuda/memory.pyx:11:0: 'fastrlock.pxd' not found
Traceback (most recent call last):
  File "setup.py", line 30, in <module>
    ext_modules = cupy_setup_build.get_ext_modules()
  File "/home/sonots/cupy/branch_fix_race_condition_memory_pool_fastrlock/cupy_setup_build.py", line 371, in get_ext_modules
    extensions = cythonize(extensions, arg_options)
  File "/home/sonots/cupy/branch_fix_race_condition_memory_pool_fastrlock/cupy_setup_build.py", line 342, in cythonize
    compiler_directives=directives, **cythonize_options)
  File "/home/sonots/.pyenv/versions/3.6.1/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 934, in cythonize
    cythonize_one(*args)
  File "/home/sonots/.pyenv/versions/3.6.1/lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1056, in cythonize_one
    raise CompileError(None, pyx_file)

although I found following works:

from fastrlock.rlock cimport create_fastrlock
from fastrlock.rlock cimport lock_fastrlock
from fastrlock.rlock cimport unlock_fastrlock