google / python-subprocess32 Goto Github PK
View Code? Open in Web Editor NEWA backport of the Python 3 subprocess module for use on Python 2.
Home Page: https://pypi.python.org/pypi/subprocess32/
License: Other
A backport of the Python 3 subprocess module for use on Python 2.
Home Page: https://pypi.python.org/pypi/subprocess32/
License: Other
Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (In
tel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess32
>>> subprocess32.run
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'run'
>>> subprocess32.__file__
'C:\\Program Files (x86)\\Python34\\lib\\subprocess.py'
C:\Users\Owner\AppData\Roaming\Python\Python34\site-packages\subprocess32_init_.py
"""When installed as subprocess32, this sets up a redirect to subprocess."""
import subprocess
import sys
if sys.version_info[:2] < (3,3):
raise ImportError('Ancient Python 3 versions are not supported.')
# Doing this could crash some older Python interpreters due to the module
# reference going away before the import is complete?
sys.modules['subprocess32'] = subprocess
Python 2 works fine. I guess something needs to be fixed in the build script?
Lines 61 to 65 in 035d02a
One gets
_posixsubprocess.c:764:22: error: use of undeclared identifier 'O_CLOEXEC'
res = pipe2(fds, O_CLOEXEC);
while building the extension. The problem is that O_CLOEXEC
defined in fcntl.h
inside an #if
, with the condition that does not hold true:
#if __POSIX_VISIBLE >= 200809
/* Defined by POSIX 1003.1-2008; BSD default, but reserve for future use. */
#define O_TTY_INIT 0x00080000 /* Restore default termios attributes */
#define O_CLOEXEC 0x00100000
#endif
To fix the build I ended up adding -DO_CLOEXEC=0x00100000
to CFLAGS
, after trying without success to do this more properly.
subprocess32 does not have a binary manylinux1 wheel, something that could speedup installation on many systems and also lower the footprint by not requiring gcc for installation.
I try to build it, but the compiler hang on this screen:
python setup.py --verbose build
running build
running build_py
not copying subprocess32.py (output up-to-date)
running build_ext
building '_posixsubprocess' extension
gcc -fno-strict-aliasing -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/usr/src/ports/python/python2-2.7.13-1.x86_64/build=/usr/src/debug/python2-2.7.13-1 -fdebug-prefix-map=/usr/src/ports/python/python2-2.7.13-1.x86_64/src/Python-2.7.13=/usr/src/debug/python2-2.7.13-1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/python2.7 -c _posixsubprocess.c -o build/temp.cygwin-2.8.0-x86_64-2.7/_posixsubprocess.o
$ gcc --version
gcc (GCC) 5.4.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ python --version
Python 2.7.13
It also hangs only using:
$ gcc -c _posixsubprocess.c -o build/temp.cygwin-2.8.0-x86_64-2.7/_posixsubprocess.o
Gentoo sandbox enforces itself into child processes, and therefore the environment is not empty:
======================================================================
FAIL: test_empty_env (__main__.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subprocess32.py", line 545, in test_empty_env
self.assertEqual(stdout.strip(), "[]")
AssertionError: "['LD_PRELOAD', 'SANDBOX_READ', 'SANDBOX_MESSAGE_P@TH', 'SANDBOX_ON', 'SANDBOX_DENY', 'SANDBOX_VERBOSE', 'SANDBOX_LOG', 'SANDBOX_PREDICT', 'SANDBOX_DEBUG_LOG', 'SANDBOX_ACTIVE', 'SANDBOX_WRITE']" != '[]'
======================================================================
FAIL: test_empty_env (__main__.ProcessTestCasePOSIXPurePython)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subprocess32.py", line 545, in test_empty_env
self.assertEqual(stdout.strip(), "[]")
AssertionError: "['LD_PRELOAD', 'SANDBOX_READ', 'SANDBOX_MESSAGE_P@TH', 'SANDBOX_ON', 'SANDBOX_DENY', 'SANDBOX_VERBOSE', 'SANDBOX_LOG', 'SANDBOX_PREDICT', 'SANDBOX_DEBUG_LOG', 'SANDBOX_ACTIVE', 'SANDBOX_WRITE']" != '[]'
======================================================================
FAIL: test_empty_env (__main__.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subprocess32.py", line 545, in test_empty_env
self.assertEqual(stdout.strip(), "[]")
AssertionError: "['LD_PRELOAD', 'SANDBOX_READ', 'SANDBOX_MESSAGE_P@TH', 'SANDBOX_ON', 'SANDBOX_DENY', 'SANDBOX_VERBOSE', 'SANDBOX_LOG', 'SANDBOX_PREDICT', 'SANDBOX_DEBUG_LOG', 'SANDBOX_ACTIVE', 'SANDBOX_WRITE']" != '[]'
----------------------------------------------------------------------
Ran 264 tests in 197.918s
FAILED (failures=3)
I have a library (subx) which depends on subprocess32.
My library needs the timeout kwarg.
I need subprocess32 only if the target platform is Python2.x.
How should I define the dependency in my project?
I get this error message, if I define a dependency to subprocess32
via "install_requires" (setup.py) and I am inside a python3 virtualenv:
===> pip install -e git+https://github.com/guettli/subx.git#egg=subx
Obtaining subx from git+https://github.com/guettli/subx.git#egg=subx
Cloning https://github.com/guettli/subx.git to ./src/subx
Collecting subprocess32 (from subx)
Using cached subprocess32-3.2.7.tar.gz
Complete output from command python setup.py egg_info:
This backport is for Python 2.x only.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-lju3nl1y/subprocess32/
Same question here: https://stackoverflow.com/questions/47098185/dependency-management-subprocess32-needed-for-python2-7
There have been some important changes since release 3.2.7
, in particular PR #13 . Are there any plans to release an up-to-date version soon? Thanks
>>> subprocess32.run(["ls", "-l"], capture_output=True)
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/vincent/.local/lib/python2.7/site-packages/subprocess32.py", line 406, in run process = Popen(*popenargs, **kwargs) TypeError: __init__() got an unexpected keyword argument 'capture_output'
This is not surprising because that's what the code does. However, this behavior has been updated in Python 3.7 (https://docs.python.org/3/library/subprocess.html#subprocess.run, "Changed in version 3.7: Added the text parameter, as a more understandable alias of universal_newlines. Added the capture_output parameter."). In the cpython repository, subprocess.py has been updated with additional parameters (see https://github.com/python/cpython/blob/master/Lib/subprocess.py#L433), that are not present in subprocess32.
While subprocess32 only supporting subprocess up to 3.5 is well documented, there's some additional updates which would be valuable to backport. Thanks!
When testing https://github.com/google/python-subprocess32/blob/master/setup.py#L20 promised ability to install under Python3.x
$ pip install subprocess32==3.5.0rc1
Collecting subprocess32==3.5.0rc1
Downloading https://files.pythonhosted.org/packages/28/91/d1283618eba07c4e8e18c58b3fd8b5ff3a8992fb652a3720535ddf2f2916/subprocess32-3.5.0rc1.tar.gz (54kB)
100% |████████████████████████████████| 61kB 843kB/s
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/subprocess32.egg-info
writing top-level names to pip-egg-info/subprocess32.egg-info/top_level.txt
writing dependency_links to pip-egg-info/subprocess32.egg-info/dependency_links.txt
writing pip-egg-info/subprocess32.egg-info/PKG-INFO
writing manifest file 'pip-egg-info/subprocess32.egg-info/SOURCES.txt'
/home/michael/cwltool/env3/lib/python3.5/site-packages/setuptools/version.py:1: UserWarning: Module _posixsubprocess was already imported from None, but /tmp/pip-install-gk2ke0g4/subprocess32 is being added to sys.path
import pkg_resources
subprocess32 == subprocess on Python 3.
error: package directory 'python3_redirect' does not exist
----------------------------------------
return an error code as -11
I'm trying to use the latest version of subprocess32 (3.2.5) on a cross
platform project and have encountered some problems using the subprocess32
module on Windows.
OS = Windows 7
Python = 2.7.6
Issue 1: Using the setup.py intsaller fails on Windows because of the posix
extension modules. Removing the line
ext_modules=[ext],
from setup.py allows the installation to proceed for windows systems.
Issue 2: The subprocess32 code assumes that _subprocess.c has an attribute
WAIT_TIMEOUT which isn't the case for Python 2.7 (line 1137 of subprocess32.py)
Nick
Original issue reported on code.google.com by [email protected]
on 24 Mar 2014 at 3:40
Is the correct way to do it to remove the /usr/lib/python2.7/subprocess.py
module and replace it with subprocess32? How should i install subprocess32 as default?
In Popen._communicate, when TimeoutExpired is called (lines 1190 and 1194) the second argument (timeout) is omitted, resulting in a TypeError.
EDIT: I'm moving too quickly today, basically a dupliate of #7
Here's a script to reproduce the issue:
import os
import sys
if os.name == 'posix' and sys.version_info[0] < 3:
import subprocess32 as subprocess
else:
import subprocess
p1 = subprocess.Popen(['echo', 'hi'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['echo', 'bye'], stdout=subprocess.PIPE)
# uncommenting these lines will produce expected result in python2.7
# subprocess._set_cloexec(p1.stdout.fileno(), False)
# subprocess._set_cloexec(p2.stdout.fileno(), False)
subs = [p1, p2]
fds = [i.stdout.fileno() for i in subs]
args = ['/dev/fd/{}'.format(fd) for fd in fds]
p2 = subprocess.Popen(
['cat'] + args,
pass_fds=fds
)
python 3.5 output:
hi
bye
python 2.7 output:
cat: /dev/fd/3: No such file or directory
cat: /dev/fd/4: No such file or directory
My C isn't very good, but I traced through the _posixsubprocess.c
and fileutils.c
source on the 2.7 tag of cpython sources on GitHub, and it looks like it should be calling ioctl(fd, FIONCLEX, NULL)
for each fd in pass_fds
, so I haven't been able to figure out why this isn't working in python 2.7.
In Python 2.x, subclasses of Exception must either call super(...).__init__(*args)
or set self.args
in order to be successfully pickled/unpickled.
Example:
>>> import subprocess32, pickle
>>> t = subprocess32.TimeoutExpired(['foo', 'bar'], 60)
>>> pickle.loads(pickle.dumps(t))
TypeError Traceback (most recent call last)
<ipython-input-6-c105e6dd98fb> in <module>()
----> 1 pickle.loads(pickle.dumps(t))
/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc in loads(str)
1386 def loads(str):
1387 file = StringIO(str)
-> 1388 return Unpickler(file).load()
1389
1390 # Doctest
/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc in load(self)
862 while 1:
863 key = read(1)
--> 864 dispatch[key](self)
865 except _Stop, stopinst:
866 return stopinst.value
/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc in load_reduce(self)
1137 args = stack.pop()
1138 func = stack[-1]
-> 1139 value = func(*args)
1140 stack[-1] = value
1141 dispatch[REDUCE] = load_reduce
TypeError: __init__() takes at least 3 arguments (1 given)
Python bug: https://bugs.python.org/issue1692335
Explanation: https://stackoverflow.com/questions/41808912/cannot-unpickle-exception-subclass
Example Solution:
>>> import subprocess32, pickle
>>> t = subprocess32.TimeoutExpired(['foo', 'bar'], 60)
>>> pickle.loads(pickle.dumps(t))
>>> t.args = (t.cmd, t.timeout, t.stdout, t.stderr) # Magic!
>>> pickle.loads(pickle.dumps(t))
# No exception now!
Over at matplotlib/matplotlib#6945 a user found that they needed to install XCode only for subprocess32, in an install of the matplotlib plotting package. All the other dependencies are either pure Python, or have wheels for OSX / Linux (e.g. numpy).
Would you consider uploading wheels for subprocess32 ? That would be very helpful.
As proof of concept, I made a wheel-building repo at:
https://github.com/MacPython/subprocess32-wheels
This already builds a functional OSX wheel for 3.2.7:
https://travis-ci.org/MacPython/subprocess32-wheels/jobs/152523378
Manylinux wheels get broken by #12 - does eae9e95 fix that problem for manylinux wheels? Tests seem to pass on Ubuntu 14.04 and a wheel from current master:
https://travis-ci.org/MacPython/subprocess32-wheels/jobs/152529549
I'm very happy to add y'all as owners of the build repo.
When Windows support was added to subprocess.getstatusoutput
we inadvertently made it backwards incompatible with the behaviour of the 2.7 version: https://bugs.python.org/issue22635
Now, backwards compatibility within the 3.x series means we can't readily change it back to make it match the Python 2.7 behaviour (even though the only reason it changed in the first place was due to a gap in our test coverage).
This impacts subprocess32 in two ways:
subprocess32
)subprocess32
were to support 3.x in addition to 2.7, it could provide a way for affected applications to readily work around the problemI'm not sure how difficult the latter would be, though.
I did a drop in replacement of python system library subprocess with subprocess32 & renamed the subprocess32 to subprocess in the standard install location. When importing multiprocessing I get an import error
>>> import multiprocessing
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/vsnag/Python-2.7.12/Lib/multiprocessing/__init__.py", line 65, in
<module>
from multiprocessing.util import SUBDEBUG, SUBWARNING
File "/home/vsnag/Python-2.7.12/Lib/multiprocessing/util.py", line 41, in <mod
ule>
from subprocess import _args_from_interpreter_flags
ImportError: cannot import name _args_from_interpreter_flags
Adding this https://github.com/python/cpython/blob/master/Lib/subprocess.py#L536 could be a possible fix.
Hey there,
thanks for making subprocess32 available, it really helped us fighting random lock-ups in our application. I would not have dared to backport subprocess from Python 3 and resorted to workarounds.
After issues with mixups between subprocess32 and subprocess (some libraries still use subprocess and know nothing about subprocess32) we decided to heed this advice:
Or if you fully control your POSIX Python 2.7 installation, this can serve as a replacement for its subprocess module. Users will thank you by not filing concurrency bugs.
So far this looked good. Out of curiousity I ran the Python 2.7 stdlib test suite against subprocess32 and was surprised that it failed.
I offer this Dockerfile to reproduce this:
$ cat Dockerfile
FROM debian:9
RUN apt-get update
RUN apt-get install -y python2.7 libpython2.7-testsuite python-pip
# Install subprocess32, replace subprocess
RUN pip install subprocess32
RUN cp /usr/local/lib/python2.7/dist-packages/subprocess32.py /usr/lib/python2.7/subprocess.py
RUN cp /usr/local/lib/python2.7/dist-packages/_posixsubprocess32.so /usr/lib/python2.7/lib-dynload/_posixsubprocess32.so
# Run regrtest
RUN python -m test.regrtest --verbose test_subprocess
This results into these failures:
$ sudo docker build .
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM debian:9
---> de8b49d4b0b3
Step 2/7 : RUN apt-get update
---> Using cache
---> 21f3d4930519
[...]
Step 7/7 : RUN python -m test.regrtest --verbose test_subprocess
---> Running in 58064776ae73
== CPython 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516]
== Linux-4.15.0-47-generic-x86_64-with-debian-9.6 little-endian
== /tmp/test_python_7
Testing with flags: sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, tabcheck=0, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0)
[1/1] test_subprocess
test_call_kwargs (test.test_subprocess.ProcessTestCase) ... ok
[...]
test test_subprocess failed -- multiple errors occurred
skipped 'mswindows only'
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
======================================================================
ERROR: test_exceptions (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 803, in test_exceptions
self.assertIn("os.chdir", c.exception.child_traceback)
AttributeError: 'exceptions.OSError' object has no attribute 'child_traceback'
======================================================================
ERROR: test_preexec_errpipe_does_not_double_close_pipes (test.test_subprocess.POSIXProcessTestCase)
Issue16140: Don't double close pipes on preexec error.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 868, in test_preexec_errpipe_does_not_double_close_pipes
stderr=subprocess.PIPE, preexec_fn=raise_it)
File "/usr/lib/python2.7/test/test_subprocess.py", line 827, in __init__
subprocess.Popen.__init__(self, *args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 614, in __init__
restore_signals, start_new_session)
TypeError: _execute_child() takes exactly 18 arguments (20 given)
======================================================================
ERROR: test_communicate_epipe (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 706, in test_communicate_epipe
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 724, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python2.7/subprocess.py", line 1538, in _communicate
orig_timeout)
File "/usr/lib/python2.7/subprocess.py", line 1684, in _communicate_with_select
bytes_written = os.write(self.stdin.fileno(), chunk)
OSError: [Errno 32] Broken pipe
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
----------------------------------------------------------------------
Ran 146 tests in 16.424s
Some tests are bound to fail given that they access the internals of the Python 2.7 implementation. But these look relevant:
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
======================================================================
ERROR: test_communicate_epipe (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 706, in test_communicate_epipe
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 724, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python2.7/subprocess.py", line 1538, in _communicate
orig_timeout)
File "/usr/lib/python2.7/subprocess.py", line 1684, in _communicate_with_select
bytes_written = os.write(self.stdin.fileno(), chunk)
OSError: [Errno 32] Broken pipe
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
I intent to find out the cause and fix this, but for reference I'd like to have this issue public in the mean time.
BTW: There is no mention of epipe in the test_subprocess32.py tests:
~/workspace/python-subprocess32$ grep -i epipe test_subprocess32.py
Greetings, Torsten
$ pip install -U subprocess32
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/graingert/.cache/pip/wheels/68/39/1a/5e402bdfdf004af1786c8b853fd92f8c4a04f22aad179654d1/subprocess32-3.5.4-cp27-cp27mu-linux_x86_64.whl
Installing collected packages: subprocess32
Successfully installed subprocess32-3.5.4
$ python2
Python 2.7.16 (default, Oct 7 2019, 17:36:04)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess32 as subprocess
>>> isinstance(subprocess.run("ls", universal_newlines=True, stdout=subprocess.PIPE).stdout, bytes)
True
$ python3.5
Python 3.5.8 (default, Oct 31 2019, 00:57:08)
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> isinstance(subprocess.run("ls", universal_newlines=True, stdout=subprocess.PIPE).stdout, bytes)
False
The PY_VERSION_HEX test should match in "init_posixsubprocess" as well.
On Python versions like 2.6.1, "imp_module" is not getting initialized, so
calls to "_PyImport_AcquireLock" fail because "imp_module" is NULL.
Original issue reported on code.google.com by [email protected]
on 30 Nov 2012 at 2:31
Attachments:
The usage is like
import subprocess32 as subprocess
subprocess.Popen(['ls', '-l'], stdout=subprocess.DEVNULL)
Otherwise continuous integration with this thing included needs an unnecessary complex way of discerning 2.7 & 3.x dependencies - i.e. I don't want to have a seperate requirements.txt for 2.x.. Or is there something i missed?
_PySequence_BytesToCharpArray contains a heap overflow after a integer wraparound occurs when large sequences are passed to subprocess_fork_exec.
This can be reproduced on a 32-bit system. It probably can be reproduced on 64bit systems with a few exabytes of RAM. (I don't have anything like that to test with unfortunately...)
import subprocess32 as subprocess
subprocess._posixsubprocess.fork_exec("a",'a'*0x3FFFFFFF,"a",set(),"a","a",1,1,1,1,1,1,1,3,1,1,"a")
*** glibc detected *** /usr/bin/python: double free or corruption (!prev): 0x09464b38 ***
Segmentation fault (core dumped)
It can be seen in _posixsubprocess_helpers.c on line 119:
argc = PySequence_Size(self);
if (argc == -1)
return NULL;
array = malloc((argc + 1) * sizeof(char *));
argc is the size of the sequence passed to the C library. Thus, if we can pass a value such that size + 1 * 4 > 0xFFFFFFFF (0x3fffffff), malloc should return a small value, and invalid heap writes will occur in the loop below.
I was unable to trigger this through the actual unhidden API interface, so this flaw seems low priority, but it's always possible someone else could figure out a better way to trigger the overflow. Trying to generate a valid args list that large doesn't seem possible on my system. Guessing the list overhead is too much to make this work. Passing string sequences work, but the API converts them to a list before sending them to the C library I believe.
I posted this issue here but as Gregory pointed out it is more suited to here, if at all. https://bugs.python.org/issue31383
I am trying to install subprocess32 with my python 2.7 installation via buildroot. It appeared to install correctly but when I import it on the embedded system I get an error:
>>> import subprocess32
/usr/lib/python2.7/site-packages/subprocess32.py:472: RuntimeWarning: The _posixsubprocess module is not being used. Child process reliability may suffer if your program uses threads.
"program uses threads.", RuntimeWarning)
Following this path I tried to import _posixsubprocess
import _posixsubprocess
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define init function (init_posixsubprocess)
I can use subprocess32 otherwise, tested out basic functionality like subprocess32.Popen(["ls"]).communicate()
As was pointed out on bugs.python.org init_posixsubprocess does exist in _posixsubprocess.c
This may be an issue with buildroot handling it, however I will post a solution if I find it here to help others.
What I see when I build it seems innocuous:
^[[7m>>> python-subprocess32 3.2.7 Extracting^[[27m
^[[7m>>> python-subprocess32 3.2.7 Patching package/python-subprocess32^[[27m
^[[7m>>> python-subprocess32 3.2.7 Configuring^[[27m
^[[7m>>> python-subprocess32 3.2.7 Building^[[27m
(cd /media/vmpart/5.1.0_x86/buildroot/output/build/python-subprocess32-3.2.7; /media/vmpart/5.1.0_x86/buildroot/output/host/usr/bin/python setup.py build)
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying subprocess32.py -> build/lib.linux-x86_64-2.7
running build_ext
building '_posixsubprocess' extension
creating build/temp.linux-x86_64-2.7
copying subprocess32.py -> build/lib.linux-x86_64-2.7
running build_ext
building '_posixsubprocess' extension
creating build/temp.linux-x86_64-2.7
/media/vmpart/5.1.0_x86/buildroot/output/host/usr/bin/ccache /usr/bin/gcc -pthread -fno-strict-aliasing -O2 -I/media/vmpart/5.1.0_x86/buildroot/output/host/include -I/media/vmpart/5.1.0_x86/buildroot/output/host/usr/include -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/media/vmpart/5.1.0_x86/buildroot/output/host/usr/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o
/media/vmpart/5.1.0_x86/buildroot/output/host/usr/bin/ccache /usr/bin/gcc -pthread -shared -L/media/vmpart/5.1.0_x86/buildroot/output/host/lib -L/media/vmpart/5.1.0_x86/buildroot/output/host/usr/lib -Wl,-rpath,/media/vmpart/5.1.0_x86/buildroot/output/host/usr/lib build/temp.linux-x86_64-2.7/_posixsubprocess.o -L/media/vmpart/5.1.0_x86/buildroot/output/host/usr/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/_posixsubprocess.so
^[[7m>>> python-subprocess32 3.2.7 Installing to target^[[27m
(cd /media/vmpart/5.1.0_x86/buildroot/output/build/python-subprocess32-3.2.7; /media/vmpart/5.1.0_x86/buildroot/output/host/usr/bin/python setup.py install --prefix=/media/vmpart/5.1.0_x86/buildroot/output/target/usr)
running install
running build
running build_py
running build_ext
running install_lib
copying build/lib.linux-x86_64-2.7/_posixsubprocess.so -> /media/vmpart/5.1.0_x86/buildroot/output/target/usr/lib/python2.7/site-packages
running install_egg_info
Removing /media/vmpart/5.1.0_x86/buildroot/output/target/usr/lib/python2.7/site-packages/subprocess32-3.2.7-py2.7.egg-info
Writing /media/vmpart/5.1.0_x86/buildroot/output/target/usr/lib/python2.7/site-packages/subprocess32-3.2.7-py2.7.egg-info
Make file (buildroot version does not have python-package):
#############################################################
#
# Subprocess32 module for python
#
#############################################################
PYTHON_SUBPROCESS32_VERSION = 3.2.7
PYTHON_SUBPROCESS32_SOURCE = subprocess32-$(PYTHON_SUBPROCESS32_VERSION).tar.gz
PYTHON_SUBPROCESS32_SITE = https://pypi.python.org/pypi/subprocess32
PYTHON_SUBPROCESS32_DEPENDENCIES = python
define PYTHON_SUBPROCESS32_BUILD_CMDS
(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
endef
define PYTHON_SUBPROCESS32_INSTALL_TARGET_CMDS
(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
endef
$(eval $(call GENTARGETS,package,python-subprocess32))
(coma) ➜ coma git:(master) ✗ pip install subprocess32
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting subprocess32
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/32/c8/564be4d12629b912ea431f1a50eb8b3b9d00f1a0b1ceff17f266be190007/subprocess32-3.5.4.tar.gz
Building wheels for collected packages: subprocess32
Building wheel for subprocess32 (setup.py) ... error
ERROR: Complete output from command ${HOME}/.conda/envs/coma/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-6Mmw2Y/subprocess32/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-hJ231z --python-tag cp27:
ERROR: running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying subprocess32.py -> build/lib.linux-x86_64-2.7
running build_ext
running build_configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for unistd.h... (cached) yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking sys/cdefs.h usability... yes
checking sys/cdefs.h presence... yes
checking for sys/cdefs.h... yes
checking for sys/types.h... (cached) yes
checking for sys/stat.h... (cached) yes
checking sys/syscall.h usability... yes
checking sys/syscall.h presence... yes
checking for sys/syscall.h... yes
checking for dirent.h that defines DIR... yes
checking for library containing opendir... none required
checking for pipe2... yes
checking for setsid... yes
checking whether dirfd is declared... yes
configure: creating ./config.status
config.status: creating _posixsubprocess_config.h
building '_posixsubprocess32' extension
creating build/temp.linux-x86_64-2.7
gcc -pthread -B ${HOME}/.conda/envs/coma/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I${HOME}/.conda/envs/coma/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o
In file included from ${HOME}/.conda/envs/coma/include/python2.7/Python.h:8,
from _posixsubprocess.c:16:
${HOME}/.conda/envs/coma/include/python2.7/pyconfig.h:1220: warning: "_POSIX_C_SOURCE" redefined
1220 | #define _POSIX_C_SOURCE 200112L
|
In file included from /usr/include/sys/cdefs.h:23,
from _posixsubprocess.c:12:
/usr/include/features.h:266: note: this is the location of the previous definition
266 | # define _POSIX_C_SOURCE 200809L
|
In file included from ${HOME}/.conda/envs/coma/include/python2.7/Python.h:8,
from _posixsubprocess.c:16:
${HOME}/.conda/envs/coma/include/python2.7/pyconfig.h:1242: warning: "_XOPEN_SOURCE" redefined
1242 | #define _XOPEN_SOURCE 600
|
In file included from /usr/include/sys/cdefs.h:23,
from _posixsubprocess.c:12:
/usr/include/features.h:203: note: this is the location of the previous definition
203 | # define _XOPEN_SOURCE 700
|
gcc -pthread -shared -B ${HOME}/.conda/envs/coma/compiler_compat -L${HOME}/.conda/envs/coma/lib -Wl,-rpath=${HOME}/.conda/envs/coma/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-2.7/_posixsubprocess.o -L${HOME}/.conda/envs/coma/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/_posixsubprocess32.so
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
build/temp.linux-x86_64-2.7/_posixsubprocess.o: file not recognized: file format not recognized
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for subprocess32
Running setup.py clean for subprocess32
Failed to build subprocess32
Installing collected packages: subprocess32
Running setup.py install for subprocess32 ... error
ERROR: Complete output from command ${HOME}/.conda/envs/coma/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-6Mmw2Y/subprocess32/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-Zv3qW2/install-record.txt --single-version-externally-managed --compile:
ERROR: running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying subprocess32.py -> build/lib.linux-x86_64-2.7
running build_ext
running build_configure
_posixsubprocess_config.h is already up to date.
building '_posixsubprocess32' extension
creating build/temp.linux-x86_64-2.7
gcc -pthread -B ${HOME}/.conda/envs/coma/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I${HOME}/.conda/envs/coma/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o
In file included from ${HOME}/.conda/envs/coma/include/python2.7/Python.h:8,
from _posixsubprocess.c:16:
${HOME}/.conda/envs/coma/include/python2.7/pyconfig.h:1220: warning: "_POSIX_C_SOURCE" redefined
1220 | #define _POSIX_C_SOURCE 200112L
|
In file included from /usr/include/sys/cdefs.h:23,
from _posixsubprocess.c:12:
/usr/include/features.h:266: note: this is the location of the previous definition
266 | # define _POSIX_C_SOURCE 200809L
|
In file included from ${HOME}/.conda/envs/coma/include/python2.7/Python.h:8,
from _posixsubprocess.c:16:
${HOME}/.conda/envs/coma/include/python2.7/pyconfig.h:1242: warning: "_XOPEN_SOURCE" redefined
1242 | #define _XOPEN_SOURCE 600
|
In file included from /usr/include/sys/cdefs.h:23,
from _posixsubprocess.c:12:
/usr/include/features.h:203: note: this is the location of the previous definition
203 | # define _XOPEN_SOURCE 700
|
gcc -pthread -shared -B ${HOME}/.conda/envs/coma/compiler_compat -L${HOME}/.conda/envs/coma/lib -Wl,-rpath=${HOME}/.conda/envs/coma/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-2.7/_posixsubprocess.o -L${HOME}/.conda/envs/coma/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/_posixsubprocess32.so
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
${HOME}/.conda/envs/coma/compiler_compat/ld: build/temp.linux-x86_64-2.7/_posixsubprocess.o: unable to initialize decompress status for section .debug_info
build/temp.linux-x86_64-2.7/_posixsubprocess.o: file not recognized: file format not recognized
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command "${HOME}/.conda/envs/coma/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-6Mmw2Y/subprocess32/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-Zv3qW2/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-6Mmw2Y/subprocess32/
Hi, I encountered build error on archlinux, possibly because of my gcc and ld version.
(coma) ➜ coma git:(master) ✗ gcc --version
gcc (GCC) 9.1.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
(coma) ➜ coma git:(master) ✗ ld --version
GNU ld (GNU Binutils) 2.32
Copyright (C) 2019 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
The dirfd()
workaround for older versions does not work here (as it's d_fd
, rather than dd_fd
), and is in fact not needed, as Solaris 11 provides dirfd()
.
building '_posixsubprocess' extension
creating build/temp.solaris-2.11-sun4v.64bit-2.7
gcc -DNDEBUG -g -O3 -Wall -D_XPG6 -fPIC -I/datapool/dima/Sage/sagetrac-mirror/local/include/python2.7 -c _posixsubprocess.c -o build/temp.solaris-2.11-sun4v.64bit-2.7/_posixsubprocess.o
_posixsubprocess.c: In function ‘_close_open_fds_maybe_unsafe’:
_posixsubprocess.c:334:50: error: ‘DIR {aka struct <anonymous>}’ has no member named ‘dd_fd’; did you mean ‘d_fd’?
int fd_used_by_opendir = dirfd(proc_fd_dir);
^~~
d_fd
error: command 'gcc' failed with exit status 1
If I export CFLAGS
with CFLAGS="$CFLAGS -DHAVE_DIRFD"
then the build works.
I am not sure what the correct way to fix this is. HAVE_DIRFD
needs to be set somehow, I guess.
e.g. custom Timeout exceptions thrown when entire jobs timeout.
Consider subprocess32's implementation vs that for the current Python.
Timeout parameter doesn't really force timeout. It's happening on both Python 3.5 and Python 2.7 + subprocess32. However it works in Python 3.9.5. Not sure if it's worth to fix it, but at least it will be documented here. :)
import os
import sys
import unittest
from datetime import datetime, timedelta
if os.name == 'posix' and sys.version_info[0] < 3:
import subprocess32 as subprocess
else:
import subprocess
class TestSubprocess(unittest.TestCase):
def test_long_sleep(self):
start = datetime.now()
try:
subprocess.run(
"sleep 10",
timeout=1,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
except Exception as exc:
print("We got error:", type(exc))
delta = datetime.now() - start
print("It took", delta.seconds, "seconds")
self.assertLess(delta, timedelta(seconds=2))
if __name__ == '__main__':
unittest.main()
Python 3.5.10:
We got error: <class 'subprocess.TimeoutExpired'>
It took 10 seconds
F
======================================================================
FAIL: test_long_sleep (__main__.TestSubprocess)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subprocess.py", line 30, in test_long_sleep
self.assertLess(delta, timedelta(seconds=2))
AssertionError: datetime.timedelta(0, 10, 2203) not less than datetime.timedelta(0, 2)
----------------------------------------------------------------------
Ran 1 test in 10.003s
FAILED (failures=1)
Python 2.7.18
('We got error:', <class 'subprocess32.TimeoutExpired'>)
('It took', 10, 'seconds')
F
======================================================================
FAIL: test_long_sleep (__main__.TestSubprocess)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subprocess.py", line 30, in test_long_sleep
self.assertLess(delta, timedelta(seconds=2))
AssertionError: datetime.timedelta(0, 10, 2108) not less than datetime.timedelta(0, 2)
----------------------------------------------------------------------
Ran 1 test in 10.003s
FAILED (failures=1)
Python 3.9.5
We got error: <class 'subprocess.TimeoutExpired'>
It took 1 seconds
.
----------------------------------------------------------------------
Ran 1 test in 1.002s
OK
subprocess32 3.5.0rc1 was released in October 2017.
Now is April 2018.
When do you plan to release the next version of subprocess32?
Hi,
I am trying to execute a process from python, specifically CASA NRAO 5.7 which is a Python 2.7.12 environment plus some radio astronomical functions. The thing is that when I run my process using subprocess whether in just Python 2.7.12 or CASA I get a std::bad_alloc error, and that does not happen in Python 3.6.9. I have installed subprocess32 to see if this behaviour continued in Python 2.7 and yes, it continues. I don't know what change could be done to the subprocess library that make my program work in Python 3.6.9.
I am basically running this code:
import numpy as np
import os
if os.name == 'posix' and sys.version_info[0] < 3:
import subprocess32 as subprocess
else:
import subprocess
import shlex
args = "/home/miguel/Documents/gpuvmem/bin/gpuvmem -X 16 -Y 16 -V 256 -i A1314.ms.selfcal -o selfcal_data/output_residuals.ms -z 0.001,3.5 -Z 0.008,0.0,0.0 -G 1 -m output_ph3.image.fits -O selfcal_data/output.fits -I input_example.dat -R 2.0 -t 100000 -g 20 --verbose --savemodel-input"
args = shlex.split(args)
p2 = subprocess.Popen(args, bufsize=-1, shell=False)
p2.wait()
Best regards
OS:RHEL 5.11
PYTHON:2.7.11/12
When i use pip or python install to install this extension:
pip install subprocess32-3.2.7.tar.gz
The error messages:
Processing ./subprocess32-3.2.7.tar.gz
Installing collected packages: subprocess32
Running setup.py install for subprocess32 ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-ekkMER-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-EebZzY-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying subprocess32.py -> build/lib.linux-x86_64-2.7
running build_ext
building '_posixsubprocess' extension
creating build/temp.linux-x86_64-2.7
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/python-2.7/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o
_posixsubprocess.c: In function 'subprocess_cloexec_pipe':
_posixsubprocess.c:764: warning: implicit declaration of function 'pipe2'
_posixsubprocess.c:764: error: 'O_CLOEXEC' undeclared (first use in this function)
_posixsubprocess.c:764: error: (Each undeclared identifier is reported only once
_posixsubprocess.c:764: error: for each function it appears in.)
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-ekkMER-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-EebZzY-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-ekkMER-build/
This happens with subprocess32==3.5.0
:
>>> import subprocess32 as s32
>>> exc = s32.CalledProcessError(-1, "eggs")
>>> raise exc
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
subprocess32.CalledProcessError: <exception str() failed>
>>> str(exc)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/jwilk/.local/lib/python2.7/site-packages/subprocess32.py", line 72, in __str__
self.cmd, signal.Signals(-self.returncode))
AttributeError: 'module' object has no attribute 'Signals'
Aside from bugginess, I wonder why this code is there in the first place.
subprocess uses signal.Signals
only since Python 3.6, whereas subprocess32 is supposed to be 3.2 with some 3.3 and 3.5 backports.
if pass_fds and not close_fds:
warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
Results in NameError: global name 'warnings' is not defined
Hi,
First, thank you for this port.
I use subprocess32 quite alot, and recently discovered that Python 3.5 actually has a new function named run() which is somewhat more advanced. Would it be possible to add it to subprocess32?
Run into an issue, see below, when using check_output and timeout. This is on Windows 10, and understand that "This code has not been tested on Windows or other non-POSIX platforms.", but thought you might want to know.
C:\Users\djea>python -V
Python 2.7.10
C:\Users\djea>pip show subprocess32
---
Metadata-Version: 1.1
Name: subprocess32
Version: 3.2.7
Summary: A backport of the subprocess module from Python 3.2/3.3 for use on 2.x.
Home-page: https://github.com/google/python-subprocess32
Author: Gregory P. Smith
Author-email: [email protected]
License: PSF license
Location: c:\python27\lib\site-packages
Requires:
When running this code,
import subprocess32
try:
p = subprocess32.check_output("timeout 5",shell=True,timeout=3)
print p
except subprocess32.TimeoutExpired:
print "expired"
Will get this Error
Traceback (most recent call last):
File "C:\Users\djea\timeout.py", line 5, in <module>
p = subprocess32.check_output("timeout 5",shell=True,timeout=3)
File "C:\Python27\lib\site-packages\subprocess32.py", line 631, in check_output
output, unused_err = process.communicate(timeout=timeout)
File "C:\Python27\lib\site-packages\subprocess32.py", line 927, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "C:\Python27\lib\site-packages\subprocess32.py", line 1190, in _communicate
raise TimeoutExpired(self.args)
TypeError: __init__() takes at least 3 arguments (2 given)
Same code (except using 'sleep' instead of 'timeout') has no problem on Linux.
Subprocess32 is a dependency for the fbprophet library.
While pip installing the fbprophet module I received the following error:
Command "/home/ubuntu/anaconda3/envs/resfuture/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-DWKA4l/subprocess32/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-zcj5OE-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-DWKA4l/subprocess32/
When I tried to pip install just subprocess I encountered the same error:
Failed building wheel for subprocess32
Command "/home/ubuntu/anaconda3/envs/resfuture/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-paHlZS/subprocess32/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-mEbekx-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-paHlZS/subprocess32/
Does anyone know how I can resolve this issue?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.