pexpect / ptyprocess Goto Github PK
View Code? Open in Web Editor NEWRun a subprocess in a pseudo terminal
Home Page: https://ptyprocess.readthedocs.io/en/latest/
License: Other
Run a subprocess in a pseudo terminal
Home Page: https://ptyprocess.readthedocs.io/en/latest/
License: Other
I'm running ptyprocess in read in a greenlet coroutine. so when I .read() and there's no data, it blocks and stops all the other greenlets. is it possible to read without blocking, or at least check if there's any new data?
Sorry it this is a silly question
Thank you
Is it possible?
I'd like to spawn a process that reads stdin.
cat requirements.txt | ./script_that_spawns.py safety --check --stdin
When doing so and trying to read safety
output, it blocks, and I have to interrupt it with control-c:
Traceback (most recent call last):
File "/home/pawamoy/.cache/pypoetry/virtualenvs/mkdocstrings-ytlBmpdO-py3.8/bin/failprint", line 8, in <module>
sys.exit(main())
File "/home/pawamoy/.cache/pypoetry/virtualenvs/mkdocstrings-ytlBmpdO-py3.8/lib/python3.8/site-packages/failprint/cli.py", line 125, in main
return run(
File "/home/pawamoy/.cache/pypoetry/virtualenvs/mkdocstrings-ytlBmpdO-py3.8/lib/python3.8/site-packages/failprint/cli.py", line 54, in run
output.append(process.read())
File "/home/pawamoy/.cache/pypoetry/virtualenvs/mkdocstrings-ytlBmpdO-py3.8/lib/python3.8/site-packages/ptyprocess/ptyprocess.py", line 818, in read
b = super(PtyProcessUnicode, self).read(size)
File "/home/pawamoy/.cache/pypoetry/virtualenvs/mkdocstrings-ytlBmpdO-py3.8/lib/python3.8/site-packages/ptyprocess/ptyprocess.py", line 516, in read
s = self.fileobj.read1(size)
KeyboardInterrupt
Here is the actual Python code I'm using:
process = PtyProcessUnicode.spawn(cmd)
output = []
while True:
try:
output.append(process.read())
except EOFError:
break
process.close()
This is somewhat related to #43.
Python 3.7.1 contains the change https://bugs.python.org/issue6721 that is trying to avoid a deadlock in the logging code. It works by having os.fork() grab all the logging locks before executing fork(). It's expensive - if the forked process is just going to exec then os.spawn() is preferred. Its also creating, lets say, interesting problems (deadlocks) with code that was working.
Since PtyProcess calls pty.fork() (which calls os.fork() ...) it triggering this code path.
Using some equivalent of os.spawn() would eliminate this.
Recently I made a Web terminal with ptyprocess, websocket, and XTerm, but eventually found that the Web terminal size could not adapt after the resize event was triggered.
I even thought my computer was actually a big phone until I switched the positions of two parameters and the page appeared normal.
Before:
def _setwinsize(fd, rows, cols):
# Some very old platforms have a bug that causes the value for
# termios.TIOCSWINSZ to be truncated. There was a hack here to work
# around this, but it caused problems with newer platforms so has been
# removed. For details see https://github.com/pexpect/pexpect/issues/39
TIOCSWINSZ = getattr(termios, 'TIOCSWINSZ', -2146929561)
# Note, assume ws_xpixel and ws_ypixel are zero.
s = struct.pack('HHHH', rows, cols, 0, 0)
fcntl.ioctl(fd, TIOCSWINSZ, s)
After using the following modified code, the interface display is normal:
def _setwinsize(fd, rows, cols):
# Some very old platforms have a bug that causes the value for
# termios.TIOCSWINSZ to be truncated. There was a hack here to work
# around this, but it caused problems with newer platforms so has been
# removed. For details see https://github.com/pexpect/pexpect/issues/39
width, height = cols, rows
TIOCSWINSZ = getattr(termios, 'TIOCSWINSZ', -2146929561)
# Note, assume ws_xpixel and ws_ypixel are zero.
s = struct.pack('HHHH', width, height, 0, 0)
fcntl.ioctl(fd, TIOCSWINSZ, s)
I see that pr #49 got merged, but no release was ever made, so this isnt available on pypi. Could you create a release for this and update pypi so that (among other things) pexpect/pexpect#546 can be completed and made available?
ExceptionPexpect is not defined.
ptyprocess/ptyprocess/_fork_pty.py
Line 66 in 8f1ba73
Found using cython and
https://gist.github.com/tzickel/ea4541b12c1298a8092c3ec5043ef77e
pip install -I --root temp_ptyprocess ptyprocess
python checkcython.py temp_ptyprocess
Running the tests on FreeBSD 11 I see:
% py.test
============================= test session starts ==============================
platform freebsd11 -- Python 2.7.9 -- py-1.4.26 -- pytest-2.6.4
collected 5 items
tests/test_invalid_binary.py .
tests/test_preexec_fn.py ..
tests/test_spawn.py FF
=================================== FAILURES ===================================
__________________________ PtyTestCase.test_spawn_sh ___________________________
self = <tests.test_spawn.PtyTestCase testMethod=test_spawn_sh>
def test_spawn_sh(self):
env = os.environ.copy()
env['FOO'] = 'rebar'
p = PtyProcess.spawn(['sh'], env=env)
p.read()
p.write(b'echo $FOO\n')
time.sleep(0.1)
response = p.read()
> assert b'rebar' in response
E AssertionError: assert 'rebar' in 'echo $FOO\r\n'
tests/test_spawn.py:15: AssertionError
______________________ PtyTestCase.test_spawn_unicode_sh _______________________
self = <tests.test_spawn.PtyTestCase testMethod=test_spawn_unicode_sh>
def test_spawn_unicode_sh(self):
env = os.environ.copy()
env['FOO'] = 'rebar'
p = PtyProcessUnicode.spawn(['sh'], env=env)
p.read()
p.write(u'echo $FOO\n')
time.sleep(0.1)
response = p.read()
> assert u'rebar' in response
\n' AssertionError: assert 'rebar' in 'echo $FOO
tests/test_spawn.py:31: AssertionError
====================== 2 failed, 3 passed in 0.80 seconds ======================
The expected output is produced, it is just not returned by the second p.read()
in the test. Inserting a dummy p.read()
before response = p.read()
gets the tests passing.
Calling wait()
on a PtyProcess object whose process has already terminated will raise an exception. This behavior is unexpected[1].
I have made a small test to show case the behavior. Using p.isalive() && p.wait()
will only make the race condition worse in my tests.
[1]: subprocess.Popen.wait() returns normally even when the process has already terminated, although it doesn't say so explicitly in the documentation. In java the waitFor()
method "returns immediately if the subprocess has already terminated."
I just ran into #38, and can confirm installing from master fixes the issue. It would be nice to have a new release of ptyprocess
that includes that fix.
Calls to self.fileobj.readline() from the PtyProcess readline() method read data one byte at a time (most likely since fileobj is opened with 'buffering=0'.) Thus, this program:
from ptyprocess import PtyProcess, PtyProcessUnicode
p = PtyProcess.spawn(['perl', '-e', '''use 5.010; foreach my $letter ('a'..'z'){ say $letter x 1000; }'''])
while(1):
try:
print p.readline()
except EOFError:
break
p.close()
Has pretty poor performance (output from strace):
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
93.48 0.465020 18 26214 1 read
2.28 0.011353 23 489 381 open
0.84 0.004197 4197 1 clone
0.61 0.003037 19 160 113 stat
Is there a compelling reason to specify that the fileobj should have unbuffered IO?
The PtyProcess read() method does not experience this behavior because it uses a default buffer size of 1024.
The following code in ptyprocess
ptyprocess/ptyprocess/ptyprocess.py
Lines 260 to 269 in 3931cd4
is looping through all possible file descriptors in order to close those (note that closerange()
implemented as a loop at least on Linux). In case the limit of open fds (aka ulimit -n
, aka RLIMIT_NOFILE
, aka SC_OPEN_MAX
) is set too high (for example, with recent docker it is 1024*1024), this loop takes considerable time (as it results in about a million close()
syscalls).
The solution (at least for Linux and Darwin) is to obtain the list of actually opened fds, and only close those. This is implemented in subprocess
module in Python3, and there is a backport of it to Python2 called subprocess32.
This issue was originally reported to docker: docker/for-linux#502
Other good reason for using subprocess
(being multithread-safe) is described in #43
Hi :)
I would like to subprocess any given command and log its stdout and stderr separately. This would seem like an easy thing to do, but im having no luck because:
So in jumps ptys and ptyprocess to the resuce. Tie the stdout and stderr to a pty, you get 1 file path like /dev/tty0021, and isatty() returns true for both. Problem is - now we can't distinguish stdout from stderr.
Ok, no problem, just make two ptys - one for stdout and one for stderr. But now although both pass isatty(), their file paths will now look like /dev/tty0021 and /dev/tty0022 (for example).
The subprocess reacts as if you weren't using a pty in the first place, and you log nothing.
I have been trying for four months to figure a way around this, and because you are the expert in ptys I thought i might ask you directly - could you think of a way to log stdout and stderr separatly in your program, and still fool a script like this: https://bpaste.net/show/000d6f70ef41
THANK YOU :D
Getting https://pypi.python.org/packages/source/p/ptyprocess/ptyprocess-0.4.tar.gz to ./ptyprocess-0.4.tar.gz
$ tar tf ./ptyprocess-0.4.tar.gz
ptyprocess-0.4/
ptyprocess-0.4/ptyprocess/
ptyprocess-0.4/ptyprocess/util.py
ptyprocess-0.4/ptyprocess/ptyprocess.py
ptyprocess-0.4/ptyprocess/_fork_pty.py
ptyprocess-0.4/ptyprocess/__init__.py
ptyprocess-0.4/setup.py
ptyprocess-0.4/PKG-INFO
ptyprocess-0.4/tests/
ptyprocess-0.4/tests/test_preexec_fn.py
ptyprocess-0.4/tests/test_spawn.py
ptyprocess-0.4/tests/test_invalid_binary.py
ptyprocess-0.4/tests/__pycache__/
ptyprocess-0.4/tests/__pycache__/__init__.cpython-34.pyc
ptyprocess-0.4/tests/__pycache__/test_spawn.cpython-27-PYTEST.pyc
ptyprocess-0.4/tests/__pycache__/test_preexec_fn.cpython-34-PYTEST.pyc
ptyprocess-0.4/tests/__pycache__/test_preexec_fn.cpython-27-PYTEST.pyc
ptyprocess-0.4/tests/__pycache__/test_spawn.cpython-34-PYTEST.pyc
ptyprocess-0.4/tests/__pycache__/test_invalid_binary.cpython-27-PYTEST.pyc
ptyprocess-0.4/tests/__pycache__/test_invalid_binary.cpython-34-PYTEST.pyc
ptyprocess-0.4/tests/__init__.pyc
ptyprocess-0.4/tests/__init__.py
ptyprocess-0.4/README.rst
This results in build errors on EL7:
+ py.test
============================= test session starts ==============================
platform linux2 -- Python 2.7.5 -- pytest-2.3.5
collected 0 items / 3 errors
==================================== ERRORS ====================================
________________ ERROR collecting tests/test_invalid_binary.py _________________
import file mismatch:
imported module 'tests.test_invalid_binary' has this __file__ attribute:
/home/takluyver/Code/ptyprocess/tests/test_invalid_binary.py
which is not the same as the test file we want to collect:
/builddir/build/BUILD/ptyprocess-0.4/tests/test_invalid_binary.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
__________________ ERROR collecting tests/test_preexec_fn.py ___________________
import file mismatch:
imported module 'tests.test_preexec_fn' has this __file__ attribute:
/home/takluyver/Code/ptyprocess/tests/test_preexec_fn.py
which is not the same as the test file we want to collect:
/builddir/build/BUILD/ptyprocess-0.4/tests/test_preexec_fn.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________ ERROR collecting tests/test_spawn.py _____________________
import file mismatch:
imported module 'tests.test_spawn' has this __file__ attribute:
/home/takluyver/Code/ptyprocess/tests/test_spawn.py
which is not the same as the test file we want to collect:
/builddir/build/BUILD/ptyprocess-0.4/tests/test_spawn.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
=========================== 3 error in 0.04 seconds ============================
When I started testing ptyprocess with Python 3.11 I found out that test_spawn_sh
fails with:
E AssertionError: assert b'echo $ENV_KEY; exit 0' in b'\rjkulik@kuldavm-sparc-6717:...process/ptyprocess-0.7.0/tests$ echo $ENV \x08_KEY; exit 0\r\nenv_value\r\n'
E + where b'echo $ENV_KEY; exit 0' = <built-in method strip of bytes object at 0x1fffc6b2a9f8b0>()
E + where <built-in method strip of bytes object at 0x1fffc6b2a9f8b0> = b'echo $ENV_KEY; exit 0\n'.strip
I did some testing and it seems that this is caused by some kind of a race condition / it's speed dependent. When I run the test suite for the second time, it passes. But then when I delete the __pycache__
directory, it fails again.
I also found out that when I add a sleep after the spawn, it fails every time in every Python (and not just this test, test_spawn_sh_unicode
as well):
--- ptyprocess-0.7.0/tests/test_spawn.py
+++ ptyprocess-0.7.0/tests/test_spawn.py
@@ -21,6 +21,7 @@ class PtyTestCase(unittest.TestCase):
def _spawn_sh(self, ptyp, cmd, outp, env_value):
# given,
p = ptyp.spawn(['sh'], env=self.env)
+ time.sleep(1)
p.write(cmd)
# exercise,
The test is likely at fault. It is attempting to see its own keyboard input in output, but the prompt is displaying late, causing the familiar "input echo intermittently spanned in output" condition:
AssertionError: assert 'echo $ENV_KEY; exit 0' in 'echo $EN(teamcity-pexpect)(teamcity-pexpect)V_KEY; exit 0\r\nenv_value\r\n'
Full output and link
self = <tests.test_spawn.PtyTestCase testMethod=test_spawn_sh_unicode>
ptyp = <class 'ptyprocess.ptyprocess.PtyProcessUnicode'>
cmd = 'echo $ENV_KEY; exit 0\n'
outp = 'echo $EN(teamcity-pexpect)(teamcity-pexpect)V_KEY; exit 0\r\nenv_value\r\n'
env_value = 'env_value'
def _spawn_sh(self, ptyp, cmd, outp, env_value):
# given,
p = ptyp.spawn(['sh'], env=self.env)
p.write(cmd)
# exercise,
while True:
try:
outp += p.read()
except EOFError:
break
# verify, input is echo to output
> assert cmd.strip() in outp
E AssertionError: assert 'echo $ENV_KEY; exit 0' in 'echo $EN(teamcity-pexpect)(teamcity-pexpect)V_KEY; exit 0\r\nenv_value\r\n'
E + where 'echo $ENV_KEY; exit 0' = <built-in method strip of str object at 0x7f2231b5f540>()
E + where <built-in method strip of str object at 0x7f2231b5f540> = 'echo $ENV_KEY; exit 0\n'.strip
There seems to be some kind of race condition in the testsuite on both python-3.4 and python-2.7 here on Fedora 21 (it seems to occour more often in python-3.4 though).
This is the log of the tests:
self = <tests.test_spawn.PtyTestCase testMethod=test_spawn_sh>
def test_spawn_sh(self):
env = os.environ.copy()
env['FOO'] = 'rebar'
p = PtyProcess.spawn(['sh'], env=env)
p.read()
p.write(b'echo $FOO\n')
time.sleep(0.1)
response = p.read()
assert b'rebar' in response
p.sendeof()
p.read()
with self.assertRaises(EOFError):
> p.read()
E AssertionError: EOFError not raised
tests/test_spawn.py:21: AssertionError
What could cause these random tests? How can I help debugging it?
Got a FreeBSD (digital ocean droplet, freebsd.pexpect.org
) build agent prepared. It raises exception very early in critical codepath causing test runner to fork and eventually the build agent is killed by the kernel due to an OOM condition.
Error is in method pty_make_controlling_tty
at:
# Verify we now have a controlling tty.
fd = os.open("/dev/tty", os.O_WRONLY)
[freebsd@freebsd ~]$ sudo -u teamcity -s
$ cd /opt/TeamCity/work/210ae16cc3f30c30/ptyprocess
$ . `which virtualenvwrapper.sh`
$ mkvirtualenv pexpect27 --python=`which python2.7`
$ pip install -e .
$ cd ../pexpect
$ python
Python 2.7.9 (default, Jan 8 2015, 21:47:19)
[GCC 4.2.1 Compatible FreeBSD Clang 3.3 (tags/RELEASE_33/final 183502)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import pexpect
>>> bash = pexpect.spawn('/bin/bash')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pexpect/pty_spawn.py", line 189, in __init__
self._spawn(command, args, preexec_fn)
File "pexpect/pty_spawn.py", line 281, in _spawn
cwd=self.cwd, **kwargs)
File "/opt/TeamCity/work/210ae16cc3f30c30/ptyprocess/ptyprocess/ptyprocess.py", line 220, in spawn
pid, fd = _fork_pty.fork_pty()
File "/opt/TeamCity/work/210ae16cc3f30c30/ptyprocess/ptyprocess/_fork_pty.py", line 30, in fork_pty
pty_make_controlling_tty(child_fd)
File "/opt/TeamCity/work/210ae16cc3f30c30/ptyprocess/ptyprocess/_fork_pty.py", line 76, in pty_make_controlling_tty
fd = os.open("/dev/tty", os.O_WRONLY)
OSError: [Errno 6] Device not configured: '/dev/tty'
/dev/tty
may be opened under normal conditions.
pexpect is failing in ptyprocess.spawn on OpenIndiana 151a9 (a distro of Illumos/OpenSolaris). pexpect is 4.1.0, ptyprocess is 0.5.1:
Traceback (most recent call last):
File "/home/majid/bin/upload_file.py", line 26, in <module>
upload(fn)
File "/home/majid/bin/upload_file.py", line 7, in upload
timeout=14400)
File "/usr/local/lib/python2.7/site-packages/pexpect/pty_spawn.py", line 198, in __init__
self._spawn(command, args, preexec_fn, dimensions)
File "/usr/local/lib/python2.7/site-packages/pexpect/pty_spawn.py", line 298, in _spawn
cwd=self.cwd, **kwargs)
File "/usr/local/lib/python2.7/site-packages/pexpect/pty_spawn.py", line 309, in _spawnpty
return ptyprocess.PtyProcess.spawn(args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/ptyprocess/ptyprocess.py", line 223, in spawn
pid, fd = _fork_pty.fork_pty()
File "/usr/local/lib/python2.7/site-packages/ptyprocess/_fork_pty.py", line 30, in fork_pty
pty_make_controlling_tty(child_fd)
File "/usr/local/lib/python2.7/site-packages/ptyprocess/_fork_pty.py", line 76, in pty_make_controlling_tty
fd = os.open("/dev/tty", os.O_WRONLY)
OSError: [Errno 6] No such device or address: '/dev/tty'
The script is run under the control of at(1)
, and the job itself is queued from a crontab. It looks like what is happening is that when the parent process, which has no controlling terminal, calls os.openpty(), it acquires the pty as a controlling terminal. The problem seems to occur when posix_openpty pushes the ptem STREAMS module onto the slave pty fd, the simple call ioctl(slave_fd, I_PUSH, "ptem")
causes the parent process to acquire the pty as its controlling terminal, and thus the child, which is in a different session, is unable to acquire it and the failure at line 76 in _fork_pty().
I diffed posixmodule.c on Python 3.5 vs 2.7.11, the implementation of posix_openpty is not much different.
Older versions of pexpect had a different implementation _svr4_openpty(), but those look very similar to the Python posixmodule.c code.
Hello! Thank you for awesome work here!
I just want to let you know about xonsh shell project and that your awesome knowledge can help this project very high.
During my effort around understanding and fixing cases around running interactive tools I diven into PTY I found ptyprocess.
If you have time and chance to take a look into how xonsh works with terminal it will be very helpful to this pure open source crowd development project.
Thanks!
⬇️ Please click the 👍 reaction instead of leaving a +1
or 👍 comment
The attached patch is an experiment in using Python 3.8's os.posix_spawn() in ptyprocess. Since it doesn't use fork() it eliminates all those problems. A simple test using pexpect.interact() seemed to work.
I know it has a race with inheritable when called in parallel; and I'm sure there's more.
ptyprocess-posix-spawn.patch.gz
The change looks bigger than it is because I indented all the old code. Below is what matters, which I've included so it is easier to pick it apart....
if hasattr(os, 'posix_spawn'):
print("using posix_spawn")
# Issue 36603: Use os.openpty() (and try to avoid the
# whole pty module) as that guarentees inheritable (if it
# ever fails then just file a bug against os.openpty()
fd, tty = os.openpty()
# Try to set window size on TTY per below; but is this
# needed?
try:
_setwinsize(tty, *dimensions)
except IOError as err:
if err.args[0] not in (errno.EINVAL, errno.ENOTTY):
raise
# Try to disable echo if spawn argument echo was unset per
# below; but does this work?
if not echo:
try:
_setecho(tty, False)
except (IOError, termios.error) as err:
if err.args[0] not in (errno.EINVAL, errno.ENOTTY):
raise
# Create the child: convert the tty into STDIO; use the
# default ENV if needed; and try to make the child the
# session head using SETSID. Assume that all files have
# inheritable (close-on-exec) correctly set.
file_actions=[
(os.POSIX_SPAWN_DUP2, tty, STDIN_FILENO),
(os.POSIX_SPAWN_DUP2, tty, STDOUT_FILENO),
(os.POSIX_SPAWN_DUP2, tty, STDERR_FILENO),
(os.POSIX_SPAWN_CLOSE, tty),
(os.POSIX_SPAWN_CLOSE, fd),
]
spawn_env = env or os.environ
pid = os.posix_spawn(command, argv, spawn_env,
file_actions=file_actions,
setsid=True)
# Child started. Now close tty and stop PTY(FD) being
# inherited. Note that there's a race here: a parallel
# fork/exec would unwittingly inherit this PTY(FD)/TTY
# pair. Probably need to wrap all this in a lock?
os.close(tty)
os.set_inheritable(fd, False)
The 0.5.2 release doesn't appear in github's list of releases: perhaps you forgot to tag it?
My company has a ssh server, which would return QR code (by sending unicode and color escape), show in screenshot.
the intact output of ssh session is in the attachment ssh_output.log
However if i tried to use ptyprocess to read it , it just read the first 3 bytes and then hangs forever.
#!/usr/bin/env python
import ptyprocess
p = ptyprocess.PtyProcess.spawn(["/usr/bin/ssh", "[email protected]"])
s = p.read(1000)
print(s) # this line return b'\x1b[c'
s = p.read(1000) # this line blocks forever
Why would this happen and how can i solve it?
I can see what's happening in the real terminal with "OK" in the end of the response.
But the read method does not return the last two lines.
@staticmethod
def communicate(write_uuid, read_uuid, data):
# Requires a writable and a readable GATT uuid and data to be written
# The method returns the data retrieved from the readable GATT uuid
hex_string = ''
for char in data:
hex_string += hex(ord(char)) + ' '
timeout = .1
read_process = ptyprocess.PtyProcessUnicode.spawn(['/bin/bash'])
commands = ['bluetoothctl', 'menu gatt', f'select-attribute {read_uuid}', 'notify on']
for command in commands:
read_process.write(command + '\n')
time.sleep(timeout)
read_process.read(1)
write_process = ptyprocess.PtyProcessUnicode.spawn(['/bin/bash'])
commands = f'bluetoothctl <<EOF\nmenu gatt\nselect-attribute {write_uuid}\nwrite "{hex_string}"\nEOF\n'
write_process.write(commands)
time.sleep(timeout)
write_process.read(1)
time.sleep(timeout)
hex_values = []
pattern = re.compile(r'\b[0-9a-fA-F]{2}\b')
lines = read_process.read(size=10000).splitlines()
for line in lines:
print(line)
This is the response of the method:
[CHG] Attribute /org/bluez/hci0/dev_CE_EE_A8_9B_17_AC/service000b/char000e Value:
[Device:/service000b/char000e]#
30 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 0.0, 0.0, 0.0, 0
[Device:/service000b/char000e]#
2e 30 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 2e .0, 0.0, 0.0, 0.
[Device:/service000b/char000e]#
30 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 0, 0.0, 0.0, 0.0
[Device:/service000b/char000e]#
2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 2c , 0.0, 0.0, 0.0,
[Device:/service000b/char000e]�[
This should be the correct response:
[Device:/service000b/char000e]# [CHG] Attribute /org/bluez/hci0/dev_CE_EE_A8_9B_17_AC/service000b/char000e Value:
[Device:/service000b/char000e]# 30 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 0.0, 0.0, 0.0, 0
[Device:/service000b/char000e]# 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 2e .0, 0.0, 0.0, 0.
[Device:/service000b/char000e]# 30 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 0, 0.0, 0.0, 0.0
[Device:/service000b/char000e]# 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 2c , 0.0, 0.0, 0.0,
[Device:/service000b/char000e]# 20 30 2e 30 2c 20 30 2e 30 2c 20 30 2e 30 2c 20 0.0, 0.0, 0.0,
[Device:/service000b/char000e]# 30 2e 30 2c 20 30 2e 30 20 4f 4b 0a 0.0, 0.0 OK.
As you can see we have 18 values and an OK at the end of the correct response.
While we only have 13 values without OK in the read() method's response.
Also: The read method returns an ESC character for some reason.
The code that closes all the file descriptors in the child process before exec breaks practically any and all efforts to debug exec errors; especially if they're not OSErrors.
In my case, I was debugging under pycharm. pydevd monkeypatched all the exec routines and because of a behavioural change in pexpect, where it re-encoded all the arguments as utf-8 encoded binary, rather than strings, when pydevd tried to detect the presence of python in it's code, it crashed out because there was encoding of the strings and bstring.endswith() needs to be passed a b'' string, or else it bombs out.
There are still a number of time.sleep()
calls, particularly in close()
and terminate()
. This is bad for async code, because this will hog control when the event loop could be doing other things. I'd like to work out how to remove these. If that isn't possible, it should at least be marked very clearly which methods do this.
From pexpect/pexpect#79
Just the same as the old pure Python subprocess
module from Python 2, the PtyProcess.spawn
method is not async signal safe. The entire fork()
to exec
code path must be written in async signal safe C code. Both pty.fork
and your own PtyProcess.spawn
method violate this constraint.
See the Python 3 subprocess
module's _posixsubprocess
extension module (and the https://pypi.python.org/pypi/subprocess32/ backport for Python 2 users).
Likely symptoms when this bites you: random seeming deadlocks and hangs.
Workaround for users today: Do not use pexpect.spawn
or pexpect.pty_spawn
in a multithreaded application.
A pexpect.popen_spawn
module appears to exist and will use the subprocess
module. If you are on 2.7 make sure you've replaced sys.modules['subprocess'] with a reference to the subprocess32
module first (if you haven't already done so within your Python 2.7 interpreter's stdlib)
The original implementation of isalive uses os.waitpid to check if the ptyprocess is alive
os.waitpid can only check the process status if that process is a child of the current process
If the pexpect object is passed from process A to process B
process B can access the pexpect object but it cannot use it because os.waitpid can't find the ptyprocess under the child list of process B
it will throw the no such process error
while the process actually exists and is still running
To solve this
It can use os.kill to send signal 0 to check if process is alive
which achieve the same functionality of os.waitpid but not limit it to child process
Is it possible to add this as an extra check for isalive()?
Hello,
I was wondering if we could make ptyprocess 0.5.1 available for [legacy] easy-install installations
Thanks
Carlos
I had tried to install the ptyprocess package from "whl" file and got an error:
$ ./bin/wheel install -d ~/wheels/ ptyprocess
Traceback (most recent call last):
File "./bin/wheel", line 11, in <module>
sys.exit(main())
File "/home/cykooz/testvenv/local/lib/python2.7/site-packages/wheel/tool/__init__.py", line 356, in main
args.func(args)
File "/home/cykooz/testvenv/local/lib/python2.7/site-packages/wheel/tool/__init__.py", line 299, in install_f
args.wheel_dirs, args.force, args.list_files)
File "/home/cykooz/testvenv/local/lib/python2.7/site-packages/wheel/tool/__init__.py", line 221, in install
wf.install(force=force)
File "/home/cykooz/testvenv/local/lib/python2.7/site-packages/wheel/install.py", line 268, in install
if self.parsed_wheel_info['Root-Is-Purelib'] == 'true':
File "/home/cykooz/testvenv/local/lib/python2.7/site-packages/wheel/decorator.py", line 17, in __get__
val = self.wrapped(inst)
File "/home/cykooz/testvenv/local/lib/python2.7/site-packages/wheel/install.py", line 238, in parsed_wheel_info
return read_pkg_info_bytes(self.zipfile.read(self.wheelinfo_name))
File "/usr/lib/python2.7/zipfile.py", line 935, in read
return self.open(name, "r", pwd).read()
File "/usr/lib/python2.7/zipfile.py", line 630, in read
data = self.read1(n)
File "/usr/lib/python2.7/zipfile.py", line 692, in read1
self._update_crc(data, eof=eof)
File "/home/cykooz/testvenv/local/lib/python2.7/site-packages/wheel/install.py", line 463, in _update_crc
raise BadWheelFile("Bad hash for file %r" % ef.name)
wheel.install.BadWheelFile: Bad hash for file 'ptyprocess-0.5.1.dist-info/WHEEL'
This is because of hashes of files (saved in file ptyprocess-0.5.1-py2.py3-none-any.whl/ptyprocess-0.5.1.dist-info/RECORD) stored in invalid format. Instead of using BASE64-encoded SHA256-digest your hashes stored as HEX-string from SHA256 digest.
Please, rebuild WHL distributive of your package and put it into pypi.python.org.
Currently, the preexec_fn
is executed after the file descriptors are closed. This has some unwanted effects:
preexec_fn
opens a file descriptor, it will be inherit by the child processpreexec_fn
relays on having some file descriptor open, it will crash (see pexpect/pexpect#368)The proposal is to move the "close the fds" section below the "execute the preexec_fn" code: https://github.com/pexpect/ptyprocess/blob/master/ptyprocess/ptyprocess.py#L266-L285
For reference, this is what it does subprocess.Popen
: https://github.com/python/cpython/blob/master/Modules/_posixsubprocess.c#L528-L549
If it is okay, I can do a PR with the fix but I would like to hear your opinions about this.
Is there any reason the ws_xpixel
and ws_ypixel
fields of the TIOCSWINSZ call are always 0? Naively, shouldn't it be possible to replace
def _setwinsize(fd, rows, cols):
# Some very old platforms have a bug that causes the value for
# termios.TIOCSWINSZ to be truncated. There was a hack here to work
# around this, but it caused problems with newer platforms so has been
# removed. For details see https://github.com/pexpect/pexpect/issues/39
TIOCSWINSZ = getattr(termios, 'TIOCSWINSZ', -2146929561)
# Note, assume ws_xpixel and ws_ypixel are zero.
s = struct.pack('HHHH', rows, cols, 0, 0)
fcntl.ioctl(fd, TIOCSWINSZ, s)
with
def _setwinsize(fd, rows, cols, pixel_height=0, pixel_width=0):
# Some very old platforms have a bug that causes the value for
# termios.TIOCSWINSZ to be truncated. There was a hack here to work
# around this, but it caused problems with newer platforms so has been
# removed. For details see https://github.com/pexpect/pexpect/issues/39
TIOCSWINSZ = getattr(termios, 'TIOCSWINSZ', -2146929561)
# Note, assume ws_xpixel and ws_ypixel default to zero.
s = struct.pack('HHHH', rows, cols, pixel_width, pixel_height)
fcntl.ioctl(fd, TIOCSWINSZ, s)
I ask because of a long chain of trying to figure out why I couldn't get some things to display correctly in a Poetry shell, which uses pexpect (which uses this). On at least my kernel (Linux 6.10), the terminal is responsive to the values supplied in those fields of the struct. Ideally, it would be nice for pexpect to be able to forward size changes to its children beyond just row and column counts.
I noticed that an old issue mentions portability problems on FreeBSD (pexpect/pexpect#39), but that involved the ioctl
call having a different value. Perhaps someone better informed than me can confirm that implementing this breaks things on other systems.
This is a very severe bug. When calling Ptyprocess.read() the value returned is different almost every time:
ptyprocess.PtyProcess.spawn(['openssl', "ec", '-noout', '-text', '-in', '/opt/key/s128r1.key']).read()
And again with the same params:
I don't know what is causing this but this is very weird.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.