ksh93 / ksh Goto Github PK
View Code? Open in Web Editor NEWThis project forked from att/ast
ksh 93u+m: KornShell lives! | Latest release: https://github.com/ksh93/ksh/releases
License: Eclipse Public License 2.0
This project forked from att/ast
ksh 93u+m: KornShell lives! | Latest release: https://github.com/ksh93/ksh/releases
License: Eclipse Public License 2.0
See prior discussion:
att#375
#45 (comment)
TODO: after fixing this bug, re-enable regression test disabled in 1ca9286.
It's not "broken", but what is the standard for ksh93/ksh?
[10:04 AM][ttys000][~/src/ksh]
[245] mbp13 $ /usr/local/bin/ksh --version
version sh (AT&T Research) 93u+m 2020-07-16
[10:05 AM][ttys000][~/src/ksh]
[246] mbp13 $ echo $KSH_VERSION
Version AJM 93u+m 2020-07-16
[10:05 AM][ttys000][~/src/ksh]
[247] mbp13 $ git log
commit 36f55f1f857f84c3ca88664b8050853e15350ade (HEAD -> master, origin/master, origin/HEAD)
Author: Martijn Dekker <[email protected]>
Date: Sun Jul 19 06:42:53 2020 +0100
Huh?
After fixing #79 there is another regression that shows up when ksh is compiled with SHOPT_SPAWN
to use sh_ntfork()
which uses AST spawnveg(3)
which uses posix_spawn(3)
. A minimal reproducer for the regression involves a combination of command substitution, a here-document, and an external command:
output=$(
cat <<EOF
$(ls /dev/null)
EOF
)
echo "$output"
Actual output: (empty)
Expected output: /dev/null
I can reproduce this regression on a 93u+ 2012-08-01 compiled to use SHOPT_SPAWN
, so 93u+m did not introduce this bug.
If ksh is compiled with -DSHOPT_SPAWN=0
the bug does not occur.
If the command substitution is forked, e.g. with ulimit -t unlimited
, the bug does not occur:
output=$(
ulimit -t unlimited
cat <<EOF
$(ls /dev/null)
EOF
)
echo "$output"
Output as expected: /dev/null
So this is a bug that is, in part, related to non-forking subshells. There must be a bug somewhere in sh_ntfork()
, or some function called directly or indirectly by it, that causes I/O not to be set up correctly.
Job control should be disabled in subshells as it makes no sense there; subshells are never interactive and job control only makes sense on interactive shells. However, for non-forking subshells, ksh currently only disables job control for command substitutions:
ksh/src/cmd/ksh93/sh/subshell.c
Lines 570 to 577 in f207cd5
ksh/src/cmd/ksh93/sh/subshell.c
Lines 629 to 636 in f207cd5
This causes "interesting" behaviour, e.g.:
$ (sleep 1 & :)
[1] 25185
This wrongly prints a job number.
$ sleep 10 &
$ (fg)
This waits for sleep
to end, but should complain about no job control instead. However:
$ (sleep 10 & fg)
[1] 25144
arch/darwin.i386-64/bin/ksh: fg: No job control
So that's inconsisistent.
Also:
$ (sleep 1 &)
$ [1] 25187
This wrongly prints a job number after the prompt, i.e. the job number is printed from a background process! So the main shell forks as well as the background process. That's a separate bug, probably a parser bug causing a double fork, akin to the one I fixed in 1026006.
On this FreeBSD version, the following regression test failure occurs:
$ uname -vm
FreeBSD 12.1-RELEASE-p6 GENERIC amd64
$ bin/shtests -p pty
[...]
pty.sh[445]: process/terminal group exercise: line 451: read timeout
On Debian 10 (Buster) as hosted by shell.xs4all.nl, this test also fails, slightly differently:
$ cat /etc/issue; uname -srm
Debian GNU/Linux 10 \n \l
Linux 4.9.0-9-amd64 x86_64
$ bin/shtests -p pty
[...]
pty.sh[445]: process/terminal group exercise: line 451: expected "Stopped", got ""
The test in question:
ksh/src/cmd/ksh93/tests/pty.sh
Lines 445 to 454 in f811482
Recent email correspondence with @kdudka has made me think that maybe, in making POSIX compliance front and centre to development here, I've been giving insufficient consideration to the fact that there is an installed base of set-and-forget ksh93 scripts still running, which may somehow depend on old non-compliant behaviour.
In principle, I'm still thinking that anyone who needs 100% bug compatibility should stick with exactly the version they are running. However, we can restore the documented or otherwise clearly intended non-compliant behaviour as part of an option that can be turned on and off.
Other shells such as bash, mksh and yash can do set -o posix
to enable POSIX compliance mode. It is worth considering adding a similar option to ksh93. ksh should probably default to POSIX mode when invoked as sh, and to "legacy" otherwise. Having the posix option off would restore behaviour such as what has been fixed in 6a49720 (edit: nah, that's just pure breakage, and completely undocumented), eeee77e (edit: done in c607c48), and maybe 36da314 (edit: nah, that was just a bug).
It would also keep the Bourne shell compatibility hack for a literal [ -t ]
or test -t
being interpreted as [ -t 1 ]
/test -t 1
. Korn hacked the parser so that only a literal [ -t ]
is changed and not something like x=-t; [ "$x" ]
. Still, that hack violates POSIX and should be deactivated in any new POSIX mode. (edit: done in 55f0f8c)
Let's think about this for a while before doing anything. We can talk about it in this issue's thread, and figure out a list of things that need to be switched by such an option.
This has happened repeatedly today:
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 ksh 0x000000010aa7a1f6 job_unpost + 49
1 ksh 0x000000010aa7a9d1 job_reap + 1639
2 libsystem_platform.dylib 0x00007fff7dbf1b5d _sigtramp + 29
3 ??? 0x000000010ab6cb08 stkdisc + 40
4 ksh 0x000000010aa6c0a9 ed_read + 313
5 ksh 0x000000010aa6c478 ed_getchar + 102
6 ksh 0x000000010aaab15f ed_emacsread + 659
7 ksh 0x000000010aa76045 slowread + 491
8 ksh 0x000000010aaf9314 sfrd + 1045
9 ksh 0x000000010aaf558a _sffilbuf + 589
10 ksh 0x000000010aaf9ad6 sfreserve + 655
11 ksh 0x000000010aa5b3af exfile + 1939
12 ksh 0x000000010aa5bb08 sh_main + 1110
13 libdyld.dylib 0x00007fff7da063d5 start + 1
While it ends up at job_unpost again, this wasn't trying to replicate either of the crashers. This was just typing bin/pscal
(I'm probably the only person who still uses pscal) at a prompt, and the shell crashed. No command substitution, no timeout reads, no backgrounding.
I've now backed up to 49ae483 to see if I can replicate it without any of the job control related patches.
Process substitution of the form >(list) is utterly broken:
echo ok > >(sed s/ok/good/)
Expected output: good
(as on bash and zsh)
Actual output:
ok
plus a newlineksh: ?: cannot create [Illegal byte sequence]
Before I spend hours searching for the cause, does anyone have any idea where to begin debugging this?
When declaring certain special variables readonly in a (virtual/nonforked) subshell, ksh fails to restore them when leaving that subshell, and errors out as if a shell assignment to these variables had failed.
$ ksh -c '(readonly RANDOM;echo one);echo two'
one
ksh: RANDOM: is read only
$ ksh -c '(readonly LINENO;echo one);echo two'
one
ksh: LINENO: is read only
As usual, a workaround is to fork the subshell using ulimit -t unlimited
. This must be done before doing the readonly
for the bug to go away, which suggests the readonly
command is erroneously affecting the main shell environment.
Does anyone care about ksh's bash compatibility mode?
It's not compiled in my default. The RELEASE file says it's incomplete and experimental.
bash is pretty ubiquitous. I think if people want to run bash scripts, they'll run bash.
Would anyone object to getting rid?
$ cat > bug.sh
(
sh -c 'echo DEBUG1 $PPID'
readonly PATH
sh -c 'echo DEBUG2 $PPID'
)
sh -c 'echo DEBUG3 $PPID'
$ ksh bug.sh
DEBUG1 7546
DEBUG2 7546
$ DEBUG3 7549
The DEBUG3 output is done in an erroneous background job, as evidenced by the fact that the process ID changed, and it was printed after returning to the shell prompt.
This means that, when the state of PATH
is restored while exiting the virtual subshell, it erroneously forks the main shell environment.
A little go-back-and-recompile trip through the git history reveals that this bug was introduced by commit 102868f - very probably by the change it made in name.c.
Sure enough, commenting out the following lines makes the bug go away (while of course reintroducing the corresponding hash table regression test failure in subshell.c):
Lines 1609 to 1614 in bd3e2a8
I would conclude that nv_putval()
is not a safe place from which to call sh_subfork()
as it is also used for restoring the state after leaving a virtual subshell. The forking should probably happen somewhere we can be sure that an assignment is being processed โ possibly in xec.c.
edit: In ab5dedd I've applied a workaround that disables the use of /dev/fd/NN pseudofiles and falls back to the older FIFO method instead. (That commit also improves the robustness of the FIFO method.) This effectively works around this bug, but is not a real fix. The /dev/fd method is the most robust as it doesn't involve the file system. So I'm leaving this issue open. To debug this, we now first need to comment out/remove the redefinition of SHOPT_DEVFD
to 0 in src/cmd/ksh93/include/defs.h
.
This was first reported on the old mailing list.
Reproducer based on the one in that message:
fdUser() {
cat "$1"
}
for ((i=1; i<10; i++))
do fdUser <(echo '========')
if [ -d /proc ]
then ls /proc/$$/fd
else lsof -p $$ | wc -l
fi
done
The output shows one more open file for each function call. This bug does not occur if you replace the function call with a built-in or external command.
The segfault has been fixed. Now the problem is that both types of functions, POSIX and ksh, silently fail to unset themselves. See b7932e8 and 9751174 and the issues referenced in them for more information.
[Original report] It should not be this easy to make ksh segfault:
$ ksh -c 'fn() { unset -f fn; echo end; }; fn'
Segmentation fault: 11
Line editing is set thus:
[44] mbp13 $ set -o
emacs on
vi off
viraw on
Start typing, get to bin/p, hit TAB. Nothing happens. Hit TAB again. (why do I have to hit TAB twice?) Get:
[04:59 PM][ttys001][~]
[45] mbp13 $ bin/p
1) pkg-config
2) pscal
3) pscal.saf
[45] mbp13 $ bin/pscal
Hit '2', hit TAB again, shell fills out bin/pscal.
[45] mbp13 $ bin/pscal
Now, backspace over the command line...
[04:59 PM][ttys001][~]
[45] mbp13 $ bin/p
1) pkg-config
2) pscal
3) pscal.saf
[45] mbp13 $ b
I'm left with the 'b', which isn't actually there, I can hit ENTER and not get an error message, and it's not stored in history.
10.15.5, ksh93u+m, ksh93u+ does it too.
This is not currently supported. See #47.
After applying #47, typeset -R10 -i foo=1; echo "[$foo]"
gives a usage error and something like typeset -R10 -E3 foo=12345; echo "[$foo]"
or typeset -R20 -F foo=12345; echo "[$foo]"
simply ignores the -R
flag.
Which precedent should we follow? Maybe they should all error out? I think giving a usage error is more logical since this combination of flags is not actually possible. But it should be preceded by an informative error message such as "incompatible options combination".
Or maybe we should just fix it so they can be combined? I see no real reason why this should be impossible. On mksh (which only supports integer arithmetic) the following works fine:
$ mksh -c 'typeset -R10 -i foo=123 bar=4567; printf "%s\n" "[$foo]" "[$bar]"'
[ 123]
[ 4567]
The other ksh-inspired shell that supports floating point arithmetic is zsh, and it has no problem combining any of these:
$ zsh -c 'typeset -R20 -F foo=12345; echo "[$foo]"'
[ 12345.0000000000]
$ zsh -c 'typeset -R20 -E3 foo=12345; echo "[$foo]"'
[ 1.23e+04]
$ zsh -c 'typeset -R20 -i foo=12345; echo "[$foo]"'
[ 12345]
Community opinions invited. Should this be supported, or should it be an error, or should the -L
/-R
be ignored for these?
Previous discussion and reproducer: att#416
A straightforward workaround/fix would be to fork when an associative array is declared in a subshell:
diff --git a/src/cmd/ksh93/bltins/typeset.c b/src/cmd/ksh93/bltins/typeset.c
index 906d0a7..0ae7d8c 100644
--- a/src/cmd/ksh93/bltins/typeset.c
+++ b/src/cmd/ksh93/bltins/typeset.c
@@ -259,6 +259,8 @@ int b_typeset(int argc,register char *argv[],Shbltin_t *context)
tdata.tname = opt_info.arg;
break;
case 'A':
+ if(tdata.sh->subshell && !tdata.sh->subshare)
+ sh_subfork();
flag |= NV_ARRAY;
break;
case 'C':
Before applying that fix/workaround I'd like to try to figure out if it's possible to implement a fix for the non-forking subshell mechanism. I don't really have time to dive deeply into this right now, soโฆ any ideas, anyone?
[32] iMac $ git pull
Updating f9d2893..c4236cc
Fast-forward
NEWS | 5 +++++
src/cmd/ksh93/include/version.h | 2 +-
src/cmd/ksh93/sh/parse.c | 2 +-
src/cmd/ksh93/tests/types.sh | 3 +++
4 files changed, 10 insertions(+), 2 deletions(-)
[ 9:59 AM][ttys000][~/src/ksh]
but...
[10:07 AM][ttys000][~/src/ksh]
[36] iMac $ bin/shtests leaks
#### Regression-testing /Users/mwilson/src/ksh/arch/darwin.i386-64/bin/ksh ####
test leaks begins at 2020-07-10+10:08:09
test leaks passed at 2020-07-10+10:08:09 [ 6 tests 0 errors ]
test leaks(C.UTF-8) begins at 2020-07-10+10:08:09
test leaks(C.UTF-8) passed at 2020-07-10+10:08:10 [ 6 tests 0 errors ]
test leaks(shcomp) begins at 2020-07-10+10:08:10
shcomp-leaks.ksh[102]: memory leak with read -C when using <<< (leaked 80 bytes)
test leaks(shcomp) failed at 2020-07-10+10:08:10 with exit code 1 [ 6 tests 1 error ]
Total errors: 1
CPU time user: system:
main: 0m00.00s 0m00.01s
tests: 0m00.27s 0m00.55s
[10:08 AM][ttys000][~/src/ksh]
[37] iMac $ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
On this system:
$ uname -vm
FreeBSD 12.1-RELEASE-p6 GENERIC amd64
The following regression test failure occurs:
pty.sh[291]: POSIX sh 101(C): line 323: expected "^backslash-erase=:\r?\n$", got EOF
The test in question:
ksh/src/cmd/ksh93/tests/pty.sh
Lines 291 to 333 in f811482
There is a severe memory allocation bug that was reported in att#1401. Too much memory is being allocated when ksh reads lines of increasing size from a file. In the following example ksh uses 12GB of RAM to read a 41MB file:
$ dd if=/dev/zero of=./zeroes count=4096 bs=10000
$ ksh -c 'read line < ./zeroes; grep VmPeak /proc/$$/status ; echo "."'
VmPeak: 12548348 kB
.
The cause is located in the vmalloc's memory allocation. This bug disappears when ksh is compiled with -D_std_malloc
added to CCFLAGS
because that flag causes vmalloc to become a wrapper for the operating system's malloc implementation (i.e., the native malloc
and free
functions are used instead of vmalloc's replacements):
# Run after deleting ./arch and recompiling ksh with `-D_std_malloc`
$ dd if=/dev/zero of=./zeroes-std count=4096 bs=10000
$ arch/*/bin/ksh -c 'read line < ./zeroes-std; grep VmPeak /proc/$$/status ; echo "."'
VmPeak: 88492 kB
.
For reference, ksh93v- and ksh2020 use the following amounts of memory with the reproducer:
# ksh93v- uses a more reasonable amount of memory, indicating vmalloc was (edit: partially) fixed after ksh93u+
$ ksh93v -c 'read line < ./zeroes; grep VmPeak /proc/$$/status ; echo "."'
VmPeak: 131572 kB
.
# ksh2020 uses standard malloc instead of vmalloc
$ ksh2020 -c 'read line < ./zeroes; grep VmPeak /proc/$$/status ; echo "."'
VmPeak: 88480 kB
.
There appears to be a spurious '=' file created at the top of the source tree after a standard build of 'bin/package make'.
$ bin/package make
...
$ ls
'=' LICENSE.md NEWS README.md TODO arch bin docs lib src
According to sh.1, tab completion is available in "each of the edit modes" (see under "In-line Editing Options").
However, there is often a broken form of tab completion when no edit mode is active. It is unpredictable whether this happens, which suggests a failure to initialise a variable somewhere.
A fix should make sure tab completion is properly deactivated when neither emacs nor vi mode is active.
#! /bin/ksh
LANG=C.UTF-8
printf '%q\n' 'Is shell-quoting garbled?'
print -nr $'\303\274' | read -n1 g
printf '%q\n' 'Is shell-quoting garbled?'
When running this, the second line is output without the final quote.
When you run this with set -x
, the issue happens to quoted strings in the xtrace as well, suggesting that the global shell-quoting algorithm is somehow affected.
This bug is also in 93u+ 2012-08-01, as well as ksh2020.
Reproducer:
tmp=/tmp/LINENO_bug.$$
trap 'rm -rf "$tmp"' EXIT
mkdir "$tmp" || exit
cd "$tmp" || exit
print $'foobar() {\n: two\n: three\n: four\n} #five' > $tmp/foobar
FPATH=$tmp
saveLINENO=$LINENO
foobar
diff=$((LINENO - saveLINENO))
if ((diff != 2))
then echo "bug: diff == $diff"
exit 1
else echo "ok"
fi
Output:
bug: diff == 7
It appears that the five lines from the function definition file are wrongly added to the main script's $LINENO
. So the fix probably involves saving and restoring $LINENO
upon autoloading a function.
Scalar arrays (-a
) and associative arrays (-A
) of a type created by enum
allow values not specified by the enum type, yielding corrupted results.
Reproducer:
$ enum Color_t=(red green blue orange yellow)
$ Color_t -a clr=(red blue WRONG)
$ printf '%s\n' "${clr[@]}"
red
blue
red
$ Color_t -A clr=([foo]=red [bar]=blue [bad]=WRONG)
$ printf '%s\n' "${clr[@]}"
1440
1728
2016
(the numbers are random, which is evidence of memory corruption)
Expected in both cases: an error message refusing to create the array due to an invalid value, as in this scalar asssignment:
$ Color_t clr=BAD
ksh: clr: invalid value BAD
This can also be reproduced in 93u+ and in ksh2020, so it doesn't look like there's an existing fix to backport.
Running return || true
or return || false
in a function causes the function to return with exit status zero instead of one:
# This bug is also in ksh93n-, ksh93u+, ksh93v- and ksh2020
$ cat ./reproducer.sh
foo() {
false
return || true
}
foo
if [ $? = 0 ]; then
echo 'Failure'
exit 1
fi
exit 0
$ ksh ./reproducer.sh
Failure
$ dash ./reproducer.sh # No output when using other shells
$
I found this problem while debugging BUG_LOOPRET2, which is similar (although that bug only applies to loops, while the above reproducer doesn't use a loop):
Filename expansion using bare asterisk expands . and .. as well.
$ print *
. .. bar foo
The behavior here should be that it excludes the files with leading periods unless the period is specified in the pattern as well.
Ref: 2.13.3.2 Patterns Used for Filename Expansion
Reproduced on: 93u+m 2020-07-02
E.g. on macOS, as well as FreeBSD:
$ LANG=nl_NL.UTF-8 TZ=Europe/Helsinki arch/*/bin/ksh -c 'date; printf %T\\n now'
di 30 jun 2020 03:49:12 EEST
di 30 jun 03:49:12 2020
(the time zone is missing from printf %T
output and the year is printed in the wrong order)
And on Linux (Debian):
$ LANG=nl_NL.UTF-8 TZ=Europe/Helsinki arch/*/bin/ksh -c 'date; printf %T\\n now'
di 30 jun 2020 3:50:40 EEST
di 30 jun 2020 di jun 30 EEST 2020:50:40 EEST
That different systems produce differently corrupted output is interesting.
On FreeBSD I get the following regression test failure (among others):
test builtins begins at 2020-06-12+22:04:08
builtins.sh[315]: printf "%T" now
This test compares the output of printf "%T" now
with that of the external command date
and checks if they are identical. On my cloud server and in my (Dutch) locale and time zone, they produce:
$ date
vr 12 jun. 2020 22:12:37 CEST
$ LC_ALL=C date
Fri Jun 12 22:13:20 CEST 2020
$ arch/freebsd12.amd64-64/bin/ksh -c 'printf %T\\n now'
vr 12 jun. 22:13:44 2020
$ LC_ALL=C arch/freebsd12.amd64-64/bin/ksh -c 'printf %T\\n now'
Fri Jun 12 22:13:54 NST 2020
The last one is funny: it puts me in Newfoundland Standard Time.
Clearly, the time zone data compiled into ksh is broken. I also suspect it's roughly 27 years out of date.
Shells really have no business dealing with time zones. Providing and maintaining time zone data is the job of the operating system. Maybe it was justifiable in 1993 when not all OSs supported those facilities, but that time is long behind us.
So, a proper fix for this bug, with backward compatibility in mind, will be to:
printf %T
date
utility. It should search for it in the default utility path as in command -p
so that it works regardless of $PATH
Commands:
(cd /; whence -v usr/bin/env)
Actual output:
usr/bin/env is //usr/bin/env
Expected output:
usr/bin/env is /usr/bin/env
Multiple slashes are usually ignored, but a double initial slash is permitted by POSIX to have a special meaning, and in fact has a special meaning on Cygwin (indicating a Windows UNC network path). So it should not be adding one for that reason alone. It's also ugly.
[ 6:38 PM][ttys001][~/test/bz573936]
[211] mbp13 $ type ksh
ksh is a tracked alias for /usr/local/bin/ksh
[ 6:38 PM][ttys001][~/test/bz573936]
[212] mbp13 $ ksh
[ 6:38 PM][ttys001][~/test/bz573936]
[213] mbp13 $ TMOUT=5
[ 6:38 PM][ttys001][~/test/bz573936]
[214] mbp13 $ read
[ 6:39 PM][ttys001][~/test/bz573936]
[215] mbp13 $
shell will timeout in 60 seconds due to inactivity
Memory fault
[ 6:39 PM][ttys001][~/test/bz573936]
[215] mbp13 $
I read https://bugzilla.redhat.com/show_bug.cgi?id=573936 and tested it, didn't see the behavior the bug talked about, figured it must no longer be a problem.
Then the window I was running ksh in closed.
The read
timed out after 5 seconds, and the prompt came back. Then ksh popped the timeout message and the memory fault, and the shell crashed.
I can't make it happen if I put it in a script and call the script, but it happens repeatably if I do it interactively. If I don't set a timeout and just let the read
sit there, it doesn't crash.
(macOS 10.15.6, Version AJM 93u+m 2020-07-29)
I get it, ksh93/ksh doesn't have nmake. Thus:
[ 7:59 AM][ttys000][../src/ksh]
iMac $ bin/package write tgz source ksh
package: must have nmake to generate archives
[ 8:00 AM][ttys000][../src/ksh]
iMac $ bin/package write tgz binary ksh
package: must have nmake to generate archives
So... how is it done now? If there's something obvious I should have seen in 'package --man', feel free to point it out.
(There was a posting on the Korn Shell list that I have to find again, that made it sound like it was terribly easy to bring nmake back... have to find that again)
This bug was discovered and identified as separate while fixing #113.
When defining and running function in a (virtual/non-forked) subshell many times, a crash occurs.
Reproducer (minimal except for the counter):
for ((n=0; n < 50000; n++))
do print -n $'\r'$n # optional counter
(function foo { :; }; foo)
done
When compiled with vmalloc, the reproducer consistently crashes at iteration #32766. This is 2^15-2, suggesting an overflow of a short
somewhere (max value: 2^15-1, 32767).
When compiled without vmalloc and using the native malloc on macOS, it crashes slightly later and at unpredictable numbers of iterations, but all quite close to a short
overflow: 32907, 33191, 32986, 33038, 32832, 32783, etc.
Backtraces of the native macOS malloc crash are here.
The consistent crash with vmalloc always has the same backtrace (edit: updated with backtrace with source line numbers after recompiling with -O0 -g
):
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 ksh 0x000000010f5fd8d0 _sfpmove + 80 (sfpool.c:235)
1 ksh 0x000000010f5f16c5 sfclose + 837 (sfclose.c:96)
2 ksh 0x000000010f5b6469 stkclose + 121 (stk.c:317)
3 ksh 0x000000010f562bb2 sh_exec + 10674 (xec.c:1516)
4 ksh 0x000000010f564a99 sh_exec + 18585 (xec.c:2012)
5 ksh 0x000000010f55c7ed sh_subshell + 2605 (subshell.c:605)
6 ksh 0x000000010f564402 sh_exec + 16898 (xec.c:1886)
7 ksh 0x000000010f564a99 sh_exec + 18585 (xec.c:2012)
8 ksh 0x000000010f56571c sh_exec + 21788 (xec.c:2201)
9 ksh 0x000000010f564a99 sh_exec + 18585 (xec.c:2012)
10 ksh 0x000000010f4e7dbb exfile + 3243 (main.c:582)
11 ksh 0x000000010f4e8ce7 sh_main + 3367 (main.c:353)
12 ksh 0x000000010f4ce796 main + 38 (pmain.c:45)
13 libdyld.dylib 0x00007fff670df3d5 start + 1
On a ksh compiled with -DSHOPT_SPAWN=0
to disable AST spawnveg
(3) (see #79), running an external command in a non-forking subshell blocks SIGINT until that subshell is exited. If the subshell never exits, ksh must be killed from another shell using another signal.
This applies to both interactive shells and scripts.
Reproducer:
$ arch/*/bin/ksh -c '(/usr/bin/true; while :; do :; done); exit'
^C^C^C^C^C
(the extra 'exit' is to avoid the subshell being the last command, which would optimise it out when using -c
)
Attn: @JohnoKing
I may have been too quick in merging #22. On my system, that commit introduced a memory fault in one of the regression tests in variables.c:
$ bin/shtests -x -p variables
#### Regression-testing /usr/local/src/ksh93/ksh/arch/darwin.i386-64/bin/ksh ####
test variables begins at 2020-06-17+19:37:26
+ [0s|S0,tests/variables.sh,L26,e0] alias err_exit='err_exit $LINENO'
[...long-ish trace trimmed...]
+ [0s|S0,tests/variables.sh,L172,e1] COUNT=0
+ [0s|S0,tests/variables.sh,L173,e1] (( COUNT++ ))
+ [0s|S0,tests/variables.sh,L174,e1] (( COUNT != 1 || ACCESS!=2 ))
+ [0s|S0,tests/variables.sh,L177,e0] LANG=C
+ [0s|S0,tests/variables.sh,L177,e0] 1> /dev/null 2>& 1
shtests[343]: eval: line 1: 61715: Memory fault
test variables failed at 2020-06-17+19:37:26 with exit code 267 [ 125 tests (killed by SIGSEGV) ]
Here is the backtrace from lldb
:
* thread #1, stop reason = signal SIGSTOP
* frame #0: 0x0000000105f605a3 ksh`lookup + 734
frame #1: 0x0000000105f8808d ksh`varsub + 6329
frame #2: 0x0000000105f84a1e ksh`copyto + 2580
frame #3: 0x0000000105f85660 ksh`sh_macexpand + 468
frame #4: 0x0000000105f8a91b ksh`sh_macpat + 135
frame #5: 0x0000000105fa595d ksh`sh_exec + 4194
frame #6: 0x0000000105fa56ea ksh`sh_exec + 3567
frame #7: 0x0000000105f5cb2b ksh`exfile + 3243
frame #8: 0x0000000105f5da27 ksh`sh_main + 3367
frame #9: 0x00007fff5832b3d5 libdyld.dylib`start + 1
frame #10: 0x00007fff5832b3d5 libdyld.dylib`start + 1
It makes little sense to me. Why SIGSTOP when it was killed by SIGSEGV?
Commit cafe33f (see there for links/documentation) fixed test -t 1
in command substitutions by implementing a workaround: fork the subshell.
A real fix would be preferable, because test -t 1
and [ -t 1 ]
and [[ -t 1 ]]
continue to be wrong for "shared" command substitutions of the form var=${ list; }
:
$ v=${ [ -t 1 ] && echo whoops; }; echo $v
whoops
The forking workaround is not possible there, as forking would defeat the "shared" aspect of the shared command substitution.
The forking fix implemented in 102868f and 9d428f8, which stops the main shell's hash table from being cleared if PATH
is changed in a subshell, can cause a significant performance penalty for certain scripts that do something like
( PATH=... command foo )
in a subshell, especially if done repeatedly. This is because the hash table is cleared (and hence a subshell forks) even for temporary PATH assignments preceding commands.
The modernish regression tests take about half a second longer to run on 93u+m than they do on 93u+ as PATH=$DEFPATH command ...
is a common idiom in that library, and there's also this report of a 5.5% slowdown running a configure
script which I suspect might be due to this change.
Virtual subshells would not need to fork when changing PATH
if they get their own hash tables. Perhaps the code for alias subshell trees (which was removed in ec88886 because they were broken and unneeded) can provide the beginning of a template for their implementation.
src/cmd/INIT/ratz.c
is a self-contained .tar.gz
extractor. It still incorporates zlib 1.2.3 from 2005, whereas zlib 1.2.11 is out since 2017.
Related: #28
When doing a ./bin/package make the compilation is broken of when compiling under alpine linux with musl library
+ cc -D_BLD_DLL -fPIC -D_BLD_ast -O -I. -I/root/git_remote/ksh/src/lib/libast -Icomp -I/root/git_remote/ksh/src/lib/libast/comp -Iinclude -I/root/git_remote/ksh/src/lib/libast/include -Istd -I/root/git_remote/ksh/src/lib/libast/std -D_PACKAGE_ast -c /root/git_remote/ksh/src/lib/libast/misc/state.c
In file included from /root/git_remote/ksh/src/lib/libast/include/ast_std.h:66,
from /root/git_remote/ksh/src/lib/libast/include/ast.h:34,
from /root/git_remote/ksh/src/lib/libast/misc/state.c:26:
./ast_fs.h:135:18: warning: 'struct statvfs64' declared inside parameter list will not be visible outside of this definition or declaration
#define statvfs statvfs64
^~~~~~~~~
./ast_sys.h:14:26: note: in definition of macro '__PROTO__'
# define __PROTO__(x) x
^
/usr/include/sys/statvfs.h:48:19: note: in expansion of macro 'statvfs'
#define statvfs64 statvfs
^~~~~~~
./ast_fs.h:135:18: warning: 'struct statvfs64' declared inside parameter list will not be visible outside of this definition or declaration
#define statvfs statvfs64
^~~~~~~~~
./ast_sys.h:14:26: note: in definition of macro '__PROTO__'
# define __PROTO__(x) x
^
/usr/include/sys/statvfs.h:48:19: note: in expansion of macro 'statvfs'
#define statvfs64 statvfs
^~~~~~~
In file included from /root/git_remote/ksh/src/lib/libast/include/ast.h:34,
from /root/git_remote/ksh/src/lib/libast/misc/state.c:26:
/root/git_remote/ksh/src/lib/libast/include/ast_std.h:300:16: error: unknown type name 'off64_t'; did you mean 'off_t'? #define off_t off64_t
^~~~~~~
/usr/include/sys/mman.h:142:17: note: in expansion of macro 'off_t'
#define off64_t off_t
^~~~~
/root/git_remote/ksh/src/lib/libast/include/ast_std.h:300:16: error: unknown type name 'off64_t'
#define off_t off64_t
^~~~~~~
/usr/include/sys/mman.h:142:17: note: in expansion of macro 'off_t'
#define off64_t off_t
^~~~~
/root/git_remote/ksh/src/lib/libast/include/ast_std.h:300:16: error: unknown type name 'off64_t'; did you mean 'off_t'? #define off_t off64_t
^~~~~~~
/usr/include/sys/mman.h:142:17: note: in expansion of macro 'off_t'
#define off64_t off_t
^~~~~
/root/git_remote/ksh/src/lib/libast/include/ast_std.h:300:16: error: unknown type name 'off64_t'; did you mean 'off_t'? #define off_t off64_t
^~~~~~~
/usr/include/sys/mman.h:142:17: note: in expansion of macro 'off_t'
#define off64_t off_t
^~~~~
mamake [lib/libast]: *** exit code 1 making state.o
mamake: *** exit code 1 making lib/libast
mamake: *** exit code 1 making all
package: make done at Wed Jun 10 14:47:48 UTC 2020 in /root/git_remote/ksh/arch/linux.i386-64
When doing this in a virtual/non-forked subshell:
(typeset -A foo=([a]=1 [b]=2 [c]=3))
a memory leak occurs. The memory occupied by the array is not freed when exiting the subshell.
A reproducer with ps
can confirm this bug exists on 93u+ 2012-08-01:
for ((i=1; i<=10; i++)); do
for ((n=1; n<=100000; n++)); do
(typeset -A foo=([a]=1 [b]=2 [c]=3))
done
ps -p "$$" -o rss=,vsz=
done
Output on macOS:
15900 4285776
29996 4299856
44092 4313968
58188 4328048
72288 4342160
86380 4356240
100480 4370352
114572 4384432
128672 4398544
142772 4412656
Reproducer with vmstate
, which is compiled into 93u+m by default (edit: as of f9364b1, you have to pass -D_AST_vmalloc
to enable it):
builtin vmstate || exit
for ((i=1; i<=10; i++)); do
(typeset -A foo=([a]=1 [b]=2 [c]=3))
vmstate
done
Output (note increasing busy
and decreasing free
):
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(282256,170,65552) free=(107776,9,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283280,185,65552) free=(106560,6,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283408,188,65552) free=(106368,7,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283440,191,65552) free=(106128,17,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283664,194,65552) free=(105952,11,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283712,197,65552) free=(105728,19,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283888,200,65552) free=(105600,13,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(283920,203,65552) free=(105392,21,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(284096,206,65552) free=(105392,7,34288)
region=0x10ffe4928 method=best flags=0 size=393216 segments=4 busy=(284208,209,65552) free=(105120,14,34288)
Any reference to the current job (%%
or %+
) before there is one, passed to the kill
command, currently causes a segfault. This is also the case in the 93u+ 2012-08-01 release. For example:
$ ksh -c 'kill -s STOP %%; echo exit status $?'
Segmentation fault: 11
Backtrace from lldb
:
* thread #1, stop reason = signal SIGSTOP
* frame #0: 0x000000010ff80944 ksh`job_kill + 20
frame #1: 0x000000010ff80809 ksh`job_walk + 309
frame #2: 0x000000010ff5856d ksh`b_kill + 806
frame #3: 0x000000010ffab18b ksh`sh_exec + 16128
frame #4: 0x000000010ff5f48b ksh`exfile + 3243
frame #5: 0x000000010ff60387 ksh`sh_main + 3367
frame #6: 0x00007fff5832b3d5 libdyld.dylib`start + 1
edit: Same happens when using %-
to refer to the previous job before there is one.
It's simple:
[6] iMac $ date +'%l:%M %p'
8:26 PM
but:
[8] iMac $ printf '%(%l:%M %p)T\n'
Jul 4 20:27:27 PM
The man page for strftime(3) says %l is supposed to be the 12-hour hour without leading zeroes:
%l is replaced by the hour (12-hour clock) as a decimal number (1-12); single digits are preceded by
a blank.
And I didn't ask for the seconds at all, but ksh is displaying them.
93u+ and 93u+m both do the same thing. I can't find %()T anywhere in the new Korn book, but Learning the Korn Shell 2nd ed says it should reformat it "according to the date(1) specification." (p225)
(macOS 10.14.6)
This showed up in ~/Library/Logs/DiagnosticReports...
readonly.exe_2020-08-13-195404_mbp13.crash
I have no idea what it is and the shell I was running didn't crash or anything else. Putting it out there in case someone else knows what it is. Not running HEAD, if that matters, my current KSH is at commit 61437b2.
Process: readonly.exe [10458]
Path: /private/var/folders/*/readonly.exe
Identifier: readonly.exe
Version: 0
Code Type: X86-64 (Native)
Parent Process: ksh [9009]
Responsible: Terminal [1301]
User ID: 501
Date/Time: 2020-08-13 19:54:02.532 -0700
OS Version: Mac OS X 10.15.6 (19G2021)
Report Version: 12
Bridge OS Version: 4.6 (17P6610)
Anonymous UUID: FDAFEF58-17EF-0FD0-8BE4-E164085D539D
Time Awake Since Boot: 4400 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x000000010c5a8fab
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Bus error: 10
Termination Reason: Namespace SIGNAL, Code 0xa
Terminating Process: exc handler [10458]
VM Regions Near 0x10c5a8fab:
--> __TEXT 000000010c5a8000-000000010c5a9000 [ 4K] r-x/r-x SM=COW /private/var/folders/*/*.exe
__LINKEDIT 000000010c5a9000-000000010c5aa000 [ 4K] r--/r-- SM=COW /private/var/folders/*/*.exe
Application Specific Information:
dyld2 mode
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 readonly.exe 0x000000010c5a8f9b main + 11
1 libdyld.dylib 0x00007fff6cf43cc9 start + 1
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x000000010c5a8f90 rbx: 0x0000000000000000 rcx: 0x00007ffee36578d8 rdx: 0x00007ffee3657798
rdi: 0x0000000000000001 rsi: 0x00007ffee3657788 rbp: 0x00007ffee3657760 rsp: 0x00007ffee3657760
r8: 0x0000000000000000 r9: 0x0000000000000000 r10: 0x0000000000000000 r11: 0x0000000000000000
r12: 0x0000000000000000 r13: 0x0000000000000000 r14: 0x0000000000000000 r15: 0x0000000000000000
rip: 0x000000010c5a8f9b rfl: 0x0000000000010246 cr2: 0x000000010c5a8fab
Logical CPU: 2
Error Code: 0x00000007 (invalid protections for user data write)
Trap Number: 14
Binary Images:
0x10c5a8000 - 0x10c5a8fff +readonly.exe (0) <BA175183-1A29-388A-9FDF-CD35D0641EA8> /private/var/folders/*/readonly.exe
0x11b650000 - 0x11b6e1f47 dyld (750.6) <F9D4DEDC-8296-3E3F-B517-9C8B89A4C094> /usr/lib/dyld
0x7fff69f28000 - 0x7fff69f29fff libSystem.B.dylib (1281.100.1) <C0C9872A-E730-37EA-954A-3CE087C15535> /usr/lib/libSystem.B.dylib
0x7fff6a20e000 - 0x7fff6a260fff libc++.1.dylib (902.1) <59A8239F-C28A-3B59-B8FA-11340DC85EDC> /usr/lib/libc++.1.dylib
0x7fff6a261000 - 0x7fff6a276ffb libc++abi.dylib (902) <E692F14F-C65E-303B-9921-BB7E97D77855> /usr/lib/libc++abi.dylib
0x7fff6bd88000 - 0x7fff6bdbbfde libobjc.A.dylib (787.1) <6DF81160-5E7F-3E31-AA1E-C875E3B98AF6> /usr/lib/libobjc.A.dylib
0x7fff6cd25000 - 0x7fff6cd2aff3 libcache.dylib (83) <AF488D13-9E89-35E0-B078-BE37CC5B8586> /usr/lib/system/libcache.dylib
0x7fff6cd2b000 - 0x7fff6cd36fff libcommonCrypto.dylib (60165.120.1) <C7912BE5-993E-3581-B2A0-6AABDC8C5562> /usr/lib/system/libcommonCrypto.dylib
0x7fff6cd37000 - 0x7fff6cd3efff libcompiler_rt.dylib (101.2) <49B8F644-5705-3F16-BBE0-6FFF9B17C36E> /usr/lib/system/libcompiler_rt.dylib
0x7fff6cd3f000 - 0x7fff6cd48ff7 libcopyfile.dylib (166.40.1) <3C481225-21E7-370A-A30E-0CCFDD64A92C> /usr/lib/system/libcopyfile.dylib
0x7fff6cd49000 - 0x7fff6cddbfdb libcorecrypto.dylib (866.140.1) <60567BF8-80FA-359A-B2F3-A3BAEFB288FD> /usr/lib/system/libcorecrypto.dylib
0x7fff6cee8000 - 0x7fff6cf28ff0 libdispatch.dylib (1173.100.2) <CD9C059C-91D9-30E8-8926-5B9CD0D5D4F5> /usr/lib/system/libdispatch.dylib
0x7fff6cf29000 - 0x7fff6cf5ffff libdyld.dylib (750.6) <789A18C2-8AC7-3C88-813D-CD674376585D> /usr/lib/system/libdyld.dylib
0x7fff6cf60000 - 0x7fff6cf60ffb libkeymgr.dylib (30) <DB3337BE-01CA-3425-BD0C-87774FC0CDC0> /usr/lib/system/libkeymgr.dylib
0x7fff6cf6e000 - 0x7fff6cf6eff7 liblaunch.dylib (1738.140.1) <AFBCBDD3-0B55-3ECD-8E04-A73A3A57356B> /usr/lib/system/liblaunch.dylib
0x7fff6cf6f000 - 0x7fff6cf74ff7 libmacho.dylib (959.0.1) <AA613A9C-961A-3B67-B696-4622FA59FC4E> /usr/lib/system/libmacho.dylib
0x7fff6cf75000 - 0x7fff6cf77ff3 libquarantine.dylib (110.40.3) <F234E51D-FD0B-3EE4-B679-AE3EE9C536C3> /usr/lib/system/libquarantine.dylib
0x7fff6cf78000 - 0x7fff6cf79ff7 libremovefile.dylib (48) <7C7EFC79-BD24-33EF-B073-06AED234593E> /usr/lib/system/libremovefile.dylib
0x7fff6cf7a000 - 0x7fff6cf91ff3 libsystem_asl.dylib (377.60.2) <1563EE02-0657-3B78-99BE-A947C24122EF> /usr/lib/system/libsystem_asl.dylib
0x7fff6cf92000 - 0x7fff6cf92ff7 libsystem_blocks.dylib (74) <0D53847E-AF5F-3ACF-B51F-A15DEA4DEC58> /usr/lib/system/libsystem_blocks.dylib
0x7fff6cf93000 - 0x7fff6d01afff libsystem_c.dylib (1353.100.2) <BBDED5E6-A646-3EED-B33A-91E4331EA063> /usr/lib/system/libsystem_c.dylib
0x7fff6d01b000 - 0x7fff6d01effb libsystem_configuration.dylib (1061.141.1) <0EE84C33-64FD-372B-974A-AF7A136F2068> /usr/lib/system/libsystem_configuration.dylib
0x7fff6d01f000 - 0x7fff6d022fff libsystem_coreservices.dylib (114) <A199156E-058D-3ABB-BCE9-4B9F20DCED0F> /usr/lib/system/libsystem_coreservices.dylib
0x7fff6d023000 - 0x7fff6d02bfff libsystem_darwin.dylib (1353.100.2) <5B12B5DB-3F30-37C1-8ECC-49A66B1F2864> /usr/lib/system/libsystem_darwin.dylib
0x7fff6d02c000 - 0x7fff6d033fff libsystem_dnssd.dylib (1096.100.3) <EBB4C2C2-E031-3094-B40A-E67BF261D295> /usr/lib/system/libsystem_dnssd.dylib
0x7fff6d034000 - 0x7fff6d035ffb libsystem_featureflags.dylib (17) <29FD922A-EC2C-3F25-BCCC-B58D716E60EC> /usr/lib/system/libsystem_featureflags.dylib
0x7fff6d036000 - 0x7fff6d083ff7 libsystem_info.dylib (538) <8A321605-5480-330B-AF9E-64E65DE61747> /usr/lib/system/libsystem_info.dylib
0x7fff6d084000 - 0x7fff6d0b0ff7 libsystem_kernel.dylib (6153.141.1) <2B6311E6-6240-3EF7-8C87-475B66F7452C> /usr/lib/system/libsystem_kernel.dylib
0x7fff6d0b1000 - 0x7fff6d0f8fff libsystem_m.dylib (3178) <00F331F1-0D09-39B3-8736-1FE90E64E903> /usr/lib/system/libsystem_m.dylib
0x7fff6d0f9000 - 0x7fff6d120fff libsystem_malloc.dylib (283.100.6) <8549294E-4C53-36EB-99F3-584A7393D8D5> /usr/lib/system/libsystem_malloc.dylib
0x7fff6d121000 - 0x7fff6d12effb libsystem_networkextension.dylib (1095.140.2) <F06C65C5-2CBE-313C-96E1-A09240F9FE57> /usr/lib/system/libsystem_networkextension.dylib
0x7fff6d12f000 - 0x7fff6d138ff7 libsystem_notify.dylib (241.100.2) <FA22F928-D91B-3AA5-96BB-3186AC0FB264> /usr/lib/system/libsystem_notify.dylib
0x7fff6d139000 - 0x7fff6d141fef libsystem_platform.dylib (220.100.1) <009A7C1F-313A-318E-B9F2-30F4C06FEA5C> /usr/lib/system/libsystem_platform.dylib
0x7fff6d142000 - 0x7fff6d14cfff libsystem_pthread.dylib (416.100.3) <62CB1A98-0B8F-31E7-A02B-A1139927F61D> /usr/lib/system/libsystem_pthread.dylib
0x7fff6d14d000 - 0x7fff6d151ff3 libsystem_sandbox.dylib (1217.141.1) <6DE13684-5A67-3009-A53A-C9086CF241C3> /usr/lib/system/libsystem_sandbox.dylib
0x7fff6d152000 - 0x7fff6d154fff libsystem_secinit.dylib (62.100.2) <F80872AA-E1FD-3D7E-8729-467656EC6561> /usr/lib/system/libsystem_secinit.dylib
0x7fff6d155000 - 0x7fff6d15cffb libsystem_symptoms.dylib (1238.120.1) <5820A2AF-CE72-3AB3-ABCC-273A3419FB55> /usr/lib/system/libsystem_symptoms.dylib
0x7fff6d15d000 - 0x7fff6d173ff2 libsystem_trace.dylib (1147.120) <04B47629-847B-3D74-8ABE-C05EF9DEEFE4> /usr/lib/system/libsystem_trace.dylib
0x7fff6d175000 - 0x7fff6d17aff7 libunwind.dylib (35.4) <42B7B509-BAFE-365B-893A-72414C92F5BF> /usr/lib/system/libunwind.dylib
0x7fff6d17b000 - 0x7fff6d1b0ffe libxpc.dylib (1738.140.1) <3E243A41-030F-38E3-9FD2-7B38C66C35B1> /usr/lib/system/libxpc.dylib
External Modification Summary:
Calls made by other processes targeting this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by this process:
task_for_pid: 0
thread_create: 0
thread_set_state: 0
Calls made by all processes on this machine:
task_for_pid: 3624
thread_create: 0
thread_set_state: 0
VM Region Summary:
ReadOnly portion of Libraries: Total=393.4M resident=0K(0%) swapped_out_or_unallocated=393.4M(100%)
Writable regions: Total=17.3M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=17.3M(100%)
VIRTUAL REGION
REGION TYPE SIZE COUNT (non-coalesced)
=========== ======= =======
Kernel Alloc Once 8K 1
MALLOC 9300K 8
MALLOC guard page 24K 5
STACK GUARD 56.0M 1
Stack 8192K 1
__DATA 596K 39
__DATA_CONST 20K 1
__LINKEDIT 388.8M 3
__OBJC_RO 32.3M 1
__OBJC_RW 1904K 1
__TEXT 4752K 40
shared memory 8K 2
=========== ======= =======
TOTAL 501.2M 103
This bug was signalled and videoed by @saper in ksh-community/ksh#8 (comment) and the subsequent comment. Video 1; video 2 (see also instructions there).
The entire ksh-community/ksh#8 thread contains background information worth reading.
Note that the first variant of this bug also applies to raw POSIX (neither vi nor emacs) editing mode if a UTF-8 locale is active, as this raw editing mode is handled by edit/vi.c in that case.
There has been a bug report of an exec format error when executing "cat TODO | while read line; do ls; done".
Ref:
I was able to reproduce this issue on 93u+m 2020-07-14 and it appears to be an issue at least since 93u+ 2012-08-01.
$ cat TODO
1
2
$ cat TODO | while read line; do ls; done
ls: ls: cannot execute [Exec format error]
ls: ls: cannot execute [Exec format error]
Tagging @chrisbertin and @mirabilos - reporters of the issue.
The problem is back, built from HEAD on 10.15.6 today. It's not the same test that fails as the one in #70.
[ 9:03 AM][ttys000][~/src/ksh][master]
[359] mbp13 $ bin/shtests leaks
#### Regression-testing /Users/mwilson/src/ksh/arch/darwin.i386-64/bin/ksh ####
test leaks begins at 2020-07-29+09:04:11
test leaks passed at 2020-07-29+09:04:11 [ 6 tests 0 errors ]
test leaks(C.UTF-8) begins at 2020-07-29+09:04:11
leaks.sh[72]: variable value reset memory leak -- 48 bytes after 1000 iterations
test leaks(C.UTF-8) failed at 2020-07-29+09:04:11 with exit code 1 [ 6 tests 1 error ]
test leaks(shcomp) begins at 2020-07-29+09:04:11
test leaks(shcomp) passed at 2020-07-29+09:04:11 [ 6 tests 0 errors ]
Total errors: 1
CPU time user: system:
main: 0m00.007s 0m00.014s
tests: 0m00.153s 0m00.220s
[ 9:04 AM][ttys000][~/src/ksh][master]
[360] mbp13 $ git log | tee
commit 05081dfc1c6ef18eee06d2a31c3f1cb4bb68d9d7
Author: Johnothan King <[email protected]>
Date: Mon Jul 27 05:27:20 2020 -0700
Originally posted by @posguy99 in #70 (comment)
@McDutchie points out that this hang pre-dates #103, see last comment in that issue.
[10:37 AM][ttys000 +1][~/src/ksh-segfault][master]
[1049] iMac $ arch/darwin.i386-64/bin/ksh
[10:37 AM][ttys000 +2][~/src/ksh-segfault][master]
[1050] iMac $ top &
[1] 72570
[10:37 AM][ttys000 +2][~/src/ksh-segfault][master]
[1051] iMac $
[1] + Stopped (SIGTTOU) top &
[10:37 AM][ttys000 +2][~/src/ksh-segfault][master]
[1051] iMac $ fg
top
[10:38 AM][ttys000 +2][~/src/ksh-segfault][master]
[1052] iMac $
And now the shell is unresponsive.
This is related to #89 , right?
I've managed to get it to do that four times now just doing the above, but it's not reliably repeatable. If it does not display the 'Stopped' line without hitting Return again, it seems to never do it. Only if it does that on its own, and not always.
I have to kill it from somewhere else or close Terminal.app.
74575 s000 S+ 0:00.02 arch/darwin.i386-64/bin/ksh
^ from the latest one.
Originally posted by @posguy99 in #103 (comment)
Commands:
(printf() { :; }; whence -a printf)
(printf() { :; }; type -a printf)
Actual output:
printf is a function
printf is /usr/bin/printf
Expected output:
printf is a function
printf is a shell builtin
printf is /usr/bin/printf
Commands:
(autoload printf; whence -a printf)
(autoload printf; type -a printf)
Actual output:
printf is an undefined function
printf is /usr/bin/printf
Expected output:
printf is an undefined function
printf is a shell builtin
printf is /usr/bin/printf
$ ksh -c 'builtin cat; whence -a cat'
cat is a shell builtin
cat is /bin/cat
cat is an undefined function
The third line should not appear.
This bug was found by @JohnoKing while working on #25.
This is a SSCCE:
$ cat bug.sh
#!/bin/ksh93
unset a b
c=0
function set_ac { a=1; c=1; }
function set_abc { ( set_ac ; b=1 ) }
set_abc
echo "a=$a b=$b c=$c"
$ ./bug.sh
a=1 b= c=0
Hence, although set_abc
calls set_ac
in a subshell, the assignment a=1 propagates to the parent shell. There are a few known workarounds [1]. Replace the definition of set_abc
with either
function set_abc { ( set_ac ; b=1 ) | echo -n; }
or
function set_abc { ( ulimit -t unlimited ; set_ac ; b=1 ) }
Here is a third workaround that shows how bad subshells step on each other's toes. By setting a in a second subshell, it prevents the first subshell to set a in the parent shell.
function set_abc { ( set_ac ; b=1 ) }
( a=1 )
With either of the three above we get
$ ./bug.sh
a= b= c=0
References:
[1] att#73 (comment)
[2] https://stackoverflow.com/q/62385690/1137388
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.