landley / toybox Goto Github PK

toybox

License: BSD Zero Clause License

Makefile 0.16% C 72.40% Shell 10.05% Python 0.14% HTML 17.26%

toybox's Introduction

Toybox: all-in-one Linux command line.

--- Getting started

You can download static binaries for various targets from:

  http://landley.net/toybox/bin

The special name "." indicates the current directory (just like ".." means
the parent directory), and you can run a program that isn't in the $PATH by
specifying a path to it, so this should work:

  wget http://landley.net/toybox/bin/toybox-x86_64
  chmod +x toybox-x86_64
  ./toybox-x86_64 echo hello world

--- Building toybox

Type "make help" for build instructions.

Toybox uses the "make menuconfig; make; make install" idiom same as
the Linux kernel. Usually you want something like:

  make defconfig
  make
  make install

Or maybe:

  LDFLAGS="--static" CROSS_COMPILE=armv5l- make defconfig toybox
  PREFIX=/path/to/root/filesystem/bin make install_flat

The file "configure" defines default values for many environment variables
that control the toybox build; if you export any of these variables into your
environment, your value is used instead of the default in that file.

The CROSS_COMPILE argument above is optional, the default builds a version of
toybox to run on the current machine. Cross compiling requires an appropriately
prefixed cross compiler toolchain, several example toolchains (built using
the file "scripts/mcm-buildall.sh" in the toybox source) are available at:

  https://landley.net/toybox/downloads/binaries/toolchains/latest

For the "CROSS_COMPILE=armv5l-" example above, download
armv5l-linux-musleabihf-cross.tar.xz, extract it, and add its "bin"
subdirectory to your $PATH. (And yes, the trailing - is significant,
because the prefix includes a dash.)

For more about cross compiling, see:

  https://landley.net/toybox/faq.html#cross
  http://landley.net/writing/docs/cross-compiling.html
  http://landley.net/aboriginal/architectures.html

For a more thorough description of the toybox build process, see:

  http://landley.net/toybox/code.html#building

--- Using toybox

The toybox build produces a multicall binary, a "swiss-army-knife" program
that acts differently depending on the name it was called by (cp, mv, cat...).
Installing toybox adds symlinks for each command name to the $PATH.

The special "toybox" command treats its first argument as the command to run.
With no arguments, it lists available commands. This allows you to use toybox
without installing it, and is the only command that can have an arbitrary
suffix (hence "toybox-armv5l").

The "help" command provides information about each command (ala "help cat"),
and "help toybox" provides general information about toybox.

--- Configuring toybox

It works like the Linux kernel: allnoconfig, defconfig, and menuconfig edit
a ".config" file that selects which features to include in the resulting
binary. You can save and re-use your .config file, but may want to
run "make oldconfig" to re-run the dependency resolver when migrating to
new versions.

The maximum sane configuration is "make defconfig": allyesconfig isn't
recommended as a starting point for toybox because it enables unfinished
commands, debug code, and optional dependencies your build environment may
not provide.

--- Creating a Toybox-based Linux system

Toybox has a built-in simple system builder (scripts/mkroot.sh) with a
Makefile target:

  make root
  sudo chroot root/host/fs /init

Type "exit" to get back out. If you install appropriate cross compilers and
point it at Linux source code, it can build simple three-package systems
that boot to a shell prompt under qemu:

  make root CROSS_COMPILE=sh4-linux-musl- LINUX=~/linux
  cd root/sh4
  ./qemu-sh4.sh

By calling scripts/mkroot.sh directly you can add additional packages
to the build, see scripts/root/dropbear as an example.

The FAQ explains this in a lot more detail:

  https://landley.net/toybox/faq.html#system
  https://landley.net/toybox/faq.html#mkroot

--- Presentations

1) "Why Toybox?" talk at the Embedded Linux Conference in 2013

    outline: http://landley.net/talks/celf-2013.txt
    video: http://youtu.be/SGmtP5Lg_t0

    The https://landley.net/toybox/about.html page has nav links breaking that
    talk down into sections.

2) "Why Public Domain?" The rise and fall of copyleft, Ohio LinuxFest 2013

    outline: http://landley.net/talks/ohio-2013.txt
    audio: https://archive.org/download/OhioLinuxfest2013/24-Rob_Landley-The_Rise_and_Fall_of_Copyleft.mp3

3) Why did I do Aboriginal Linux (which led me here)

    260 slide presentation:
      https://speakerdeck.com/landley/developing-for-non-x86-targets-using-qemu

    How and why to make android self-hosting:
      http://landley.net/aboriginal/about.html#selfhost

    More backstory than strictly necessary:
      https://landley.net/aboriginal/history.html

4) What's new with toybox (ELC 2015 status update):

    video: http://elinux.org/ELC_2015_Presentations
    outline: http://landley.net/talks/celf-2015.txt

5) Toybox vs BusyBox (2019 ELC talk):

    outline: http://landley.net/talks/elc-2019.txt
    video: https://www.youtube.com/watch?v=MkJkyMuBm3g

--- Contributing

The three important URLs for communicating with the toybox project are:

  web page: http://landley.net/toybox

  mailing list: http://lists.landley.net/listinfo.cgi/toybox-landley.net

  git repo: http://github.com/landley/toybox

The maintainer prefers patches be sent to the mailing list. If you use git,
the easy thing to do is:

  git format-patch -1 $HASH

Then send a file attachment. The list holds messages from non-subscribers
for moderation, but I usually get to them in a day or two.

I download github pull requests as patches and apply them with "git am"
(which avoids gratuitous merge commits). Sometimes I even remember to close
the pull request.

If I haven't responded to your patch after one week, feel free to remind
me of it.

Android's policy for toybox patches is that non-build patches should go
upstream first (into vanilla toybox, with discussion on the toybox mailing
list) and then be pulled into android's toybox repo from there. (They
generally resync on fridays). The exception is patches to their build scripts
(Android.mk and the checked-in generated/* files) which go directly to AOSP.

(As for the other meaning of "contributing", https://patreon.com/landley is
always welcome but I warn you up front I'm terrible about updating it.)

toybox's People

Contributors

Stargazers

Watchers

Forkers

subfuzion redpig adelva1984 jbergstroem ajoyacr durgamurali lipilee kmikhailov izabera pohly yashdsaraf drinkcat connlan andychu iwonasado oooverflow emaste powerblock davisnt geekmaster andyincode d4rken busterb kosslab-kr ivop kiyous eugenis haiqu bygit djnando eeewxj josephatgmems usta yodamaster fbarchard icasimpan-experimental jmgao joyounger test4d kungfoo-linux thestartcup rowhit thor1979 neo2756 zachriggle ecbtnrt garnica1 thomasadam skyformat99 technosaurus danyboy-pc-pc mkmkme forkme7 fenglin-ubnt winsx minghui-liu rivy acidburn0zzz filipperich 0x746b73 wrtcoder jebuskrust happy-ferret baileyforrest tpimh spiterikevin leeyh8901 lobesitdoll rajnikant12345 fevv9 cicerolneto bookgin moneytech wesleyc23 makazmk jasonspiro uprestor juno-explorer jmakip koalaman mingminglee leond08 ubuntu-repo sameerppradhan dedlous leok7v kevinmiles michaanjol surachai35 theapachecats pombredanne rrozestw pocomane macmac0000 danil-ebali staceb 437177786 e5ten proakis balsini

toybox's Issues

bzcat segfault

File here:
http://www.cs.utah.edu/~regehr/foo.bz2
causes a segfault when run like this:
toybox bzcat < foo.bz2
I haven't checked if this can be used to get remote code execution but out-of-bounds stores controlled by untrusted data are generally bad.

env doesn't like unsetting variables

Test case:

env BLAH=/dev/null BLUH="" env | grep -e BLAH -e BLUH

On a system using GNU coreutils, this gives:

somasis/discord:~/ % env BLAH=/dev/null BLUH="" env | grep -e BLAH -e BLUH
BLAH=/dev/null
BLUH=

On toybox, the env invocation gives this:

somasis/cicero:~/ % env BLAH=/dev/null BLUH=""
env: exec BLUH: No such file or directory

blkid uuid's print order

# ./toybox blkid | grep sda1 ; blkid | grep sda1                                          
/dev/sda1: LABEL="EFI" UUID="ed17-e367" TYPE="vfat"
/dev/sda1: LABEL="EFI" UUID="67E3-17ED" TYPE="vfat"

toybox ps give out a segment fault

Hi，

we have meet ps segment fault while doing test

pid: 6149, tid: 6149, name: ps  >>> /system/bin/ps <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x10
Cause: null pointer dereference

Stack Trace:
  RELADDR           FUNCTION                     FILE:LINE
  0000000000034294  get_threads+228              external/toybox/toys/posix/ps.c:917
  000000000000da2c  dirtree_handle_callback+36   external/toybox/lib/dirtree.c:112
  000000000000db38  dirtree_recurse+128          external/toybox/lib/dirtree.c:156
  000000000000da84  dirtree_handle_callback+124  external/toybox/lib/dirtree.c:115
  0000000000033628  ps_main+952                  external/toybox/toys/posix/ps.c:1238
  000000000001375c  toy_exec+104                 external/toybox/main.c:166
  000000000001330c  toybox_main+48               external/toybox/main.c:179
  0000000000013818  main+124                     external/toybox/main.c:237

The ERROR code suspected as blow in get_threads, while in multi-thread , passing the parameters of the process is still running, the upper layer is not stopped, and ran to this function may have been out of the process, the marked code, will lead to access to empty pointers, causing crashes.

I also find some one has reported similar question in mailing list

Do you got any solution to this？

/ Recurse down into tasks, retaining thread groups.
// Disable show_process at least until we can calculate tcount
kcount = TT.kcount;
sprintf(toybuf, "/proc/%u/task", pid);
new->child = dirtree_flagread(toybuf, DIRTREE_SHUTUP|DIRTREE_PROC, get_ps);  ///////<<<<< suspected
if (new->child == DIRTREE_ABORTVAL) new->child = 0;  ///////<<<<< suspected
TT.threadparent = 0;
kcount = TT.kcount-kcount+1;
tb = (void *)new->extra;
tb->slot[SLOT_tcount] = kcount;
 
// Fill out tid and thread count for each entry in group
if (new->child) for (dt = new->child->child; dt; dt = dt->next) {
  tb = (void *)dt->extra;
  tb->slot[SLOT_pid] = pid;
  tb->slot[SLOT_tcount] = kcount;
}
 
// Save or display
if (!TT.show_process) return DIRTREE_SAVE;
TT.show_process((void *)new->extra);
dt = new->child;              ///////<<<<< suspected
new->child = 0;               ///////<<<<< suspected
while (dt->child) {            ///////<<<<< suspected
  new = dt->child->next;
  TT.show_process((void *)dt->child->extra);
  free(dt->child);
  dt->child = new;
}
free(dt);

find -H/-L gives an error on broken symlinks

$ ln -s nowhere broken
$ find -H broken
broken
$ ./toybox find -H broken
find: broken: No such file or directory

Same with -L.

./configure option to embed zsh (MIT licensed) into toybox

There can be a optional ./configure option in toybox - to point to source code location of zsh ; so that one can embed compile "zsh too into toybox" (zsh is a full implementation of bash, but MIT licensed)

note that you are not actually putting zsh source code of zsh into toybox. you are just letting people an option to embed an external source code tree into toybox, so that they can get "applet" of zsh embedded into single binary of "toybox"

combine file, blkid/fstype and readelf

file shares multiple purposes with blkid/fstype (file has similar functionality for filesystems) and readelf (parts of readelf have already been implemented in the current version of file)

I already did a lot of scraping of offsets and magic values for the MIME type data here.

Or perhaps their functions should go in the library?

`du` applet prints wrong size

Nexus5 with Android 6.0.

shell@hammerhead:/sdcard $ busybox du -sh DCIM
3.6G    DCIM
shell@hammerhead:/sdcard $ busybox du -sk DCIM
3822620 DCIM
shell@hammerhead:/sdcard $ toolbox du -sk DCIM
3822620 DCIM
shell@hammerhead:/sdcard $ toybox du -sk DCIM
18446744073709179932    DCIM
shell@hammerhead:/sdcard $ toybox du -sh DCIM
-.3 GB  DCIM

Help building toybox with the NDK/bionic

I've tried to setup the latest NDK to test #36. I followed the instructions here, to export a standalone toolchain.
The built starts but errors out because it either can't find cutils/sched_policy.h or pty.h.

The only place I can find a sched_policy.h and pty.h is in the AOSP repo.

What am I doing wrong?

Cannot compile wget with Clang-3.9.1

It seems the utility wget in toybox cannot be compiled with clang-3.9.1.

The output in question:

clang -O2 -pipe -funsigned-char -Wno-string-plus-int -I . -Os -ffunction-sections -fdata-sections -fno-asynchronous-unwind-tables -fno-strict-aliasing -c toys/posix/cat.c -o generated/obj/cat.o
toys/pending/wget.c:148:16: error: use of undeclared identifier
      'TOYBOX_VERSION'; did you mean '_LIB_VERSION'?
  strncpy(ver, TOYBOX_VERSION, 5);
               ^~~~~~~~~~~~~~
               _LIB_VERSION
/usr/include/math.h:327:26: note: '_LIB_VERSION' declared here
extern _LIB_VERSION_TYPE _LIB_VERSION;
                         ^
1 error generated.
Makefile:19: recipe for target 'toybox' failed
make: *** [toybox] Error 1

Rewrite all bash-specific code to be POSIX shell compatible

Most systems have bash installed, so that is not an issue for them, but some lack it. I suggest changing default interpreter of scripts/*.sh and tests/*.test from /bin/bash to /bin/sh and checking that they run fine with dash which is the closest to full POSIX shell implementation. I'm not proposing any changes yet, I'm just starting a discussion here (or maybe it should be reposted to the mailing list?)

expr build warning - implicit cast and missing initializer

external/toybox/toys/pending/expr.c:116:28: warning: field precision should have type 'int', but argument has type 'long' [-Wformat]
ret->s = xmprintf("%.*s", m[1].rm_eo-m[1].rm_so, target+m[1].rm_so);
~~^~ ~~~~~~~~~~~~~~~~~~~~~
external/toybox/toys/pending/expr.c:247:24: warning: missing field 'i' initializer [-Wmissing-field-initializers]
struct value ret = {0};
^

Make zlib functions compatible with miniz

miniz is a small implementation of zlib, it tries to be compatible with zlib, but it's not 100% compatible.

It lacks gzFile. I think toys/pending/gzip.c should be rewritten to be compatible with miniz.

chown/chgrp aborts on numeric ids if they can't be resolved

From #36 (comment)

I just noticed that "chown 123:123 toybox" is saying:
chown: user '123'
Which is broken in a couple ways (there should be a ":errno message" on the end of that if it's an actual error, and numeric user IDs should work even if they're not in /etc/passwd) so I need to off and fix THAT before cycling back to this. :)

chown uses xgetpwnamid (chgrp similar) to resolve the command input independent of whether it is a uid or a name, but if the function fails to resolve it throws an error and we exit.

Idea:
If getpwnam returns something we have a uid we can use.
if getpwnam returns null the input can't be resolved to a uid. So it's either a uid or a user we can resolve. In this case we try turning the input into a number and use that as uid.
Same solution for group.

I'll try something and send a patch to the mailing list.

`cp -p` does not preserve timestamps

In toybox version 0.7.1-android and in toybox version 0.7.5 for x86_64 files copied with cp -p have their copy time set to all timestamps. However when using cp --preserve=t timestamps are as expected.

xargs -0 drops arguments

Actual output:

$ printf 'a\0b\0c\0d\0e\0f' | ~/toybox/toybox xargs -0 -n1 echo _
_ a
_ c
_ e
_

$ printf 'a\0b\0c\0d\0e\0f' | ~/toybox/toybox xargs -0 -n2 echo _
_ a b
_ d e
_

$ printf 'a\0b\0c\0d\0e\0f' | ~/toybox/toybox xargs -0 -n3 echo _
_ a b c
_ e f

Expected output:

$ printf 'a\0b\0c\0d\0e\0f' | xargs -0 -n1 echo _
_ a
_ b
_ c
_ d
_ e
_ f

$ printf 'a\0b\0c\0d\0e\0f' | xargs -0 -n2 echo _
_ a b
_ c d
_ e f

$ printf 'a\0b\0c\0d\0e\0f' | xargs -0 -n3 echo _
_ a b c
_ d e f

printf is working:

$ printf 'a\0b\0c\0d\0e\0f' | ~/toybox/toybox od -tx1
0000000 61 00 62 00 63 00 64 00 65 00 66
0000013

FWIW: I was using xargs -0 to print /proc/<pid>/cmdline. It's also commonly used with find -print0.

Test applet NOT operator issues

The int not is never initliazed but still applied at the end of the function to the exitvalue.

Also:
If the file does not exist the ! operator is never applied to the exitvalue.

(I submitted this issue to the mailing list. But I think having open issues for "TODO stuff " here is good, even if discussion on the mailing list is preferred.)

Any plans to support extra utilities...

...like Git, Ruby, or Python?

I'd love to use Android as a real programming device someday.

Sure, it may seem nonsensical to some, but I think making quick Git commits or Ruby code on-the-go would be fantastic.

Side note: nobody on Earth seems to care about cross-compiling utilities to iOS/Android. The process is rather difficult though, and Stack Overflow is of no help. Perhaps someone on this repository has a better understanding of GCC/Clang than I do?

toysh segfaults after "clear" or "reset"

Hi,

playing with toybox I've found that any command after "clear" or "reset" causes "Double free or corruption". Simple test for this is

echo -e 'clear\nuname -a' | toybox sh
echo -e 'reset\nuname -a' | toybox sh

You can also reproduce this in interactive mode.

Tested with 0.7.4-21-g105595524ead and 0.7.5-58-g1fec45a2a57f

portability.c bug fix for config2help on OSX

config2help currently doesn't work on OS X, it terminates parsing of Config.in at first blank line. This is because getdelim() in portability.c returns -1 whenever the line comprises only a single linefeed character. Fixing this was a trivial change to two lines (see below), and config2help now works on OS X but I haven't regression tested this on any other commands which rely on getdelim()

diff --git a/lib/portability.c b/lib/portability.c
index 78e500b..38cf5cb 100644
--- a/lib/portability.c
+++ b/lib/portability.c
@@ -61,9 +61,8 @@ ssize_t getdelim(char **linep, size_t *np, int delim, FILE *stream)
       line = *linep = new_line;
     }
 
-    line[i] = ch;
+   line[i++] = ch;
     if (ch == delim) break;
-    i += 1;
   }
 
   if (i > *np) {
@@ -74,7 +73,7 @@ ssize_t getdelim(char **linep, size_t *np, int delim, FILE *stream)
     *np = new_len;
     line = *linep = new_line;
   }
-  line[i + 1] = '\0';
+  line[i] = '\0';
 
   return i > 0 ? i : -1;
 }

README: link broken

$ wget http://landley.net/bin/toybox-x86_64
--2016-04-07 22:40:26--  http://landley.net/bin/toybox-x86_64
Resolving landley.net (landley.net)... 208.113.171.142
Connecting to landley.net (landley.net)|208.113.171.142|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2016-04-07 22:40:27 ERROR 404: Not Found.

stat: block size differences

Toybox 0.7.1, Busybox 1.24.2
On a [email protected].

Why is there a difference in blocksize?

root@hammerhead:/sdcard # busybox stat -c %B:%b:%o:%s twrp-3.0.0-0-hammerhead.img
512:28632:4096:14657536
root@hammerhead:/sdcard # toybox stat -c %B:%b:%o:%s twrp-3.0.0-0-hammerhead.img
4096:28632:4096:14657536

%B Bytes per block

busybox says 512Byte while toybox says 4096Byte.

28632 Blocks * 512Byte = 14659584 Byte
Which is a lot closer to the actual file size of 14657536 Byte (reported by both toybox&busybox).

Other commands from both binaries also show a block size of 4096 Byte though.

root@hammerhead:/sdcard # busybox stat twrp-3.0.0-0-hammerhead.img                                       <
  File: twrp-3.0.0-0-hammerhead.img
  Size: 14657536        Blocks: 28632      IO Block: 4096   regular file
Device: 11h/17d Inode: 218474      Links: 1
Access: (0660/-rw-rw----)  Uid: (    0/ UNKNOWN)   Gid: ( 1015/ UNKNOWN)
Access: 2016-03-18 14:16:00.000000000
Modify: 2016-03-18 14:16:03.000000000
Change: 2016-03-18 14:16:03.000000000

root@hammerhead:/sdcard # busybox stat -f .
  File: "."
    ID: 0        Namelen: 255     Type: UNKNOWN
Block size: 4096
Blocks: Total: 7015287    Free: 189096     Available: 189096
Inodes: Total: 1785856    Free: 1747803

root@hammerhead:/sdcard # toybox stat twrp-3.0.0-0-hammerhead.img
  File: `twrp-3.0.0-0-hammerhead.img'
  Size: 14657536         Blocks: 28632   IO Blocks: 4096        regular file
Device: 11h/17d  Inode: 218474   Links: 1
Access: (660/-rw-rw----Segmentation fault

root@hammerhead:/sdcard # toybox stat -f .                                                 <
  File: "."
    ID: 0000000000000000 Namelen: 255    Type: 65735546
Block Size: 4096    Fundamental block size: 4096
Blocks: Total: 7015287  Free: 189096    Available: 189096
Inodes: Total: 1785856  Free: 1747803

cp -R --preserve timing

cp -R --preserve copies the content of a folder, before copying owner/permissions.
This can be problematic if the copy is canceled.

For instance, if we have
/a 1000 1000 0755
/a/b 2000 2000 0755

We'll have a point in time where during cp -R --preserve as root, we have:
/a 0 0 0775
/a/b 2000 2000 0755

The "cp" might command might be behind a daemon-wall, which leads to an unrecoverable state.

For instance, in Android, vold uses cp -R --preserve to move data from internal to external storage and back.
vold runs as root, this means that if the copy is stopped during the migration, /data/media will contain a root-owned directory. This means this folder won't ever be removable or writable ever, unless the user does a factory reset.

When I call any applet with the --help option, please show this general information first

Hi! When I call any applet with the --help option, it would be good if you could please start the output with the following lines, then a newline.

Toybox multi-call binary $(git describe)
Web page: http://landley.net/toybox/
Bug reports: https://github.com/landley/toybox/issues
Feedback: http://lists.landley.net/listinfo.cgi/toybox-landley.net
Contributors, see: https://landley.net/toybox/cleanup.html#intro

Some users have Toybox because it came preinstalled on their device. The above information will make it easier for such users to figure out where and how to submit bug reports and patches.

git describe looks at tag names and commits, and hopefully outputs a version number like "0.7.1-106-g409a8e0" or "0.7.2".

tar to stdout prints "removing leading / from member names"

Using tar as tar cf - /bin/sh or similar, I could not get the archive recognised, doing a hexdump I realised it was outputing the warning to stdout before the archive, which is not very helpful:

00000000  72 65 6d 6f 76 69 6e 67  20 6c 65 61 64 69 6e 67  |removing leading|
00000010  20 27 2f 27 20 66 72 6f  6d 20 6d 65 6d 62 65 72  | '/' from member|
00000020  20 6e 61 6d 65 73 0a 62  69 6e 2f 73 68 00 00 00  | names.bin/sh...|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Code of Conduct link have been moved.

The old link: https://engineering.twitter.com/opensource/code-of-conduct (can be found from README file)
The new link: https://github.com/twitter/code-of-conduct/blob/master/code-of-conduct.md

Del

I'm sorry, wrong repo.
Created same in CM's repo:
CyanogenMod/android_external_toybox#3

Toybox printf is printing incorrect hex values

printf of an hex value is printing strange values. Please see the below code

root:/ # printf "%x\n" 100
300000078

64 is expected but it prints strange values.

I am working in android environment. printf in android env uses printf of toybox.

root:/ # which printf
/system/bin/printf
root:/ # ls -l /system/bin/printf
lrwxr-xr-x root     shell             2016-11-14 21:02 printf -> toybox

If this issue is already fixed, please tell the corresponding patch.

stat does not resolve GUID or UID on Android

Just to track known issues, came up in #36

It works using the stock toybox that ships with Android 6.0.1 on the Nexus5 (--version shows c96e424-android):

root@hammerhead:/sdcard # stat -c "%G" twrp-3.0.0-0-hammerhead.img
    root
root@hammerhead:/sdcard # stat -c "%U" twrp-3.0.0-0-hammerhead.img
    root

According to @landley this is not a toolchain issue and already a work in progress.

0.7.0: scripts/make.sh: line 270: wait: pid XYZ is not a child of this shell

We use toybox-0.7.0 as part of the Chromium OS project, and sometimes hit an issue when building it on our automated builders (see this issue):

toybox-0.7.0: armv7a-cros-linux-gnueabi-gcc -O2 -O2 -pipe -march=armv7-a -mtune=cortex-a15 -mfpu=neon -mfloat-abi=hard -g -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -clang-syntax -funsigned-char -Wno-string-plus-int -I . -Os -ffunction-sections -fdata-sections -fno-asynchronous-unwind-tables -fno-strict-aliasing -c toys/posix/tail.c -o generated/obj/tail.o
toybox-0.7.0: scripts/make.sh: line 270: wait: pid 8477 is not a child of this shell
toybox-0.7.0: Makefile:19: recipe for target 'toybox' failed
toybox-0.7.0: make: *** [toybox] Error 1
toybox-0.7.0:  * ERROR: sys-apps/toybox-0.7.0::portage-stable failed (compile phase):
toybox-0.7.0:  *   emake failed

For some reason we cannot reproduce locally (it only happens on these builders that are compiling many other packages at the same time).

Looking at the code (script/make.sh), we are wondering about your use of $(jobs -rp). Wouldn't it be more correct to add jobs to PENDING using $! right after you launch the job (do_loudly)?

Thanks!

busybox.exe for MSWindows is available, but not toybox.exe for MSWindows

busybox compiled for ms windows is available at
https://frippery.org/busybox/
https://github.com/rmyorston/busybox-w32

but i do not find any such native .exe binary for toybox. Is toybox not compilable/ported to MSwindows? If yes, then is there any manual for compiling it on MSWindows? If no, then When this port to MSwindows will be available?
I am more interested in readymade toybox.exe for MSWindows available for download somewhere

top batch mode does not free resources

mix.tb and plold->tb dot not get freed.
Which leads to improper behavior of the mode, data does not get updated.

toybox/toys/posix/ps.c

Line 1554 in e949f63

// In batch mode, we ignore the keyboard.

-size doesn't support c suffix (bytes)

$ echo Hello world >foo
$ find foo -size 12c
foo
$ ./toybox find foo -size 12c
find: not integer: 12c

switch_root implementation lacks complete functionality

The Toybox switch_root implementation lacks some important functionality.

Most importantly is actually moving /dev, /proc, /sys from the old root to the new root.

See the implementation below for recommendation of moving mounts
ref: https://github.com/karelzak/util-linux/blob/master/sys-utils/switch_root.c
ref: https://github.com/landley/toybox/blob/master/toys/other/switch_root.c

« sed -i (edit in place) » used as root doesn't preserve uid nor gid

Hi!

Samsung Galaxy S4 Mini Plus I9195I (CM 13 unofficial)
toybox version : 0.7.0-892c602b10e7-android

# ls -l
-rw-rw---- 1 u0_a201 u0_a201 4 2016-09-30 11:35 bb_test.txt
-rw-rw---- 1 u0_a201 u0_a201 4 2016-09-30 11:35 tb_test.txt

# sed -i s/foo/bar/ tb_test.txt (toybox)
# /system/xbin/sed -i s/foo/bar/ bb_test.txt (busybox)

# ls -l
-rw-rw---- 1 u0_a201 u0_a201 4 2016-09-30 11:39 bb_test.txt (busybox)
-rw-rw---- 1 root root 4 2016-09-30 11:39 tb_test.txt (toybox)

beastiebox(bsd license) has few commands which are not yet in toybox. Why not merge those parts into toybox

[1] http://beastiebox.sourceforge.net/ is another attempt which tries to be port some utilities of busybox. I think it has few commands implemented which are not in toybox, may be toybox can benefit by getting it integrated into toybox

fsck,fsck_ffs, ps, [, ping (fully implemented), less,more,sysctl,pfctl,wiconfig,traceroute(fully implemented) ,stty,reboot,halt,poweroff,ex,vi,fdisk,disklabel,tar,getty

these above ^ commands are in http://beastiebox.sourceforge.net/ (BSD licensed alternative of busybox), which are probably not (or not fully implemented) in toybox. So,we can probably merge it into toybox.

bzcat segfault

File here:
http://www.cs.utah.edu/~regehr/foo2.bz2
causes a segfault when run like this:
toybox bzcat foo2.bz2

xargs: exec ... Argument list too long

toybox 0.7.1

On some Android devices ([email protected]) using a chain of find .... | xargs .... stat ...., yields the error Argument list too long.

Some googlfu shows that this is likely related to the argument exceeding the kernels ARG_MAX value.
Androids ARG_MAX according to this 131072.

Manually limiting the xargs argument length using -s bytes fixes this for me.

I've looked at the code for xarg in toybox but could not figure out yet how the max argument length is determined.

Busybox doesn't seem to run into this issue using the same command on the same device. Busybox seems to have some sanity checks, adjusting the argument length if necessary.

Could there be an issue with how toybox's xarg determines the default argument length on Android?

linker failure for make menuconfig

kconfig/mconf: $(SHIPPED)
$(HOSTCC) -o $@ kconfig/mconf.c kconfig/zconf.tab.c
kconfig/lxdialog/*.c -lcurses -ltinfo -DCURSES_LOC="<ncurses.h>"
-DKBUILD_NO_NLS=1 -DPROJECT_NAME="$(KCONFIG_PROJECT)"

I had to add -ltinfo in kconfig/Makefile otherwise it was missing symbols. This was on Gentoo amd64 just ask if you need any more specifics! Thanks.

Can not cross compile on Max OS X (Mavericks)

make menuconfig works
make fails because a sed expression isn't understood by Mac OS X (and hence BSD) sed
It may be possible to have a different expression. If you can give me a little insight into what the expression is trying to do, I can have a go at sorting this out.

I'm really keen to get a minimal toybox working on other POSIX platforms, because it provides an excellent baseline for a consistent, secure and sensible set of tools to enable cross compilation toolchains. Oh, and not using autocruft gets it an extra star...

More details:-

maclaptop:toybox raph$ make
scripts/make.sh
Generate headers from toys/*/*.c...
Make generated/config.h from .config.
generated/flags.h sed: 2: "s/" *"//g;/^#/d;t clear ...": undefined label 'clear;:clear;s/"/"/p;t;s/\( [AB] \).*/\1 " "/p'
sed: 1: "s/ A / /;t pair;h;s/\([ ...": undefined label 'pair;h;s/\([^ ]*\).*/\1 " "/;x;b single;:pair;h;n;:single;s/[^ ]* B //;H;g;s/\n/ /;p'
clang: error: unable to execute command: Broken pipe: 13
clang: error: clang frontend command failed due to signal (use -v to see invocation)
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix
clang: note: diagnostic msg: PLEASE submit a bug report to http://developer.apple.com/bugreporter/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg: Error generating preprocessed source(s) - ignoring input from stdin.
clang: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs.
sed: 2: "s/" *"//g;/^#/d;t clear ...": undefined label 'clear;:clear;s/"/"/p;t;s/\( [AB] \).*/\1 " "/p'
clang: error: unable to execute command: Broken pipe: 13
clang: error: clang frontend command failed due to signal (use -v to see invocation)
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix
clang: note: diagnostic msg: PLEASE submit a bug report to http://developer.apple.com/bugreporter/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg: Error generating preprocessed source(s) - ignoring input from stdin.
clang: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs.
Makefile:19: recipe for target 'toybox' failed
make: *** [toybox] Error 1

Segfault using toybox stat on Android emulator

toybox stat is crashing on Android emulator running Android 5.1.1 (Lollipop)

root@generic:/data/local # ./toybox-armv6l stat script.sh                      
  File: `script.sh'
  Size: 14722    Blocks: 32  IO Blocks: 4096    regular file
Device: 1f01h/7937d  Inode: 14524    Links: 1
Access: (770/-rwxrwx---Segmentation fault

I tried with the latest release version from the toybox website for arm-v6l as well as a version built from latest git using your cross-compiler for armv6 from your homepage:

root@generic:/data/local # ./toybox stat toybox
  File: `toybox'
  Size: 467296   Blocks: 920     IO Blocks: 4096    regular file
Device: 1f01h/7937d  Inode: 14531    Links: 1
Access: (555/-r-xr-xr-xSegmentation fault

It seems that the emulator is running ARMv7l, but AFAIK it should be compatible to ARMv6l binaries:

root@generic:/ # cat /proc/cpuinfo
Processor   : ARMv7 Processor rev 0 (v7l)
BogoMIPS    : 766.77
Features    : swp half thumb fastmult vfp edsp neon vfpv3 tls 
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0xc08
CPU revision    : 0

Hardware    : Goldfish
Revision    : 0000
Serial      : 0000000000000000

the mount command behaves incorrectly

The traditional way to remount a filesystem is with:

# mount -o remount,rw /mountpoint

However this does not work in toybox.

There is a workaround, though.

# mount -o rw,remount /mountpoint

Any thoughts as to why the "correct" way doesn't work? You can find the "remount,rw" ordering in tons of documentation, yet it doesn't work in toybox.

ls applet doesn't take options if not build with CONFIG_LS_COLOR

buildvm@buildvm-VirtualBox:~/toybox_darken$ ./toybox_x86 --version
toybox 0.7.2-11-g382057f588fb

With CONFIG_LS_COLOR:

buildvm@buildvm-VirtualBox:~/toybox_darken$ ./toybox_x86 ls -l
total 1440
-rw-rw-r-- 1 buildvm buildvm    814 2016-08-22 03:19 0001-Fix-test-applet-randomly-applying-operator.patch
-rw-rw-r-- 1 buildvm buildvm    781 2016-08-22 03:19 0002-Fix-test-applet-NOT-operator.patch
-rw-rw-r-- 1 buildvm buildvm   5355 2016-12-05 01:18 Config.in
-rw-rw-r-- 1 buildvm buildvm    820 2016-08-14 15:08 LICENSE
-rw-rw-r-- 1 buildvm buildvm   3106 2016-12-05 01:18 Makefile
-rw-rw-r-- 1 buildvm buildvm   6999 2016-08-14 15:08 README
-rwxrwxr-x 1 buildvm buildvm    792 2016-08-12 03:09 build_for_sdm.sh
-rwxrwxr-x 1 buildvm buildvm    912 2016-08-12 11:25 build_for_sdm_bionic.sh
-rwxrwxr-x 1 buildvm buildvm    817 2016-08-12 11:37 build_for_sdm_musl.sh
-rw-rw-r-- 1 buildvm buildvm    954 2016-08-14 15:08 configure
drwxrwxr-x 4 buildvm buildvm   4096 2016-12-05 03:03 generated
drwxrwxr-x 3 buildvm buildvm   4096 2016-12-05 03:02 kconfig
drwxrwxr-x 2 buildvm buildvm   4096 2016-12-05 01:18 lib
-rw-rw-r-- 1 buildvm buildvm   6691 2016-12-05 01:18 main.c
drwxrwxr-x 2 buildvm buildvm   4096 2016-12-05 01:21 oldpatches
drwxrwxr-x 2 buildvm buildvm   4096 2016-12-05 01:18 scripts
drwxrwxr-x 3 buildvm buildvm   4096 2016-12-05 01:18 tests
-r-xr-xr-x 1 buildvm buildvm 428304 2016-12-05 03:03 toybox_arm
-r-xr-xr-x 1 buildvm buildvm 555992 2016-12-05 03:04 toybox_mips
-r-xr-xr-x 1 buildvm buildvm 390636 2016-12-05 03:02 toybox_x86
drwxrwxr-x 9 buildvm buildvm   4096 2016-08-14 15:08 toys
-rw-rw-r-- 1 buildvm buildvm   3400 2016-08-14 15:08 toys.h
drwxrwxr-x 2 buildvm buildvm   4096 2016-12-05 01:18 www

Without CONFIG_LS_COLOR:

buildvm@buildvm-VirtualBox:~/toybox_darken$ ./toybox_x86 ls -l
www          tests       generated                Makefile                                               
toys.h       scripts     configure                LICENSE                                                
toys         oldpatches  build_for_sdm_musl.sh    Config.in                                              
toybox_x86   main.c      build_for_sdm_bionic.sh  0002-Fix-test-applet-NOT-operator.patch                
toybox_mips  lib         build_for_sdm.sh         0001-Fix-test-applet-randomly-applying-operator.patch  
toybox_arm   kconfig     README

Seems to ignore the flag -l and with -lah it errors out:

buildvm@buildvm-VirtualBox:~/toybox_darken$ ./toybox_x86 ls -lah
usage: ls [-ACFHLRSZacdfhiklmnpqrstux1] [directory...]

list files

what to show:
-a  all files including .hidden    -b  escape nongraphic chars
-c  use ctime for timestamps       -d  directory, not contents
-i  inode number                   -p  put a '/' after dir names
-q  unprintable chars as '?'       -s  storage used (1024 byte units)
-u  use access time for timestamps -A  list all files but . and ..
-H  follow command line symlinks   -L  follow symlinks
-R  recursively list in subdirs    -F  append /dir *exe @sym |FIFO
-Z  security context

output formats:
-1  list one file per line         -C  columns (sorted vertically)
-g  like -l but no owner           -h  human readable sizes
-l  long (show full details)       -m  comma separated
-n  like -l but numeric uid/gid    -o  like -l but no group
-x  columns (horizontal sort)

sorting (default is alphabetical):
-f  unsorted    -r  reverse    -t  timestamp    -S  size

ls: Unknown option h
buildvm@buildvm-VirtualBox:~/toybox_darken$

Weirdly enough my custom config (e.g. https://github.com/d4rken/toybox/blob/sdmaid/.config), even causes the -l option to throw an error:

buildvm@buildvm-VirtualBox:~/toybox_darken$ ./toybox_x86 ls -l
usage: ls [-ACFHLRSZacdfhiklmnpqrstux1] [directory...]

list files

what to show:
-a  all files including .hidden    -b  escape nongraphic chars
-c  use ctime for timestamps       -d  directory, not contents
-i  inode number                   -p  put a '/' after dir names
-q  unprintable chars as '?'       -s  storage used (1024 byte units)
-u  use access time for timestamps -A  list all files but . and ..
-H  follow command line symlinks   -L  follow symlinks
-R  recursively list in subdirs    -F  append /dir *exe @sym |FIFO
-Z  security context

output formats:
-1  list one file per line         -C  columns (sorted vertically)
-g  like -l but no owner           -h  human readable sizes
-l  long (show full details)       -m  comma separated
-n  like -l but numeric uid/gid    -o  like -l but no group
-x  columns (horizontal sort)

sorting (default is alphabetical):
-f  unsorted    -r  reverse    -t  timestamp    -S  size

ls: Unknown option l

Large DNS responses cause host to segfault

sargun@sargun-dev:~/toybox$ generated/unstripped/toybox host value.testing.express
value.testing.express has address 3.1.1.10
value.testing.express has address 1.1.1.9
value.testing.express has address 3.1.1.5
value.testing.express has address 3.1.1.9
value.testing.express has address 1.1.1.8
value.testing.express has address 3.1.1.3
value.testing.express has address 1.1.1.12
value.testing.express has address 1.1.1.2
value.testing.express has address 2.1.1.12
value.testing.express has address 1.1.1.7
value.testing.express has address 1.1.1.3
value.testing.express has address 1.1.1.10
value.testing.express has address 2.1.1.1
value.testing.express has address 3.1.1.1
value.testing.express has address 2.1.1.11
value.testing.express has address 1.1.1.6
value.testing.express has address 2.1.1.2
value.testing.express has address 2.1.1.3
value.testing.express has address 2.1.1.4
value.testing.express has address 2.1.1.5
value.testing.express has address 1.1.1.11
value.testing.express has address 3.1.1.4
value.testing.express has address 2.1.1.8
value.testing.express has address 2.1.1.9
value.testing.express has address 2.1.1.10
value.testing.express has address 3.1.1.8
value.testing.express has address 3.1.1.11
value.testing.express has address 3.1.1.12
value.testing.express has address 3.1.1.2
value.testing.express has address 46.49.46.50
Segmentation fault (core dumped)

vs.

sargun@sargun-dev:~/toybox$ dig value.testing.express

; <<>> DiG 9.10.3-P4-Ubuntu <<>> value.testing.express
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42125
;; flags: qr rd ra; QUERY: 1, ANSWER: 36, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;value.testing.express.		IN	A

;; ANSWER SECTION:
value.testing.express.	299	IN	A	2.1.1.11
value.testing.express.	299	IN	A	1.1.1.4
value.testing.express.	299	IN	A	3.1.1.12
value.testing.express.	299	IN	A	2.1.1.8
value.testing.express.	299	IN	A	1.1.1.2
value.testing.express.	299	IN	A	2.1.1.10
value.testing.express.	299	IN	A	1.1.1.8
value.testing.express.	299	IN	A	1.1.1.6
value.testing.express.	299	IN	A	3.1.1.8
value.testing.express.	299	IN	A	3.1.1.7
value.testing.express.	299	IN	A	1.1.1.10
value.testing.express.	299	IN	A	1.1.1.9
value.testing.express.	299	IN	A	2.1.1.1
value.testing.express.	299	IN	A	1.1.1.5
value.testing.express.	299	IN	A	3.1.1.11
value.testing.express.	299	IN	A	2.1.1.12
value.testing.express.	299	IN	A	2.1.1.7
value.testing.express.	299	IN	A	1.1.1.7
value.testing.express.	299	IN	A	2.1.1.4
value.testing.express.	299	IN	A	3.1.1.4
value.testing.express.	299	IN	A	2.1.1.6
value.testing.express.	299	IN	A	2.1.1.2
value.testing.express.	299	IN	A	3.1.1.10
value.testing.express.	299	IN	A	3.1.1.1
value.testing.express.	299	IN	A	2.1.1.9
value.testing.express.	299	IN	A	1.1.1.12
value.testing.express.	299	IN	A	3.1.1.9
value.testing.express.	299	IN	A	1.1.1.11
value.testing.express.	299	IN	A	3.1.1.3
value.testing.express.	299	IN	A	3.1.1.2
value.testing.express.	299	IN	A	2.1.1.5
value.testing.express.	299	IN	A	3.1.1.5
value.testing.express.	299	IN	A	3.1.1.6
value.testing.express.	299	IN	A	2.1.1.3
value.testing.express.	299	IN	A	1.1.1.3
value.testing.express.	299	IN	A	1.1.1.1

;; Query time: 33 msec
;; SERVER: 169.254.169.254#53(169.254.169.254)
;; WHEN: Wed Nov 23 18:22:04 UTC 2016
;; MSG SIZE  rcvd: 626

find -name doesn't match command line arguments with multiple components

$ mkdir -p foo/bar
$ find foo/bar -name bar
foo/bar
$ ./toybox find foo/bar -name bar
$

$ touch foo
$ find -path '*foo'
./foo
$ ./toybox find -path '*foo'
$

Looks like FNM_PATHNAME is being set?

need stty, tr, gawk to run inxi

Hi, this is a great toolbox. Thanks! However it still misses some tools for inxi to run. They are stty, tr and gawk.

find -name doesn't ignore trailing slashes

$ mkdir foo
$ find foo/ -name foo
foo/
$ ./toybox find foo/ -name foo
$