Code Monkey home page Code Monkey logo

source's Introduction

KBOX is a set of common, Linux console utilities ported to Android, in such a way that the user is presented with a reasonably Linux-like command-line environment. It works without rooting or any other warrany-busting strategies. Packages available so far include heavyweights like Perl, Python 3, and gcc; but many simpler utilties are also available. KBOX is designed to be installed on top of an existing Android terminal emulator, using a relatively simple command-line process. After installing the base system, further packages can be downloaded and installed.

This latest release of KBOX, version 4, offers the same utilities that were available in version 3, but the build process has been completely redesigned, to separate the source of the build from the source of the utilties themselves. People who have wanted to contribute to the KBOX effort have frequently asked if they could have the "source for KBOX"; until now, however, there has been no such thing.

The main end-user documentation for KBOX is here:

http://kevinboone.net/kbox.html

This document provides technical information for people who might want to get involved in developing the project.

Structure of this archive

This archive consists of two main parts -- the directories base and packages. base contains the parts needed to build the base installer, whilst "packages" consists of the build scripts for individual packages, one per directory. Each directory, in either section, contains a script build.sh that does the build, based on settings in env.sh. This defines the important build parameters -- the location of the C compiler, where downloaded tarballs are to be cached, where they are to be unpacked, etc. It should not be necessary, in practice, to change anything outside of env.sh to build an existing package.

The base installer section consists of three directories, each with its own build script. The scripts must be run in the right order, so there is a "master" build script in base/. This won't build any packages -- at present these must be built individually, and there is a proper order in which to build them (see the file doc/build_order.txt).

Prerequisites

To build the whole KBOX system requires a fairly substantial set of development tools, and it's difficult to predict exactly what they all are -- anything that was installed on my workstation before I started work on KBOX, I probably wouldn't even know about. See doc/prerequisites.txt for a list of dependencies that I'm sure about.

Most obviously you'll need C/C++ compilers and all the usual, associated tools -- both for compiling code for the Android target, and for the build machine. For the Android target, it's probably best to use the official Android NDK from Google, as it has the (more or less) correct libraries and headers in place from the start. At present, I am using release r14b. You can expect, sadly, to have to change things in the code if a different version is used. It will be necessary to create a "standalone toolchain" from the NDK for each Android architecture (ARM, x86) to be supported.

The build process

Each build script carries out the same basic set of steps.

1. Retrieves the source code from (ideally) a repository, and caches it in the location indicated by $STAGING/tarballs. In a very few cases the source is unpacked from a tarball bundled into this archive. I have only done this where I can no longer find a reliable on-line repository for the code.

2. Unpacks the source into a directory under $STAGING, unless that directory already exists. Note that no checks are made at this stage beyond that the directory exists. The files in it might be completely mangled by previous unsuccessful build attemps, but they won't be restored. To clean the build, you will need to delete the directory in $STAGING. Note that everything in $STAGING is intended to be ephemeral -- it can all be recreated by running the build scripts. Any changes made in this area during the course of making an application work must be converted into patches that can be applied when the source is next unpacked.

3. Patches the configure script, or similar, if there is one, and if it is necessary. This is only necessary in a few cases, when the configuration script won't run at all.

4. Runs configure (or similar) with the appropiate arguments for cross-compiling.

5. If necessary patches the Makefiles generated by configure. The usual reason for needing to do this is because either (a) configure has used information from the build system, rather than about the target system (which it has no way of knowing); or (b) the Google NDK headers and libraries don't match, so functions seem to be present when they aren't.

6. Patches the source code. This is nearly always necessary. Sometimes the patches are minor, but sometimes they are almost applications in their own right (Perl).

7. Runs make, or equivalent, sometimes with specific environment settings.

8. Runs make install, with environment settings to direct the output to a directory called image in the build directory.

9. Builds an IPK package from the image directory. A working directory called out is created during this process.

In practice, some packages require significant departures from this process. Python 3, for example, has to be built completely twice over, because the loadable modules require different build settings from the main binary.

10. Populates the Android NDK sysroot. This applies to libraries. Once build, the libraries static objects and C/C++ headers are copied into the sysroot. This way they are available to be used by packages built later. Because of these dependencies, there is a recommended build order (see doc/build_order.txt).

11. Copies the final package to a location identified by $DIST.

Packages missing from this archive

Not everything available for the KBOX distribution can be built using these scripts. Some packages were contributed in more-or-less complete form, using methods that only their contributors know. In some cases, they were build on an Android device -- this process has a lot to recommend it, but it doesn't lend itself to centralizing in a repository.

Build nasties

The build scripts do make some attempt to stop if a particular step fails. However, it isn't always possible to tell when it has failed, and you have to be prepared for the fact that some scripts will carry on uselessly after a failure. Worse, some parts of some builds actually fail, but we still get a usable build (because, for example, we don't need all of it). In such cases the script will always carry on because it can't tell a "good" failure from a "bad" one.

Some build scripts require patches to be applied to the source. In some cases, these patches cannot be applied twice. So if a build fails, it may be fruitless to run the build script again after fixing whatever the problem was -- the source may no longer be patchable. In such cases, it is necessary to restart the whole build by deleting the build directory in $STAGING.

source's People

Contributors

giuliolunati avatar kevinboone avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

source's Issues

Can't run Dalvik/ART JVM after installing openssl library

The JVM segfaults. logcat reveals that a library fails to load because of missing SSL symbols. These symbols are (presumably) in the Google BoringSSL library, but not in OpenSSL. No obvious solution presents itself. The ugly workaround is to switch libssl.so and libcryto.so in the /usr/lib directory between the KBOX versions and the BoringSSL versions in /system/lib as required. Ugh.

rsync crash

When I try to transfer files with rsync I get error:
invalid address 0xxxxxx passed to free: value not allocated
(Android 7.1.2, lineage-os-14)

symbol not found in libssl.so

Trying git clone https://....

  1. error:
    CANNOT LINK EXECUTABLE: cannot locate symbol "SSL_COMP_free_compression_methods" referenced by git-remote-https
  2. installed openssl package
    then error:
    CANNOT LINK EXECUTABLE DEPENDENCIES: cannot locate symbol "X509_free" referenced by "libssl.so".

WARNING: linker: unused DT entry

andii_nr wrote:
hii kevinboon
help me
why kbox4 busybox warning linker like this ?

u0_a106@great_a900f:/ $ su
root@great_a900f:/ # cd /data/data/jackpal.androidterm
/kbox4-install-base <
WARNING: linker: ./kbox4-install-base: unused DT entry: type 0x6ffffffe arg 0xae4
WARNING: linker: ./kbox4-install-base: unused DT entry: type 0x6fffffff arg 0x1
UnZipSFX 6.00 of 20 April 2009, by Info-ZIP (http://www.info-zip.org).
creating: kbox4-base-installer/
inflating: kbox4-base-installer/install.sh
inflating: kbox4-base-installer/etc_profile
inflating: kbox4-base-installer/install-using-busybox.sh
inflating: kbox4-base-installer/libfakechroot.so
inflating: kbox4-base-installer/busybox
inflating: kbox4-base-installer/kbox_shell

Auto-run command: ./kbox4-base-installer/install.sh
Execute this command? [y/n] y
WARNING: linker: ./kbox4-base-installer/busybox: unused DT entry: type 0x6ffffffe arg 0x2e28
WARNING: linker: ./kbox4-base-installer/busybox: unused DT entry: type 0x6fffffff arg 0x2
....

Installer issue

'cannot link executable: cannot find verneed/verdef for version index=2 referenced by symbol "__sF" at ./kbox4-install-base'
on Android 6.0.1 on a Galaxy S5

KBOX4 not working on Android 8 (oreo)

Installation is succesfull with "not-cleaned" installer, but after starting kbox_shell it fail with message "segmentation fault". Problem is maybe around fakechroot.so, I tried to recompile kbox_shell - when I removed LD_PRELOAD from env, bash started without error.

cannot locate symbol "signal"

For Asus ME302KL Android 4.2.2 the installation elf binary of kbox4 yields the following error:
soinfo_relocate(linker.cpp:975): cannot locate symbol "signal" referenced by "./kbox4-install-base"...CANNOT LINK EXECUTABLE
It doesn't matter if I try the normal installer or the "cleaned" one.

"Can't create backup status file"

Seen when installing a new package. This seems to be a defect in the Busybox dpkg implementation. A directory gets created, /var/lib/dpkg/udeb.status.backup, which dpkg then tries to delete as a file. The workaround is to delete the directory manually.

"Cleaned" base installer fails on Xperia XZ with Android 7

Binaries compiled for Arm with version 14 of the Google NDK produce a slew of irritating (but harmless) error messages are about "unused DT entry". I have been using an open-source utility called "android-elf-cleaner" to remove the offending symbols -- but so far only from the base installer, not packages. There is a KBOX4 version of the utility so users can "clean" offending binaries on-device if the error messages are troublesome. Cleaning the base parts (busybox, etc) on-device is much more troublesome.

Unfortunately, it seems that android-elf-cleaner produces binaries that fail to load on the Xperia XZ (at least) with Android 7 (at least). I suspect this is an A7 issue, and not a device issue, but it's hard to be sure with only one report. Anyway, this means that in the long term we can't use android-elf-cleaner to remove the annoying error messages on A7. Interestingly, the symbols that the cleaner removes cause no problems on A7 -- presumably the compiler puts them there for a reason.

At present I don't think this is urgent, as there are few A7 devices out there. However, I think it would be advisable to halt plans to apply android-elf-cleaner to the entire distribution, unless we want to maintain a complete separate Android 7 distribution :/

odd error in ANDROID N(7)

Tried it many times, still the same error, no way to bypass it without rewritting the installer

HWVNS-H:/ $ cd /data/data/jackpal.androidterm
d/kbox4-install-base-arm > kbox4-install-base <
hmod 755 kbox4-install-base <
/kbox4-install-base <
CANNOT LINK EXECUTABLE "./kbox4-install-base": cannot find verneed/verdef for version index=2 referenced by symbol "__sF" at "/data/data/jackpal.androidterm/kbox4-install-base"
Aborted
kbox4-install-base <
/system/bin/sh: .kbox4-install-base: not found
127|HWVNS-H:/data/data/jackpal.androidterm $

curl seems to fail on install

I'm using the kbox installer as a solution to #10.

$> dpkg -i /sdcard/Download/curl-7.54.0_kbox4_arm.deb
Unpacking curl from (/path/to/curl)
Setting up curl (7.54.0)....
dkpg can't create backup status file

When i simply try to run curl...

CANNOT LINK EXECUTABLE "curl": cannot locate symbol "UI_get_string_type" referenced by "/data/data/jackpal.androidterm/kbox4/usr/binm/curl"....
aborted

Multiple issues with git and dropbear

kbox$ sshd CANNOT LINK EXECUTABLE: cannot locate symbol "OpenSSL_add_all_algorithms" referenced by "sshd"... page record for 0xb6f5801c was not found (block_size=16) kbox$ ssh-keygen CANNOT LINK EXECUTABLE: cannot locate symbol "OpenSSL_add_all_algorithms" referenced by "ssh-keygen"... page record for 0xb6ec201c was not found (block_size=16) kbox$ git clone https://github.com/kbox4/source Cloning into 'source'... warning: templates not found /usr/share/git-core/templates fatal: unable to access 'https://github.com/kbox4/source/': error setting certificate verify locations: CAfile: /usr/share/certificates/cacert.pem

OpenSsl is installed, android version:6 lineage OS 13..

Updated from kbox3 which was working perfectly :(

kbox4-install-base-i686 (cleaned or not) errors during execution on Android 6.0.1 x86_64

root@sofia3gr_car:/data # ./kbox4-install-base-i686 CANNOT LINK EXECUTABLE: cannot find verneed/verdef for version index=2 referenced by symbol "__sF" at "./kbox4-install-base-i686"
1|root@sofia3gr_car:/data # uname -a
Linux localhost 3.14.0 #25 SMP PREEMPT Wed Jan 10 18:02:49 CST 2018 x86_64

I tried using the "uncleaned" version posted in another issue, but that one failed as well.

KBOX4 fails on Android 9

As it is built for KBOX4, all operations on busybox on Android 9 fail with a "bad system call" error message. There are a few other report of similar problems in other places -- KBOX4 is not the only thing that tried to use Busbox on Android -- but it isn't clear what the solution is, if there is one. This isn't a general problem with the Google Android NDK, because the Busybox unpacker -- which is built with the same NDK using the same switches as Busybox -- does run correctly (although the irritating distiction between "cleaned" and "not cleaned" versions still exists). However, the unpacker invokes a Busybox script to do the rest of the installation, and that fails.

I do not know if this is the same problem that others are reporting on Android 8.

I don't see any obvious solution here, except to debug BB laboriously, looking for the failing system call. Maybe it is something crucial that is broken, and maybe it can be worked around -- at this stage, I really don't know.

Provide a stub library for common user API features, etc/

Almost every Linux utility seems to use getpwent() and related calls. Rather than stubbing these out in every single build, it would be nicer to have (stub) implementations in a library that could be linked. It would need to have its own .h files, of course, and there would need to be some way to link the library, both during configuration and when building the final executable.

Need a proper repository for binaries

...other than Kevin's personal website. I thought we could just create a binary repository on github, but it chokes with 100Mb+ files. I don't mind having the binaries on my website, but it does limit maintenance to, well, me. Perhaps there is some free web space somewhere that would do the trick? It would need to be something that accepted batch uploads by ftp or rsync or something like that. Ideally I'd like to get the HTML documentation off my personal website as well.

Error installing Entware

I tried to install opkg-based entware in the root directory, with same instructions they have provided for a rooted Android phone.

It doesn't work.. The opkg binary can't be executed, even with appropriate permissions.

Is it a problem with fakechroot or Kbox4 itself? Can anyone find a workaround? Adding Entware/OPKG as a package manager will be nice...

Compile C++ code: library "libstdc++.so.6" not found

I am trying to compile a very simple C++ program on my Android 5.0 device using kbox4's gcc:

#include <iostream>
int main(int argc, char* argv[])
{
    std::cout << "Hello world!" << std::endl;
    return 0;
}

I can successfully compile and link it using:

$ g++ -c test.cpp
$ g++ test.o -o test

Unfortunately, when I attempt to run it, it gives me an error that I do not know how to fix:

$ ./test
CANNOT LINK EXECUTABLE DEPENDENCIES: library "libstdc++.so.6" not found

I can see that kbox4 has libstdc++.so.6 inside /usr/gcc-4.9-pie/lib/, but I can not figure out how to get it to work. Making a similar hello world test program in C with stdio.h and printf works fine, just C++ seems to fail.

groff does not build cleanly for i686

/bin/sh: /home/kevin/lib/kbox4_staging/i686/groff-1.22.2/src/roff/groff/groff: /system/bin/linker: bad ELF interpreter: No such file or directory

Of course, we can't run "groff" here, because it is built for Android. I wonder if the reason that the build tries to run it is because the "host" and "build" platforms are both i-something? This problem does not arise with ARM, so far as I can tell. I presume it isn't necessary, and that the build will still have enough of groff to work adequately for KBOX purposes.

man pages are not formatted properly

man pages are not formatted properly by groff -- there are a bunch of terminal
codes in the page. I don't think the problem is with groff itself, because I
tested that it can format man page text correctly on the command line. Rather,
I think the problem is in the way that busybox man.c formats man pages

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.