Code Monkey home page Code Monkey logo

pydk's Introduction

pydk : Android generic ( 4.4/32 api19 , 5+/64 api21) + emscripten tip of tree/incoming

PyDK (Python Developpement Kit) allows you to efforless build CPython , Panda3D and gives you prebuilt libraries ready to use pip modules and runtime loaders.

for use on :

  • Android starting with API19(kitkat 4.4)

  • Web browsers, Electron.

There's two gitter channels for support and general interest in supporting the platforms for cpython: for Android: https://gitter.im/Wasm-Python/pydk Browser: https://gitter.im/Wasm-Python/community

embedding is one part of the process and packaging is another : embedding is not specific to PyDK You are welcome to discuss how to improve Python embedding on mobile/web/wasi no matter which packager you prefer (Panda3D/Beeware/Kivy).

PyDK packaging ability is minimal, just enough to run tests. The main goal is and will remain building interpreter + extensions and support libs.

PyDK was made to make Python core adjustments easier Its next big step is to package only testsuites and provides local wheels.

pydk : Android generic ( 4.4/32 api19 , 5+/64 api21)

Requirements:
    * C-Python ® 3.6+ installed
    * clang/clang++ 8+ full build env <- this is very important.
    * git, wget, patch
    * android-sdk + android-ndk 21 or 23
    * all Panda3D requirements
    * about 16 GiB free disk space
    * 4 GiB MEM-SWAP for panda3d linking

( for debian/ubuntu/mint maybe just have a look at the .github/workflows/blank.yml CI apt-get lines )

Clone this repo and go into folder pydk

add sdk as "android-sdk" and ndk as "android-sdk/ndk-bundle" (ndk21 or 23) in the folder, symlink is ok too

Tehen launch :

ARCHITECTURES="armeabi-v7a arm64-v8a x86 x86_64" ./pydk-all.sh

use PYMINOR/PYMICRO to pick the CPython version ( look in sources.* for what's available as i'm often dragging behind )

To launch step by step use instead for eg CPython 3.7.13 :

STEP=true ARCHITECTURES="armeabi-v7a arm64-v8a x86 x86_64" PYMINOR=7 PYMICRO=13 ./pydk-all.sh

sources used:
  +  bzip2 from : https://sourceware.org/pub/bzip2/bzip2-1.0.8.tar.gz
  +  lzma from : https://tukaani.org/xz/xz-5.2.4.tar.bz2
  +  libffi from : http://sourceware.org/pub/libffi/libffi-3.3.tar.gz + Brion Vibber patches
  +  sqlite3 from : https://github.com/azadkuh/sqlite-amalgamation.git
  +  openssl from : https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
  +  python3 from : https://github.com/python/cpython/archive/v${PYVER}.tar.gz
  +  freetype2 from : https://download.savannah.gnu.org/releases/freetype/freetype-2.10.0.tar.bz2
  +  harfbuzz from : https://github.com/harfbuzz/harfbuzz.git
  +  ft2_hb from : https://download.savannah.gnu.org/releases/freetype/freetype-2.10.0.tar.bz2
  +  bullet3 from : https://github.com/bulletphysics/bullet3/archive/2.89.tar.gz
  +  openal from : https://github.com/pmp-p/pydk-openal-soft.git
  +  ogg from : http://downloads.xiph.org/releases/ogg/libogg-1.3.4.tar.xz
  +  vorbis from : https://github.com/xiph/vorbis.git
  +  panda3d from : GIT + android patches + webgl port branch merged
  +  SDL2/SDL2_image/SDL2_mixer/SDL2_ttf stable from libsdl.org

NOTE: now has limited pip support and will detect your ./projects/*/requirements.txt. and will do its best to package them for an android apk.

You can also use pip-cross via the various venv shell.[ARCHITECTURE].sh (to be sourced) ( when in the shell you have python3-xxxxxx and pip3-xxxxxx cross compiler scripts )

NOTE: an emscripten android project emulation is actually brewing. it allows deploying your apk untouched in browsers.

Beerware : PayPayl

la rache

A simple python created button calling Panda3D engine to render on a python created View :

p3d1(https://www.panda3d.org)

p3d2(https://www.panda3d.org)

pydk : Emscripten

Requirements : GNU/Linux os , internet connection.

ARCHITECTURES="wasm" ./pydk-all.sh

demo : https://pmp-p.github.io/panda3d-next/py3/

p3d3(https://pmp-p.github.io/panda3d-next/py3/)

ref:

https://pythondev.readthedocs.io/android.html

https://pythondev.readthedocs.io/wasm.html

--

pydk's People

Contributors

dylancheetah avatar pmp-p avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pydk's Issues

No releases?

Looks very nice. I would like to test it on my kitkat device but I cannot find any releases. Sorry, no laptop's bad so I can't build from scratch

Panda3d CMake Errors

CMake Error at cmake/macros/PackageConfig.cmake:151 (message): NOT FOUND: GLES2. Disable HAVE_GLES2 to continue.

CMake Error at panda/src/audiotraits/CMakeLists.txt:4 (message): You must have an audio backend for audio support! Turn off HAVE_AUDIO to ignore this.

Am I missing something?

How to build apk

The pydk-all.sh completes but there is no apk. Is there another step? Also, I'm trying to use the usrinaengine which has a dependency on Pillow which seems to be x86 only. Is there a way to get Pillow working?

ModuleNotFoundError: No module named '_sysconfigdata__android_aarch64_21_android'

File "/home/osboxes/Desktop/Code/pydk/aosp/apkroot-arm64-v8a/usr/lib/python3.8/sysconfig.py", line 422, in _init_posix _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0) ModuleNotFoundError: No module named '_sysconfigdata__android_aarch64_21_android' ERROR: pip install /home/osboxes/Desktop/Code/pydk/projects/requirements.txt failed
Any ideas?

No releases?

Looks very nice.
I would like to test it on my kitkat device but I cannot find any releases.

//(Sorry if it's not really an issue)

How to build?

I would like to build cpython 3.x for the browser. I don't know how to proceed.

configure: error: C compiler cannot create executables

configure: error: in `/home/osboxes/Desktop/Code/pydk/src/python3-prefix/src/python3': configure: error: C compiler cannot create executables See `config.log' for more details make[2]: *** [CMakeFiles/python3.dir/build.make:114: python3-prefix/src/python3-stamp/python3-configure] Error 77 make[1]: *** [CMakeFiles/Makefile2:378: CMakeFiles/python3.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs....

I can't locate config.log, and I noticed python3-configure doesn't exist.

pip modules and platform MULTIARCH identification

Following the pattern <cpuarch>-<libc-api>-<libc-abi> , values defined for sysconfig.get_platform() and sysconfig.get_config_vars('MULTIARCH'):

shell.wasm.sh : wasm_1_emscripten or wasm_1pic_emscripten
TODO: iirc there are variants API/ABI wasm/asm.js flavours regarding asyncify and threading for emscripten.

shell.armeabi-v7a.sh : armv7a_19_android

shell.arm64-v8a.sh : aarch64_21_android

shell.x86.sh : i686_19_android

shell.x86_64.sh : x86_64_21_android

and also :

shell.wasi64.sh : wasm64_1_wasi or wasm64_1pic_wasi

shell.wasi32.sh : wasm32_1_wasi or wasm32_1pic_wasi

shell.asm.js.sh : asmjs_1_fastcomp

Exemple of a pip 3.9 wheel build session :

[PyDK:armeabi-v7a] /data/cross/pydk $ pip3-armeabi-v7a wheel numpy
Collecting numpy
  Using cached numpy-1.19.4.zip (7.3 MB)
Building wheels for collected packages: numpy
  Building wheel for numpy (setup.py) ... done
  Created wheel for numpy: filename=numpy-1.19.4-cp39-cp39-armv7a_19_android.whl size=13205762 sha256=98b79f11fdc6f2c30042551145d5814249f27ebc1921ac26f2bcc5d1658ffb01
  Stored in directory: /cache/pip/wheels/6f/d7/25/0fe5a4a476d4881c00f6e6914d4d40f9c42cfe334faafd79a3
Successfully built numpy
[PyDK:armeabi-v7a] /data/cross/pydk $ file numpy-1.19.4-cp39-cp39-armv7a_19_android.whl 
numpy-1.19.4-cp39-cp39-armv7a_19_android.whl: Zip archive data, at least v2.0 to extract
[PyDK:armeabi-v7a] /data/cross/pydk $ 

skip configure step for cpython/wasm

the configure part of cpython wasm is a real time bottleneck for automated builds
but emsdk is a quite deterministic environnement, it should be possible to provide a config.cache
to speed up things

Cannot instantiate Showbase

There seems to be another error when I try to import Showbase.
? libmultify.so
? libpzip.so
? libinterrogate.so
? libinterrogate_module.so
? libpunzip.so
? libparse_file.so
? libtest_interrogate.so
Traceback (most recent call last):
File "test1.py", line 5, in
game=Window()
File "test1.py", line 4, in init
ShowBase.init(self)
File "/data/data/u.r/usr/lib/python3/direct/showbase/ShowBase.py", line 388, in init
if info:info(self,"Android found, will try to reuse main window")
NameError: name 'info' is not defined

I tried commenting out the affected lines since they were just logging functions. Then, I got a second error.

Traceback (most recent call last):
File "test1.py", line 5, in
game=Window()
File "test1.py", line 4, in init
ShowBase.init(self)
File "/data/data/u.r/usr/lib/python3/direct/showbase/ShowBase.py", line 390, in init
if not hasattr(RunTime,'showbase'):
NameError: name 'RunTime' is not defined

I'm not very conversant with the panda3d engine(only started using it when I saw this project) but a missing "Runtime" seems like a major issue. Neither "Runtime" nor "info" appear in the official panda3d repo and going through ShowBase.py, there seems to be no explicit import statement. I also searched this repository but no breakthroughs.

android: dynamic modules loading and packages namespaces

current cpython builds produce modules which are normally dlopen'ed from package folder. But on android dynamic libs must follow lib*.so unversionned name pattern and stay in <apkroot>/lib special folder ( arch dependant )
prefixing modules with "lib" is not enough there will be collisions when :

module_a/tool.so => lib/libtool-<soabi>.so
module_b/tool.so => lib/libtool-<soabi>.so
( in case of android not recent enough and no soname set ).

this should be fixed in pip/setuptools or whereever is the module builder in cpython, not with importlib hacks + patchelf post processing. maybe with that kind of layout :

module_a/tool.so => lib/libmodule_a-tool-<soabi>.so
module_b/tool.so => lib/libmodule_b-tool-<soabi>.so

sidenote: only root can load .so from outside <apkroot>/lib special folder, so avoid running modules testsuites via adb shell that may use elevated privileges.

open questions: how does termux really handle dlopen(), is it relying on LD_LIBRARY_PATH / LD_PRELOAD which are not allowed for java apk ?

cf: https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md

pip wheel.py is forcing compileall

disregarding PYTHONDONTWRITEBYTECODE

around 687 replace

if pycompile:

by

if not os.environ.get('PYTHONDONTWRITEBYTECODE', not pycompile):

freetype/harfbuzz may conflict with os bundled libs

Not sure if this is still alive but on an android 4.4.2 kitkat, panda3d causes this error
back (most recent call last):
File "", line 1, in
ImportError: dlopen failed: could not load library "libpanda.so" needed by "core.cpython-37m.so"; caused by could not load library "libpandaexpress.so" needed by "libpanda.so"; caused by could not load library "libandroid.so" needed by "libpandaexpress.so"; caused by could not load library "libandroid_runtime.so" needed by "libandroid.so"; caused by cannot locate symbol "hb_utf16_script_run_next" referenced by "libandroid_runtime.so"...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.