Code Monkey home page Code Monkey logo

python-appimage's Introduction

AppImage distributions of Python

Ready to use AppImages of Python are available as GitHub releases.

Quickstart

We provide relocatable Python runtimes as AppImages for Linux systems. These runtimes are extracted from Manylinux Docker images, and they are available as GitHub releases. Our Python AppImages are updated weekly, on every Sunday.

Instructions for installing and running Python AppImages are provided on Read the Docs.

In addition, the online documentation describes the python-appimage utility, for application developers. This utility can facilitate the building of Python apps, given an existing Python AppImage and a recipe folder. Examples of recipes are available from GitHub.

Projects using python-appimage

  • grand/python - Contained, portable and modern python for GRAND running from an AppImage
  • rever - Cross-platform software release tool.
  • ssh-mitm - ssh mitm server for security audits
  • xonsh - Python-powered, cross-platform, Unix-gazing shell language and command prompt
  • xxh - Bring your favorite shell wherever you go through the ssh

License

The python-appimage package (A) is under the GNU GPLv3 license, except for files located under python_appimage/data which are MIT licensed. Thus, the produced Manylinux Python AppImages (B) are not GPL'd. They contain a CPython distribution that is (mostly) under the PSF license. Other parts of B (e.g. AppRun) are under the MIT license.

python-appimage's People

Contributors

anki-code avatar dotzborro avatar manfred-kaiser avatar mrf345 avatar niess avatar srevinsaju avatar vlad-touchlab avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

python-appimage's Issues

Build Python on MacOS

Hi! Thank you for your big work!

Is there a way to build Python on MacOS?

My idea is about to using unpacked AppImage with Python that can be run on MacOS. In this case I can build zip archive with the same structure as AppImage and use it on MacOS for xxh project.

What do you think?

Thanks!

xonsh appimage stop after first command

Hi! Thank you for the python-appimage! It's straight forward!

I tested xonsh appimage inside xxh and it worked. Now I try to run xonsh appimage as is and I've got unexpected behavior. I tried linuxdeploy-plugin-python xonsh appimage version and got the same.

  1. xonsh stops after any command and freeze process:
bash
mkdir -p /tmp/test && cd /tmp/test
git clone https://github.com/niess/python-appimage && cd python-appimage
python -m python_appimage build app applications/xonsh
./xonsh-x86_64.AppImage
xonsh> echo 123
[1]+  Stopped                 ./xonsh-x86_64.AppImage

Failback to /bin/sh
Traceback (most recent call last):
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 24189, in main
    return main_xonsh(args)
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 24218, in main_xonsh
    shell.shell.cmdloop()
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/xonsh/ptk2/shell.py", line 197, in cmdloop
    line = self.singleline(auto_suggest=auto_suggest)
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/xonsh/ptk2/shell.py", line 166, in singleline
    line = self.prompter.prompt(**prompt_args)

  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/prompt_toolkit/shortcuts/prompt.py", line 994, in prompt
    return self.app.run(set_exception_handler=set_exception_handler)
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 811, in run
    return loop.run_until_complete(
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 778, in run_async
    return await _run_async2()
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 760, in _run_async2
    result = await _run_async()
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 682, in _run_async
    with self.input.raw_mode():
  File "/tmp/.mount_xonsh-DFQ6bT/usr/python/lib/python3.8/site-packages/prompt_toolkit/input/vt100.py", line 259, in __enter__
    termios.tcsetattr(self.fileno, termios.TCSANOW, newattr)
termios.error: (5, 'Input/output error')

  1. And the process not really closed:
$ ps -ax | grep xonsh
  862 pts/0    T      0:00 /bin/bash /tmp/.mount_xonsh-10lrWf/AppRun
  866 ?        Ssl    0:00 ./xonsh-x86_64.AppImage
  869 pts/0    T      0:00 /bin/bash /tmp/.mount_xonsh-10lrWf/usr/bin/python3.8 -u -c from xonsh.main import main; main()
  882 pts/0    S      0:00 /home/ubuntu/python-appimage/xonsh-x86_64.AppImage -u -c from xonsh.main import main; main()
  1. And when I try to type any commands in the bash terminal where I run ./xonsh-x86_64.AppImage I've got endless errors:
Traceback (most recent call last):
  File "/tmp/.mount_xonsh-10lrWf/opt/python3.8/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 24206, in func_sig_ttin_ttou
    def func_sig_ttin_ttou(n, f):
BlockingIOError: [Errno 11] Resource temporarily unavailable
Exception ignored when trying to write to the signal wakeup fd:
  1. After kill xonsh process the bash returns to ordinary work.

If you can help it will be great. Thanks!

Why do the old Python versions get rebuilt so often?

Hi @niess, thank you for this extremely useful project, highly appreciated. 👍
Brilliant idea to extract the manylinx Python from the official Docker container, hats off!

Just a question, why do the old Python versions get rebuilt so often? Isn't it a little wasteful?

My example of building an appimage is stop working

Hi @niess !

Could you please take a look into this case:

cd /tmp 
# this is my test example that was working before
git clone https://github.com/anki-code/appimage-test
mv appimage-test rvr
cd rvr
python -m python_appimage build app --python-version 3.8 appimage

And I've got:

python -m python_appimage build app --python-version 3.8 appimage
[2021-04-30 20:22:17,349] EXTRACT  python3.8.9-cp38-cp38-manylinux1_x86_64.AppImage
[2021-04-30 20:22:19,682] BUNDLE   xonsh.desktop
[2021-04-30 20:22:19,683] BUNDLE   xonsh.appdata.xml
[2021-04-30 20:22:22,572] BUNDLE   /tmp/rvr
Traceback (most recent call last):
  File "/opt/miniconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/miniconda3/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/miniconda3/lib/python3.8/site-packages/python_appimage/__main__.py", line 109, in <module>
    main()
  File "/opt/miniconda3/lib/python3.8/site-packages/python_appimage/__main__.py", line 105, in main
    command.execute(*command._unpack_args(args))
  File "/opt/miniconda3/lib/python3.8/site-packages/python_appimage/commands/build/app.py", line 240, in execute
    system(('./AppDir/AppRun', '-m', 'pip', 'install', '-U',
  File "/opt/miniconda3/lib/python3.8/site-packages/python_appimage/utils/system.py", line 42, in system
    raise RuntimeError(err)
RuntimeError: WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /tmp/python-appimage-iw219et5/AppDir/opt/python3.8/include/python3.8/UNKNOWN
sysconfig: /tmp/python-appimage-iw219et5/AppDir/opt/python3.8/include/python3.8
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /tmp/python-appimage-iw219et5/AppDir/opt/python3.8/include/python3.8/UNKNOWN
sysconfig: /tmp/python-appimage-iw219et5/AppDir/opt/python3.8/include/python3.8
WARNING: Additional context:
user = False
home = None
root = None
prefix = None

Could you please give an advice what I'm doing wrong or probably this is an upstream issue? Because this example was working before.

If this issue appeared after the updating the Python appimages may be we should start doing the releases instead of continuous release? You can use latest keyword to download the image from the latest release i.e.:

https://github.com/xonsh/xonsh/releases/latest/download/xonsh-x86_64.AppImage

Thanks!

Update packages to fix CVE-2021-3177

CVE-2021-3177 is a buffer overflow in the ctypes PyCArg_repr() function. NIST has given it a CVSS score of 9.8 (critical). It has been fixed in the following python versions:

  • Python 3.6.13 (2021-02-16) fixed by commit 34df10a (branch 3.6) (2021-01-18)
  • Python 3.7.10 (2021-02-16) fixed by commit d9b8f13 (branch 3.7) (2021-01-18)
  • Python 3.8.8 (2021-02-19) fixed by [commit ece5dfd (branch 3.8)](commit ece5dfd (branch 3.8)) (2021-01-18)
  • Python 3.9.2 (2021-02-19) fixed by [commit c347cbe (branch 3.9)](commit c347cbe (branch 3.9)) (2021-01-18)

Please consider providing updated AppImages that do not contain this vulnerability.

For more information, see https://python-security.readthedocs.io/vuln/ctypes-buffer-overflow-pycarg_repr.html

Feature Request

Hi, very very very good jobs...

Is it possible to add supports to tcl/tk v8.6 in "recent" (>= 3.6) appimages ?
It would be very great.

By the way thank for you work

RuntimeError: Running command git clone

Hello! Thank you for your work here!

To install from git like:

pip install git+https://github.com/xonsh/xonsh

I've added git url to appliactions/xonsh/requirements.txt:

git+https://github.com/xonsh/xonsh
prompt_toolkit
Pygments

And I've got an error:

$ python -m python_appimage build app applications/xonsh
[2020-04-21 09:09:23,913] EXTRACT  python3.8.2-cp38-cp38-manylinux1_x86_64.AppImage
[2020-04-21 09:09:40,715] BUNDLE   xonsh.desktop
[2020-04-21 09:09:40,716] BUNDLE   xonsh.appdata.xml
[2020-04-21 09:09:42,202] BUNDLE   git+https://github.com/xonsh/xonsh
Traceback (most recent call last):
  File "/home/pc/miniconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/pc/miniconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/pc/xxh/python-appimage/python_appimage/__main__.py", line 92, in <module>
    main()
  File "/home/pc/xxh/python-appimage/python_appimage/__main__.py", line 88, in main
    command.execute(*command._unpack_args(args))
  File "/home/pc/xxh/python-appimage/python_appimage/commands/build/app.py", line 224, in execute
    exclude=deprecation)
  File "/home/pc/xxh/python-appimage/python_appimage/utils/system.py", line 42, in system
    raise RuntimeError(err)
RuntimeError:   Running command git clone -q https://github.com/xonsh/xonsh /tmp/pip-req-build-t16gwmrn

If I run just git clone -q https://github.com/xonsh/xonsh /tmp/pip-req-build-t16gwmrn it works.

What I'm doing wrong?

Thanks!

The unwanted crashing in case of warning: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager.

Hi!

The xxh project built system based on docker containers was crashed with error:

Step 9/10 : RUN python -m python_appimage build app /xxh/appimage
 ---> Running in 83823cdd8082
[2021-12-20 00:19:42,633] EXTRACT  python3.9.9-cp39-cp39-manylinux1_x86_64.AppImage
[2021-12-20 00:19:43,561] BUNDLE   xxh.desktop
[2021-12-20 00:19:43,562] BUNDLE   xxh.appdata.xml
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.8/site-packages/python_appimage/__main__.py", line 113, in <module>
    main()
  File "/usr/local/lib/python3.8/site-packages/python_appimage/__main__.py", line 109, in main
    command.execute(*command._unpack_args(args))
  File "/usr/local/lib/python3.8/site-packages/python_appimage/commands/build/app.py", line 245, in execute
    system(('./AppDir/AppRun', '-m', 'pip', 'install', '-U', in_tree_build,
  File "/usr/local/lib/python3.8/site-packages/python_appimage/utils/system.py", line 42, in system
    raise RuntimeError(err)
RuntimeError: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Please add ignoring of this pip warning.

Thanks!

ImportError for libffi.so.7 When Trying to Run PyGObject App

Thanks for the great library! ❤️

I am trying to bundle Gaphor with is a PyGObject app in to an AppImage. Everything works fine locally (openSUSE Tumbleweed), but when I build it using Ubuntu 20.04 with the GitHub Actions runner the AppImage fails to load.

dan@localhost:~/Downloads> ./gaphor-2.1.1.dev0+e2c18321.AppImage 
Traceback (most recent call last):
  File "/tmp/.mount_gaphorDjYPzc/opt/python3.9/bin/gaphor", line 6, in <module>
    from gaphor.ui import main
  File "/tmp/.mount_gaphorDjYPzc/opt/python3.9/lib/python3.9/site-packages/gaphor/ui/__init__.py", line 10, in <module>
    import gi
  File "/tmp/.mount_gaphorDjYPzc/opt/python3.9/lib/python3.9/site-packages/gi/__init__.py", line 40, in <module>
    from . import _gi
ImportError: libffi.so.7: cannot open shared object file: No such file or directory

Taking a look inside the AppImage:

dan@localhost:~/Downloads> ./gaphor-2.1.1.dev0+e2c18321.AppImage --appimage-extract | grep libffi
squashfs-root/usr/lib/libffi.so.5

libffi.so.5 is bundled, but not libffi.so.7. I also tried to apt-get install libffi7 on the runner to make sure it is installed.

This is the output of running the build:

python-appimage build app /home/runner/work/gaphor/gaphor/appimage
[2020-12-29 18:59:07,226] EXTRACT  python3.9.1-cp39-cp39-manylinux1_x86_64.AppImage
[2020-12-29 18:59:08,108] BUNDLE   org.gaphor.Gaphor.desktop
[2020-12-29 18:59:08,109] BUNDLE   org.gaphor.Gaphor.png
[2020-12-29 18:59:08,111] BUNDLE   org.gaphor.Gaphor.appdata.xml
[2020-12-29 18:59:12,603] BUNDLE   gaphas==3.0.0b3
[2020-12-29 18:59:13,465] BUNDLE   generic==1.0.0
[2020-12-29 18:59:14,054] BUNDLE   importlib-metadata==3.3.0
[2020-12-29 18:59:14,791] BUNDLE   pycairo==1.20.0
[2020-12-29 18:59:15,318] BUNDLE   pygobject==3.38.0
[2020-12-29 18:59:15,877] BUNDLE   tinycss2==1.1.0
[2020-12-29 18:59:16,539] BUNDLE   typing-extensions==3.7.4.3
[2020-12-29 18:59:17,147] BUNDLE   webencodings==0.5.1
[2020-12-29 18:59:17,786] BUNDLE   zipp==3.4.0
[2020-12-29 18:59:18,425] BUNDLE   gaphor==2.1.1
[2020-12-29 18:59:20,072] BUNDLE   entrypoint.sh
[2020-12-29 18:59:20,084] BUILD    AppDir
[2020-12-29 18:59:20,084] INSTALL  appimagetool from https://github.com/AppImage/AppImageKit/releases/download/continuous

RuntimeError while running as root user and in docker (Ubuntu 18.04)

$ python-appimage build app appimage/
[2020-07-02 12:14:03,534] EXTRACT  python3.8.3-cp38-cp38-manylinux1_x86_64.AppImage
Traceback (most recent call last):
  File "/usr/local/bin/python-appimage", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/python_appimage/__main__.py", line 103, in main
    command.execute(*command._unpack_args(args))
  File "/usr/local/lib/python3.6/dist-packages/python_appimage/commands/build/app.py", line 151, in execute
    system((base_image, '--appimage-extract'))
  File "/usr/local/lib/python3.6/dist-packages/python_appimage/utils/system.py", line 42, in system
    raise RuntimeError(err)
RuntimeError: /bin/sh: 1: ./base.AppImage: not found

Steps to Reproduce:

  • Open an Ubuntu 18.04 docker container
  • Install python-appimage
  • Attempt to build appimage

Create the GitHub Actions guide to create AppImage for python packages

Hi!

While I creating #18 I thought that it will be great to have the similar algorithm for GitHub Actions that you're already using.

May be create python-appimage-github-actions-example repo with an example of simple PyPi package and with basic GitHub Actions to build AppImage and add it to release asssets. Then packages developers just add it and the world will become a bit portable ;)

Thanks!

pip install: option -c not recognized

Hi! Thank you for python-appimage! It's really help!

I've faced with error when I run pip install in xonsh.AppImage using python from AppImage:

# which python                                                                                                                                                                      
/tmp/.mount_xonshxIlqpg/usr/bin/python

# python -m pip install xontrib-schedule 
Collecting xontrib-schedule
  Downloading xontrib-schedule-0.0.1.tar.gz (1.9 kB)
    ERROR: Command errored out with exit status 1:
     command: /root/.xxh/.xxh/shells/xxh-shell-xonsh-appimage/build/xonsh -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vfdplxou/xontrib-schedule/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vfdplxou/xontrib-schedule/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-vfdplxou/xontrib-schedule/pip-egg-info
         cwd: /tmp/pip-install-vfdplxou/xontrib-schedule/
    Complete output (6 lines):
    usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: setup.py --help [cmd1 cmd2 ...]
       or: setup.py --help-commands
       or: setup.py cmd --help
    
    error: option -c not recognized
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I think this appears because:

# ls -la /tmp/.mount_xonshP4mlJm/usr/bin/python*                                                                                                                                    
/tmp/.mount_xonshP4mlJm/usr/bin/python -> python3
/tmp/.mount_xonshP4mlJm/usr/bin/python3 -> python3.8
/tmp/.mount_xonshP4mlJm/usr/bin/python3.8

Where python3.8 is a wrapper script over $APPDIR/opt/python3.8/bin/python3.8.

Pip install works when I do with binary python:

# /tmp/.mount_xonshP4mlJm/opt/python3.8/bin/python3.8 -m pip install xontrib-schedule                                                                                               
Successfully installed xontrib-schedule-0.0.1

So the questions is what is the right way to use python from AppImage in $PATH? Before I faced with this issue I did just:

$PATH = [f'{$APPDIR}/usr/bin'] + $PATH

because usr/bin has all python symlinks.

(For workaround I'm going to use $APPDIR/opt/python3.8/bin/python3.8 -m pip as pip alias)

Thanks!

Package dependencies of C based libraries

Python has quite a few C based libraries available which include .so files. It seems that currently, the libraries linked to those files are not being included in AppImages generated by python-appimage. So the user needs to have them installed for the application in run in a lot of cases.

I think it would be possible to implement packaging all required libraries using ldd on .so files and then copying the libraries found in the result. The linuxdeploy software does something like this as well. Would that work or would it require more work to get that working?

Site-packages not isolated from ~/.local

While trying to package https://github.com/lukas-blecher/LaTeX-OCR I ran into problems with the appimage finding packages from my ~/.local. In particular, transformers tried to import tensorflow and found a copy there, which caused problems. I noticed there is some attempt to isolate from system package so I thought I might file a bug here to discuss.

I'm not sure what the solution is here. One of the advantages of AppImage is the lack of isolation makes a lot of things easier. There are reasons people could want global site packages available, e.g. installing plugins. On the other hand, it can definitely cause things to interfere with each other.

What do you think? Perhaps there could be an option of site-packages isolation (no/weak/full) where weak is the current behavior, and full nukes the whole thing before adding stuff from the AppImage.

AppRun: shebang customization

Hi @niess! Thank you for your work!

I've found that #8 appears because xonsh was running in not interactive bash mode.

If AppRun shebang will be #! /bin/bash -i instead of #! /bin/bash the xonsh will work perfect :)

I've made the fix for my build and just want to let you know.

It will be cool if there will be an option to AppRun shebang customization.

Thanks!

Re-enable version-less executables

First thanks for the great work!

On previous releases such as 3.10.4 version-less executables were available:

$ ls -l /srv/python/python-3.10.4/usr/bin/
total 0
lrwxrwxrwx 1 admin admin  4 Jun  3 08:22 pip -> pip3
lrwxrwxrwx 1 admin admin  7 Jun  3 08:22 pip3 -> pip3.10
lrwxrwxrwx 1 admin admin 32 Jun  3 08:22 pip3.10 -> ../../opt/python3.10/bin/pip3.10
lrwxrwxrwx 1 admin admin  7 Jun  3 08:22 python -> python3
lrwxrwxrwx 1 admin admin 10 Jun  3 08:22 python3 -> python3.10
lrwxrwxrwx 1 admin admin 35 Jun  3 08:22 python3.10 -> ../../opt/python3.10/bin/python3.10

Since last release (3.10.5 and 3.11.0) only version dependent executable are provided:

$ ls -l /srv/python/python-3.10.5/usr/bin/ /srv/python/python-3.11.0/usr/bin/
/srv/python/python-3.10.5/usr/bin/:
total 4
lrwxrwxrwx 1 admin admin  32 Jun 14 08:28 pip3.10 -> ../../opt/python3.10/bin/pip3.10
-rwxr-xr-x 1 admin admin 647 Jun 14 08:28 python3.10

/srv/python/python-3.11.0/usr/bin/:
total 4
lrwxrwxrwx 1 admin admin  32 Jun 14 08:26 pip3.11 -> ../../opt/python3.11/bin/pip3.11
-rwxr-xr-x 1 admin admin 495 Jun 14 08:26 python3.11

This is painful since it breaks compatibility with exiting scripts.

Thanks in advance.

xonsh AppImage is broken after Jun changes

Hi @niess!

We've noticed that xonsh.AppImage 0.12.4 (May 9) is working but starting from xonsh release 0.12.5 (Jun 17) it's not working.
Example:

cd /tmp
wget https://github.com/xonsh/xonsh/releases/download/0.13.3/xonsh-x86_64.AppImage
chmod +x xonsh-x86_64.AppImage
./xonsh-x86_64.AppImage --no-rc
# Freez OR bash: warning: shell level (1000) too high, resetting to 1

I think some of python-appimage commits in Jun 11-14 is the cause.

Could you please test the python-appimage with xonsh and give us the advice?
The building process is:

mkdir -p /tmp/build && cd /tmp/build
git clone --depth 1 https://github.com/xonsh/xonsh
cd xonsh/appimage
echo 'xonsh' > requirements.txt
cat pre-requirements.txt >> requirements.txt  # here you can add your additional PyPi packages to pack them into AppImage
cd ..
pip install git+https://github.com/niess/python-appimage
python -m python_appimage build app --python-version 3.10 ./appimage
./xonsh-x86_64.AppImage

Thanks!

AttributeError: module 'importlib' has no attribute 'util'

$ python-appimage build app main.py
Traceback (most recent call last):
  File "/home/fakeminer/.local/bin/python-appimage", line 8, in <module>
    sys.exit(main())
  File "/home/fakeminer/.local/lib/python3.10/site-packages/python_appimage/__main__.py", line 100, in main
    command = import_module(module, package=__package__)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/fakeminer/.local/lib/python3.10/site-packages/python_appimage/commands/build/app.py", line 10, in <module>
    from ...appimage import build_appimage
  File "/home/fakeminer/.local/lib/python3.10/site-packages/python_appimage/appimage/__init__.py", line 1, in <module>
    from .build import build_appimage
  File "/home/fakeminer/.local/lib/python3.10/site-packages/python_appimage/appimage/build.py", line 7, in <module>
    from ..utils.compat import decode
  File "/home/fakeminer/.local/lib/python3.10/site-packages/python_appimage/utils/compat.py", line 27, in <module>
    find_spec = importlib.util.find_spec
AttributeError: module 'importlib' has no attribute 'util'

Operating system Linux Ubuntu 22.04

[Question] Adding wxPython

I am trying to pack a piece of software using python2.7. I am struggling to get the dependencies properly.

I download python2.7:

wget https://github.com/niess/python-appimage/releases/download/python2.7/python2.7.18-cp27-cp27m-manylinux1_x86_64.AppImage
chmod +x python2.7.18-cp27-cp27m-manylinux1_x86_64.AppImage
./python2.7.18-cp27-cp27m-manylinux1_x86_64.AppImage  --appimage-extract

Then I install the dependencies:

usr/bin/pip2.7 install future zeroconf==0.19.1 numpy==1.16.5 matplotlib==2.0.2 lxml==4.6.2 pyro sslpsk pyserial

But I struggle to get wxPython given than only v4.x is available, while I need v3.0. I found it in Ubuntu. So I downloaded:

apt-get download python-wxgtk3.0
dpkg-deb -xv python-wxgtk3.0_3.0.2.0+dfsg-8build1_amd64.deb ./

Is there a way to get the extracted data from the .deb package integrated with the extracted data from python-appimage?

Feature request - pass directory for tmp dis creation

I want to place the recipe in the python project that I want to package. If I can give path where to create the tmp dirs I can put relative path in the requirements.txt. The function used for creation of directories tempfile.mkdtemp supports parameter dir which gives root for the tmp directories. I`m not sure how to manage the passing of parameter. Do you think that it is possible?

PyPy AppImages

This might be out of scope as far a suggestions go but would it be possible to also offer PyPy AppImages?

My motivation is that I've heard, not yet tested, that PyPy performs generally faster than CPython. And, by using an AppImage, the user doesn't even have to be instructed to install an alternative Python interpreter to make use of the faster speed. I think that things like these, increased performance or features requiring a difficult setup, is really one of the strongest points of using AppImages in the first place.

Cannot find Python.h

I am trying to

  • unpack a python appimage
  • pip-install stuff inside it
  • repack it

Final goal is to produce an autonomous appimage of a python app

Bad stuff happens when gcc has to build stuff, it does not find Python.h :

  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DNETIFACES_VERSION=0.10.9 -DHAVE_GETIFADDRS=1 -DHAVE_GETNAMEINFO=1 -DHAVE_NETASH_ASH_H=1 -DHAVE_NETATALK_AT_H=1 -DHAVE_NETAX25_AX25_H=1 -DHAVE_NETECONET_EC_H=1 -DHAVE_NETIPX_IPX_H=1 -DHAVE_NETPACKET_PACKET_H=1 -DHAVE_LINUX_IRDA_H=1 -DHAVE_LINUX_ATM_H=1 -DHAVE_LINUX_LLC_H=1 -DHAVE_LINUX_TIPC_H=1 -DHAVE_LINUX_DN_H=1 -DHAVE_SOCKADDR_AT=1 -DHAVE_SOCKADDR_AX25=1 -DHAVE_SOCKADDR_IN=1 -DHAVE_SOCKADDR_IN6=1 -DHAVE_SOCKADDR_IPX=1 -DHAVE_SOCKADDR_UN=1 -DHAVE_SOCKADDR_ASH=1 -DHAVE_SOCKADDR_EC=1 -DHAVE_SOCKADDR_LL=1 -DHAVE_SOCKADDR_ATMPVC=1 -DHAVE_SOCKADDR_ATMSVC=1 -DHAVE_SOCKADDR_DN=1 -DHAVE_SOCKADDR_IRDA=1 -DHAVE_SOCKADDR_LLC=1 -DHAVE_PF_NETLINK=1 -I/tmp/appimage-build-lukpB4/AppDir/opt/python3.7/include/python3.7m -c netifaces.c -o build/temp.linux-x86_64-3.7/netifaces.o

  netifaces.c:1:20: fatal error: Python.h: No such file or directory

  compilation terminated.

  error: command 'gcc' failed with exit status 1

complete log : https://travis-ci.com/github/libreosteo/Libreosteo/jobs/386916555#L3369-L3372

What raise my attention is the -I/tmp/appimage-build-lukpB4/AppDir/opt/python3.7/include/python3.7m that seems to point to the wrong dir. Right path would be the same without the final "m".

I was able to workaround using a symlink /tmp/appimage-build-lukpB4/AppDir/opt/python3.7/include/python3.7m -> /tmp/appimage-build-lukpB4/AppDir/opt/python3.7/include/python3.7

But that remains a hack

My build script (failing) is:

#!/bin/bash

set -x
set -e

# requirements
#   yarn
#   rsync
#   git

# building in temporary directory to keep system clean
# use RAM disk if possible (as in: not building on CI system like Travis, and RAM disk is available)
if [ "$CI" == "" ] && [ -d /dev/shm ]; then
    TEMP_BASE=/dev/shm
else
    TEMP_BASE=/tmp
fi

PYTHON_APPIMAGE=python3.7.9-cp37-cp37m-manylinux1_x86_64.AppImage
PYTHON_APPIMAGE_URL=https://github.com/niess/python-appimage/releases/download/python3.7/${PYTHON_APPIMAGE}

export VERSION=develop-$(git rev-parse --short HEAD)
BUILD_DIR=$(mktemp -d -p "$TEMP_BASE" appimage-build-XXXXXX)
APP_DIR="$BUILD_DIR/AppDir"
mkdir $APP_DIR

# make sure to clean up build dir, even if errors occur
cleanup () {
    if [ -d "$BUILD_DIR" ]; then
        rm -rf "$BUILD_DIR"
    fi
}
trap cleanup EXIT

# Store repo root as variable
REPO_ROOT=$(dirname $(dirname  $(dirname $(realpath $0))))
OLD_CWD=$(readlink -f .)

# switch to build dir
pushd "$BUILD_DIR"

# Fetch a python relocatable installation
wget -c ${PYTHON_APPIMAGE_URL}
chmod +x ${PYTHON_APPIMAGE}
./${PYTHON_APPIMAGE} --appimage-extract

mv squashfs-root/usr $APP_DIR/usr
mv squashfs-root/opt $APP_DIR/opt
rm -rf squashfs-root/

# Pack required source code into AppDir
# Avoid using .git and other unrequired stuff
rsync -av "$REPO_ROOT/" "$APP_DIR/src" \
      --exclude '.git/' \
      --exclude-from="$REPO_ROOT/.gitignore"

# Install requirements (JS and Python)
pushd $APP_DIR
ls ./opt/python3.7/include/python3.7

./usr/bin/python3 -m pip install -r src/requirements/requirements.txt
yarn --cwd "$REPO_ROOT"
./usr/bin/python3 src/manage.py collectstatic --no-input

mkdir -p usr/share/metainfo
mv src/pkg/libreosteo.metainfo.xml usr/share/metainfo/

popd


# Get commit version from repository
pushd $REPO_ROOT
export VERSION=develop-$(git rev-parse --short HEAD)
popd


# Now, build AppImage using linuxdeploy
export ARCH=x86_64
wget -c https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage
chmod +x linuxdeploy*.AppImage
./linuxdeploy-x86_64.AppImage \
  --appdir $APP_DIR \
  --icon-file $APP_DIR/src/libreosteoweb/static/images/libreosteo.png \
  --desktop-file $APP_DIR/src/pkg/libreosteo.desktop \
  --custom-apprun $APP_DIR/src/pkg/appimage/AppRun \
  --output appimage

# move built AppImage back into original CWD
mv LibreOsteo*.AppImage "$OLD_CWD/"
echo 'Hello LibreOsteo*.Appimage !'

`system()` method from `system.py` propogates runtime warnings as RuntimeErrors

While working on getting together an Appimage for one of my favorite programs I encountered a build failure because warnings were propagated as errors. this system call method should be edited to ignore such warnings

$ python-appimage build app -p 3.11 novelwriter                                                   
[2022-07-07 18:54:41,192] EXTRACT  python3.11.0-cp311-cp311-manylinux2010_x86_64.AppImage
Traceback (most recent call last):
  File "/home/ryex/.local/bin/python-appimage", line 8, in <module>
    sys.exit(main())
  File "/home/ryex/.local/lib/python3.10/site-packages/python_appimage/__main__.py", line 109, in main
    command.execute(*command._unpack_args(args))
  File "/home/ryex/.local/lib/python3.10/site-packages/python_appimage/commands/build/app.py", line 162, in execute
    system((base_image, '--appimage-extract'))
  File "/home/ryex/.local/lib/python3.10/site-packages/python_appimage/utils/system.py", line 42, in system
    raise RuntimeError(err)
RuntimeError: Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.

this is already done with exclude.append('fuse: warning:') but a more generic check for warnings should be used

these warnings may need to be propagated out to stderr but they should not halt.

License question

Hi!
Thanks for this project. We are considering using this as a based for AppImages for https://github.com/nexB/scancode-toolkit and a few other FOSS projects and I have a licensing question:

Your project code here is GPL-licensed alright, but do you consider the generated AppImages to be also subject to the GPL?
@JonoYang FYI

{install,build,which} response

I was told to ask here:
I have no idea what to input into the terminal after, no I don't know linux at all. I'm just trying to get python working on my chromebook.
tylonsecret@penguin:~$` python3 python_appimage-1.2.1-py2.py3-none-any/python_appimage/main.py
usage: python-appimage [-h] [-q] [-v] {install,build,which} ...

Bundle a Python installation into an AppImage

optional arguments:
-h, --help show this help message and exit
-q, --quiet disable logging
-v, --verbose print extra information

command:
{install,build,which}
Command to execute

SSL: CERTIFICATE_VERIFY_FAILED unable to get local issuer certificate

The python3.7 AppImage released by this repo appears to be unable to make safe https requests using the python built-in urllib module. Any request over https results in the following error:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)

Steps to Reproduce

wget https://github.com/niess/python-appimage/releases/download/python3.7/python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage
chmod +x python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage 
./python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage -c "import urllib.request; urllib.request.urlopen('https://wikipedia.org' )"

Example Execution

user@disp569:~$ wget https://github.com/niess/python-appimage/releases/download/python3.7/python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage
--2020-08-09 18:19:04--  https://github.com/niess/python-appimage/releases/download/python3.7/python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage
Resolving github.com (github.com)... 140.82.118.3
Connecting to github.com (github.com)|140.82.118.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/251001753/16f68580-d46d-11ea-8073-30739bb2f961?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200809%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200809T123406Z&X-Amz-Expires=300&X-Amz-Signature=e9a3c5ba6eb1091ce71cb60a26965550af337ef911e3cfefc55aa1f3540447e8&X-Amz-SignedHeaders=host&actor_id=0&repo_id=251001753&response-content-disposition=attachment%3B%20filename%3Dpython3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage&response-content-type=application%2Foctet-stream [following]
--2020-08-09 18:19:05--  https://github-production-release-asset-2e65be.s3.amazonaws.com/251001753/16f68580-d46d-11ea-8073-30739bb2f961?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200809%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200809T123406Z&X-Amz-Expires=300&X-Amz-Signature=e9a3c5ba6eb1091ce71cb60a26965550af337ef911e3cfefc55aa1f3540447e8&X-Amz-SignedHeaders=host&actor_id=0&repo_id=251001753&response-content-disposition=attachment%3B%20filename%3Dpython3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.133.99
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.133.99|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17485864 (17M) [application/octet-stream]
Saving to: ‘python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage’

python3.7.8-cp37-cp 100%[===================>]  16.68M  1.02MB/s    in 14s     

2020-08-09 18:19:21 (1.17 MB/s) - ‘python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage’ saved [17485864/17485864]

user@disp569:~$ 

user@disp569:~$ chmod +x python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage 
user@disp569:~$ ./python3.7.8-cp37-cp37m-manylinux2014_x86_64.AppImage -c "import urllib.request; urllib.request.urlopen( 'https://wikipedia.org' )"
Traceback (most recent call last):
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/urllib/request.py", line 1350, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/http/client.py", line 1262, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/http/client.py", line 1308, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/http/client.py", line 1257, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/http/client.py", line 1028, in _send_output
    self.send(msg)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/http/client.py", line 968, in send
    self.connect()
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/http/client.py", line 1432, in connect
    server_hostname=server_hostname)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/ssl.py", line 423, in wrap_socket
    session=session
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/ssl.py", line 870, in _create
    self.do_handshake()
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/urllib/request.py", line 1393, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/tmp/.mount_pythonqGMWe3/opt/python3.7/lib/python3.7/urllib/request.py", line 1352, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)>
user@disp569:~$ 

Expected Behaviour

The built-in python3 to debian 10 makes the request without any certificate issues

user@disp569:~$ cat /etc/issue
Debian GNU/Linux 10 \n \l

user@disp569:~$ sudo dpkg -l | grep python3
ii  libpython3-stdlib:amd64               3.7.3-1                             amd64        interactive high-level object-oriented language (default python3 version)
ii  libpython3.7:amd64                    3.7.3-2+deb10u2                     amd64        Shared Python runtime library (version 3.7)
ii  libpython3.7-minimal:amd64            3.7.3-2+deb10u2                     amd64        Minimal subset of the Python language (version 3.7)
ii  libpython3.7-stdlib:amd64             3.7.3-2+deb10u2                     amd64        Interactive high-level object-oriented language (standard library, version 3.7)
ii  python3                               3.7.3-1                             amd64        interactive high-level object-oriented language (default python3 version)
ii  python3-apt                           1.8.4.1                             amd64        Python 3 interface to libapt-pkg
ii  python3-asn1crypto                    0.24.0-1                            all          Fast ASN.1 parser and serializer (Python 3)
ii  python3-bcrypt                        3.1.6-1                             amd64        password hashing library for Python 3
ii  python3-cairo:amd64                   1.16.2-1+b1                         amd64        Python3 bindings for the Cairo vector graphics library
ii  python3-certifi                       2018.8.24-1                         all          root certificates for validating SSL certs and verifying TLS hosts (python3)
ii  python3-cffi-backend                  1.12.2-1                            amd64        Foreign Function Interface for Python 3 calling C code - runtime
ii  python3-chardet                       3.0.4-3                             all          universal character encoding detector for Python3
ii  python3-croniter                      0.3.24-2                            all          provides iteration for datetime object with cron like format - Python 3.x
ii  python3-crypto                        2.6.1-9+b1                          amd64        cryptographic algorithms and protocols for Python 3
ii  python3-cryptography                  2.6.1-3+deb10u2                     amd64        Python library exposing cryptographic recipes and primitives (Python 3)
ii  python3-cups                          1.9.73-2+b1                         amd64        Python3 bindings for CUPS
ii  python3-cupshelpers                   1.5.11-4                            all          Python utility modules around the CUPS printing system
ii  python3-daemon                        2.2.3-1                             all          library for making a Unix daemon process  Python 3
ii  python3-dateutil                      2.7.3-3                             all          powerful extensions to the standard Python 3 datetime module
ii  python3-dbus                          1.2.8-3                             amd64        simple interprocess messaging system (Python 3 interface)
ii  python3-debconf                       1.5.71                              all          interact with debconf from Python 3
ii  python3-debian                        0.1.35                              all          Python 3 modules to work with Debian-related data formats
ii  python3-debianbts                     2.8.2                               all          Python interface to Debian's Bug Tracking System
ii  python3-distro                        1.3.0-1                             all          Linux OS platform information API
ii  python3-distutils                     3.7.3-1                             all          distutils package for Python 3.x
ii  python3-gi                            3.30.4-1                            amd64        Python 3 bindings for gobject-introspection libraries
ii  python3-gi-cairo                      3.30.4-1                            amd64        Python 3 Cairo bindings for the GObject library
ii  python3-httplib2                      0.11.3-2                            all          comprehensive HTTP client library written for Python3
ii  python3-idna                          2.6-1                               all          Python IDNA2008 (RFC 5891) handling (Python 3)
ii  python3-jinja2                        2.10-2                              all          small but fast and easy to use stand-alone template engine
ii  python3-jmespath                      0.9.4-1                             all          JSON Matching Expressions (Python 3)
ii  python3-kerberos                      1.1.14-2                            amd64        GSSAPI interface module - Python 3.x
ii  python3-lib2to3                       3.7.3-1                             all          Interactive high-level object-oriented language (2to3, version 3.6)
ii  python3-libcloud                      2.4.0-1                             all          unified Python interface into the cloud (Python3 version)
ii  python3-lockfile                      1:0.12.2-2                          all          file locking library for Python  Python 3 library
ii  python3-markupsafe                    1.1.0-1                             amd64        HTML/XHTML/XML string library for Python 3
ii  python3-minimal                       3.7.3-1                             amd64        minimal subset of the Python language (default python3 version)
ii  python3-msgpack                       0.5.6-1+b1                          amd64        Python 3 implementation of MessagePack format
ii  python3-nacl                          1.3.0-2                             amd64        Python bindings to libsodium (Python 3)
ii  python3-netaddr                       0.7.19-1                            all          manipulation of various common network address notations (Python 3)
ii  python3-ntlm-auth                     1.1.0-1                             all          NTLM low-level Python library
ii  python3-numpy                         1:1.16.2-1                          amd64        Fast array facility to the Python 3 language
ii  python3-olefile                       0.46-1                              all          Python module to read/write MS OLE2 files
ii  python3-paramiko                      2.4.2-0.1                           all          Make ssh v2 connections (Python 3)
ii  python3-pil:amd64                     5.4.1-2+deb10u2                     amd64        Python Imaging Library (Python3)
ii  python3-pkg-resources                 40.8.0-1                            all          Package Discovery and Resource Access using pkg_resources
ii  python3-psutil                        5.5.1-1                             amd64        module providing convenience functions for managing processes (Python3)
ii  python3-pyasn1                        0.4.2-3                             all          ASN.1 library for Python (Python 3 module)
ii  python3-pycurl                        7.43.0.2-0.1                        amd64        Python bindings to libcurl (Python 3)
ii  python3-pysimplesoap                  1.16.2-1                            all          simple and lightweight SOAP Library (Python 3)
ii  python3-qubesdb                       4.0.13-1+deb10u1                    amd64        QubesDB python bindings.
ii  python3-reportbug                     7.5.3~deb10u1                       all          Python modules for interacting with bug tracking systems
ii  python3-requests                      2.21.0-1                            all          elegant and simple HTTP library for Python3, built for human beings
ii  python3-requests-kerberos             0.11.0-2                            all          Kerberos/GSSAPI authentication handler for python-requests - Python 3.x
ii  python3-requests-ntlm                 1.1.0-1                             all          Adds support for NTLM authentication to the requests library
ii  python3-scour                         0.37-2                              all          SVG scrubber and optimizer (Python 3 module)
ii  python3-selinux                       2.8-1+b1                            amd64        Python3 bindings to SELinux shared libraries
ii  python3-simplejson                    3.16.0-1                            amd64        simple, fast, extensible JSON encoder/decoder for Python 3.x
ii  python3-six                           1.12.0-1                            all          Python 2 and 3 compatibility library (Python 3 interface)
ii  python3-smbc                          1.0.15.6-1+b2                       amd64        Python 3 bindings for the Samba client library
ii  python3-tornado4                      4.5.3-3                             amd64        scalable, non-blocking web server and tools - Python 3 package
ii  python3-tz                            2019.1-1                            all          Python3 version of the Olson timezone database
ii  python3-uno                           1:6.1.5-3+deb10u6                   amd64        Python-UNO bridge
ii  python3-urllib3                       1.24.1-1                            all          HTTP library with thread-safe connection pooling for Python3
ii  python3-winrm                         0.3.0-2                             all          Python 3 library for Windows Remote Management
ii  python3-xcffib                        0.8.1-1+deb10u1                     amd64        This package is a Python binding for XCB (Python 3)
ii  python3-xdg                           0.25-5                              all          Python 3 library to access freedesktop.org standards
ii  python3-xmltodict                     0.11.0-2                            all          Makes working with XML feel like you are working with JSON (Python 3)
ii  python3-yaml                          3.13-2                              amd64        YAML parser and emitter for Python3
ii  python3.7                             3.7.3-2+deb10u2                     amd64        Interactive high-level object-oriented language (version 3.7)
ii  python3.7-minimal                     3.7.3-2+deb10u2                     amd64        Minimal subset of the Python language (version 3.7)
user@disp569:~$ python3.7 --version
Python 3.7.3
user@disp569:~$ python3.7 -c "import urllib.request; urllib.request.urlopen( 'https://wikipedia.org' )"
user@disp569:~$ 

pip is missing when venv is created out of appimage

~ ./python3.9.9-cp39-cp39-manylinux2010_x86_64.AppImage -m venv venvs/test
 △ ~ venvs/test/bin/python -m pip
/home/keskfrad/venvs/test/bin/python: No module named pip

is it even possible to create and use virtual envs out of such .AppImages?

Error on executing `python-appimage`

I just ran python-appimage as a command, which returned this

❯ python-appimage
Traceback (most recent call last):
  File "/home/xxx/.local/bin/python-appimage", line 8, in <module>
    sys.exit(main())
  File "/home/xxx/.local/lib/python3.8/site-packages/python_appimage/__main__.py", line 86, in main
    module = '.commands.' + args.command
TypeError: can only concatenate str (not "NoneType") to str

Maybe, the help should be triggered by default.

tk and tcl path

After updating to a newer release my tkinter app no longer worked.
Fixed it by changing these lines in AppRun:
export TCL_LIBRARY="${APPDIR}/usr/share/tcltk/tcltcl-version"
export TK_LIBRARY="${APPDIR}/usr/share/tcltk/tktk-version"
to this:
export TCL_LIBRARY="${APPDIR}/usr/share/tcltk/tcl8.5"
export TK_LIBRARY="${APPDIR}/usr/share/tcltk/tk8.5"

Tested with python3.8.2-cp38-cp38-manylinux2014_x86_64.AppImage

Everything else works great. Thanks a lot for doing this :)

SSL CERTFILE commit breaks new appimages

running an appimage built using python-appimage fails with the following error:

/tmp/appimage_extracted_ff11d2f4ca2d1e1dbfa7e9958e75c7e0/AppRun: line 12: cert-file: command not found

This is how my AppRun looks like

#! /bin/bash

# Export APPRUN if running from an extracted image
self="$(readlink -f -- $0)"
here="${self%/*}"
APPDIR="${APPDIR:-${here}}"

# Export TCl/Tk
export TCL_LIBRARY="${APPDIR}/usr/share/tcltk/tcl8.4"
export TK_LIBRARY="${APPDIR}/usr/share/tcltk/tk8.4"
export TKPATH="${TK_LIBRARY}"
cert-file

# Call the entry point
export LD_LIBRARY_PATH="${APPDIR}/usr/lib:${LD_LIBRARY_PATH}"
export ZAP="TRUE"

${APPDIR}/usr/bin/python3.8 -s ${APPDIR}/opt/python3.8/bin/zap "$@"

I am not sure why the cert-file popped in. Maybe it was not formatted correctly?

venv: site-packages path is incorrect

root@f310d0f1dfc9:/tmp/demo# python3.10 -m venv venv
root@f310d0f1dfc9:/tmp/demo# source venv/bin/activate


(venv) root@f310d0f1dfc9:/tmp/demo# python -I -m site
sys.path = [
    '/tmp/.mount_pythonCEi1JB/opt/python3.10/lib/python310.zip',
    '/tmp/.mount_pythonCEi1JB/opt/python3.10/lib/python3.10',
    '/tmp/.mount_pythonCEi1JB/opt/python3.10/lib/python3.10/lib-dynload',
    '/tmp/.mount_pythonCEi1JB/opt/python3.10/lib/python3.10/site-packages',
]
USER_BASE: '/root/.local' (exists)
USER_SITE: '/root/.local/lib/python3.10/site-packages' (doesn't exist)
ENABLE_USER_SITE: False


(venv) root@f310d0f1dfc9:/tmp/demo# python -m site
sys.path = [
    '/tmp/demo',
    '/tmp/.mount_pythonUgmxt7/opt/python3.10/lib/python310.zip',
    '/tmp/.mount_pythonUgmxt7/opt/python3.10/lib/python3.10',
    '/tmp/.mount_pythonUgmxt7/opt/python3.10/lib/python3.10/lib-dynload',
    '/tmp/demo/venv/lib/python3.10/site-packages',
]
USER_BASE: '/root/.local' (exists)
USER_SITE: '/root/.local/lib/python3.10/site-packages' (doesn't exist)
ENABLE_USER_SITE: False

Which should always be /tmp/demo/venv/lib/python3.10/site-packages.

Attribute error on run

Hi,
when I run the module the I have an error:
AttributeError: module 'importlib' has no attribute 'util'

When I change the import in python_appimage/utils/compat.py from
import importlib
to
import importlib.utils
all works.

My colleagues and I tested it on different machines - on Ubuntu 20.04 and 22.04 with python 3.8, 3.9 and 3.10 and the results are consistent.

The command and output:

user@server# ./tt/bin/python-appimage -v build app -p 3.9 -l manylinux_2_28_aarch64 appimage_meta
Traceback (most recent call last):
  File "/home/user/./tt/bin/python-appimage", line 8, in <module>
    sys.exit(main())
  File "/home/user/tt/lib/python3.9/site-packages/python_appimage/__main__.py", line 100, in main
    command = import_module(module, package=__package__)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 855, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/user/tt/lib/python3.9/site-packages/python_appimage/commands/build/app.py", line 10, in <module>
    from ...appimage import build_appimage
  File "/home/user/tt/lib/python3.9/site-packages/python_appimage/appimage/__init__.py", line 1, in <module>
    from .build import build_appimage
  File "/home/user/tt/lib/python3.9/site-packages/python_appimage/appimage/build.py", line 7, in <module>
    from ..utils.compat import decode
  File "/home/user/tt/lib/python3.9/site-packages/python_appimage/utils/compat.py", line 27, in <module>
    find_spec = importlib.util.find_spec
AttributeError: module 'importlib' has no attribute 'util'```

To prevent braking the current behavior maybe it is possible to put the import in try block and when import of importlib.lib fails, the import of the importlib will be done?

Add a shebang patcher

Add a tool for setting relative shebangs for python scipts installed inside the AppImage

in-tree-build raising exception with app build

Hello,
While trying python-appimage build app appdir/ --python-version 3.7 I got an exception, my pip version is 20.0.2

Exception

Traceback (most recent call last):
  File "/home/mrf3/Projects/FQM/.venv/bin/python-appimage", line 8, in <module>
    sys.exit(main())
  File "/home/mrf3/Projects/FQM/.venv/lib/python3.8/site-packages/python_appimage/__main__.py", line 105, in main
    command.execute(*command._unpack_args(args))
  File "/home/mrf3/Projects/FQM/.venv/lib/python3.8/site-packages/python_appimage/commands/build/app.py", line 248, in execute
    system(('./AppDir/AppRun', '-m', 'pip', 'install', '-U', in_tree_build,
  File "/home/mrf3/Projects/FQM/.venv/lib/python3.8/site-packages/python_appimage/utils/system.py", line 42, in system
    raise RuntimeError(err)
RuntimeError: DEPRECATION: In-tree builds are now the default. pip 22.1 will enforce this behaviour change. A possible replacement is to remove the --use-feature=in-tree-build flag.

Thanks so much for the awesome package!

tcltk-env: command not found

Hi! Thank you for your work!

Looks like after 5552c1f I get

/tmp/.mount_xonsh-ifYiMy/AppRun: line 7: tcltk-env: command not found

When run xonsh.AppImage.

Python Environment Does Not Work Under Directories With Spaces

Summary

Type: Bug

Explanation

Python environments made with the current AppImage does not work when put inside a directory with a space in it.

Proposed Solution

At python_appimage/data/python-wrapper.sh, line 7, column 28 to 30 (the $0) needed to be encapsulated with double quotation marks (").

Environment

OS: Arch Linux
Kernel: 5.19.7-arch1-1
Shell: zsh 5.9 (x86_64-pc-linux-gnu)
AppImage: python3.9.14-cp39-cp39-manylinux2014_x86_64.AppImage

Explanation

Greetings! Let it be known that I do not have a lot of expertise in this kind of stuff.

To start, I've been using your python appimages to make a python environment if the version is not available at my package manager (I use Arch Linux, and AUR does not keep older versions of python, nor does it for any other programs).

In any case, this works well for me. I've never had any problems with it for a month that I've been using it. But it had been at least a month since I last touched that thingy. And now, I tried to make a python environment using a python 3.9 appimage, as the service that my project deploys to does not have python 3.10 available.

In any case, this fails specifically at the point where the python environment tries to install pip. This is an example of the error:

Error: Command '['/home/whinee/directory with space/project/pyenv/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 127.
pyenv:source:19: no such file or directory: pyenv/bin/activate
/home/whinee/projects/personal/directory with space/project/pyenv/bin/python: line 25: /home/whinee/opt/python3.9/bin/python3.9: No such file or directory
/home/whinee/projects/personal/directory with space/project/pyenv/bin/python: line 25: /home/whinee/opt/python3.9/bin/python3.9: No such file or directory
/home/whinee/projects/personal/directory with space/project/pyenv/bin/python: line 25: /home/whinee/opt/python3.9/bin/python3.9: No such file or directory

How to Replicate

  • At $HOME directory, make 2 folder deep file structure, without any spaces in its name. In this example, we will use a/b.
  • Get inside the folder created earlier, and make a folder with a space in its name. In this example, we will use c d.
  • Get inside the folder created earlier, and make another folder without any spaces in its name. In this example, we will use e.
  • In your favorite shell, set pyver variable with python version you wish to download from this repository. In this example, we will set it as 3.9
  • Run the following command:
tmpf=$(mktemp)
wget -O "$tmpf" "$(wget -qSO - "https://api.github.com/repos/niess/python-appimage/releases/tags/python$pyver" 2>/dev/null | grep -E "browser_download_url.*x86_64" | cut -d'"' -f4 | tail -2 | head -1)" >/dev/null
chmod +x "$tmpf"
"$tmpf" --appimage-extract >/dev/null
rm -rf "$tmpf" pyenv/
mkdir pyenv
mv squashfs-root/opt squashfs-root/usr pyenv/
pyenv/usr/bin/python -m venv pyenv/ # point of failure

With this, the last command should fail with the following error:

Error: Command '['$HOME/a/b/c d/e/pyenv/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 127.
pyenv:source:19: no such file or directory: pyenv/bin/activate
$HOME/a/b/c d/e/pyenv/bin/python: line 25: $HOME/opt/python3.9/bin/python3.9: No such file or directory
$HOME/a/b/c d/e/pyenv/bin/python: line 25: $HOME/opt/python3.9/bin/python3.9: No such file or directory
$HOME/a/b/c d/e/pyenv/bin/python: line 25: $HOME/opt/python3.9/bin/python3.9: No such file or directory

Tracking Down Bug

I tried to track where this error came from, as the error did not provide any information on what file is throwing this error, and I found it on ./pyenv/opt/python3.9/lib/python3.9/venv/__main__.py, at the part where it tries to install pip, as said earlier.

Well... I though that the thingy running that command just do not have enough quotations. But nawp! So I tried tracing the problem in the scripts, starting from the activate script, and got me to the pyenv/bin/python.

I kinda debugged it, and saw that $APPDIR's value is /home/whinee/ instead of /home/whinee/projects/personal/directory with space/project/pyenv/. So I started fiddling with the quotations, and sure enough, it lacked one. I encapsulated $0 at line 7 with double quotations, and it worked like it should!

Proposed Solution

As said earlier, encapsulating python_appimage/data/python-wrapper.sh, line 7, column 28 to 30 with double quotation marks (") solved the problem for me.

Rationale for Not Opening Pull Request

But you ask:

whi_ne, why didn't you just made a pull request, instead of writing this issue which can be solved by adding literally 2 characters?

Well... many reasons, actually:

  • I might break something unexpectedly, and as I said earlier, I am no expert with this stuff.
  • You might have pending changes on your local machine, and making a pull request for this small thing can be quite annoying.
  • Me too shy...

Anyways, I hope that this issue will be resolved soon, thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.