Error building wheels for isal about python-isal HOT 19 CLOSED

I tried to build it manually on my Raspberry Pi 4 (Linux pi4 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux, Raspbian 12.5/Bookworm) and it works:

git clone https://github.com/pycompression/python-isal.git
cd python-isal
git submodule update --init --recursive
pip install .

Processing /home/pi/python-isal
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: isal
  Building wheel for isal (pyproject.toml) ... done
  Created wheel for isal: filename=isal-1.7.0.dev0-cp311-cp311-linux_aarch64.whl size=294199 sha256=161f31f9e50167aaa9d3c8d27bbdb1f35e566b3546a90bb116004a6ea5267af0
  Stored in directory: /home/pi/.cache/pip/wheels/b2/90/b9/870714573c96eb7b73cc4ec4c2cfbd2ebbd3a4bce83f8cf1cc
Successfully built isal
Installing collected packages: isal
Successfully installed isal-1.7.0.dev0

I had trouble updating homeassistant on FreeBSD because of python-isal and fixed it by manually installing python-isal. Maybe try the same?

from python-isal.

@jan666 Did your 'manually installing' require the changes in your PR? I tried installing via pip from the sources on NetBSD and started down the same path as your PR.

from python-isal.

Also, were is the need to deal with submodules documented? I don't see it in the README.

Now filed as #197

from python-isal.

@jan666 Did your 'manually installing' require the changes in your PR?

Yes, but only only feeebsd. On my Raspberry Pi it works without changes. Clone, init submodule, pip install

from python-isal.

I tried to build it manually on my Raspberry Pi 4 (Linux pi4 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux, Raspbian 12.5/Bookworm) and it works:

git clone https://github.com/pycompression/python-isal.git
cd python-isal
git submodule update --init --recursive
pip install .

Processing /home/pi/python-isal
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: isal
  Building wheel for isal (pyproject.toml) ... done
  Created wheel for isal: filename=isal-1.7.0.dev0-cp311-cp311-linux_aarch64.whl size=294199 sha256=161f31f9e50167aaa9d3c8d27bbdb1f35e566b3546a90bb116004a6ea5267af0
  Stored in directory: /home/pi/.cache/pip/wheels/b2/90/b9/870714573c96eb7b73cc4ec4c2cfbd2ebbd3a4bce83f8cf1cc
Successfully built isal
Installing collected packages: isal
Successfully installed isal-1.7.0.dev0

I had trouble updating homeassistant on FreeBSD because of python-isal and fixed it by manually installing python-isal. Maybe try the same?

I tried your steps of cloning the git repo and then installing that version. I still get the same error message from the compiler

from python-isal.

The issue is

igzip/aarch64/encode_df.S: Assembler messages:
igzip/aarch64/encode_df.S:32: Error: unknown architecture `armv8-a+crc'

and the architecture armv8-a+crc looks unusual to me. But that seems like feature flags being extracted and the arch value being a composite of uname -p and flags.

I would suggest that you get the isal library to build, separately from the python wrapper, and also see if Debian is using a different architecture value than what others expect. But, looking at your uname, aarch64 looks normal and perhaps the isal build is adding the rest.

from python-isal.

I git cloned the isa-l library from intel and tried to build it. However I get this issue:

make --no-print-directory all-am
  CC       erasure_code/aarch64/ec_aarch64_dispatcher.lo
In file included from erasure_code/aarch64/ec_aarch64_dispatcher.c:29:
./include/aarch64_multibinary.h:32:2: error: #error "This file is for aarch64 only"
   32 | #error "This file is for aarch64 only"
      |  ^~~~~
erasure_code/aarch64/ec_aarch64_dispatcher.c: In function ‘gf_vect_dot_prod_dispatcher’:
erasure_code/aarch64/ec_aarch64_dispatcher.c:36:22: error: ‘HWCAP_SVE’ undeclared (first use in this function); did you mean ‘HWCAP_SWP’?
   36 |         if (auxval & HWCAP_SVE)
      |                      ^~~~~~~~~
      |                      HWCAP_SWP
erasure_code/aarch64/ec_aarch64_dispatcher.c:36:22: note: each undeclared identifier is reported only once for each function it appears in
erasure_code/aarch64/ec_aarch64_dispatcher.c:38:22: error: ‘HWCAP_ASIMD’ undeclared (first use in this function)
   38 |         if (auxval & HWCAP_ASIMD)
      |                      ^~~~~~~~~~~
erasure_code/aarch64/ec_aarch64_dispatcher.c: In function ‘gf_vect_mad_dispatcher’:
erasure_code/aarch64/ec_aarch64_dispatcher.c:54:22: error: ‘HWCAP_SVE’ undeclared (first use in this function); did you mean ‘HWCAP_SWP’?
   54 |         if (auxval & HWCAP_SVE)
      |                      ^~~~~~~~~
      |                      HWCAP_SWP
erasure_code/aarch64/ec_aarch64_dispatcher.c:56:22: error: ‘HWCAP_ASIMD’ undeclared (first use in this function)
   56 |         if (auxval & HWCAP_ASIMD)
      |                      ^~~~~~~~~~~
erasure_code/aarch64/ec_aarch64_dispatcher.c: In function ‘ec_encode_data_dispatcher’:
erasure_code/aarch64/ec_aarch64_dispatcher.c:72:22: error: ‘HWCAP_SVE’ undeclared (first use in this function); did you mean ‘HWCAP_SWP’?
   72 |         if (auxval & HWCAP_SVE)
      |                      ^~~~~~~~~
      |                      HWCAP_SWP
erasure_code/aarch64/ec_aarch64_dispatcher.c:74:22: error: ‘HWCAP_ASIMD’ undeclared (first use in this function)
   74 |         if (auxval & HWCAP_ASIMD)
      |                      ^~~~~~~~~~~
erasure_code/aarch64/ec_aarch64_dispatcher.c: In function ‘ec_encode_data_update_dispatcher’:
erasure_code/aarch64/ec_aarch64_dispatcher.c:90:22: error: ‘HWCAP_SVE’ undeclared (first use in this function); did you mean ‘HWCAP_SWP’?
   90 |         if (auxval & HWCAP_SVE)
      |                      ^~~~~~~~~
      |                      HWCAP_SWP
erasure_code/aarch64/ec_aarch64_dispatcher.c:92:22: error: ‘HWCAP_ASIMD’ undeclared (first use in this function)
   92 |         if (auxval & HWCAP_ASIMD)
      |                      ^~~~~~~~~~~
erasure_code/aarch64/ec_aarch64_dispatcher.c: In function ‘gf_vect_mul_dispatcher’:
erasure_code/aarch64/ec_aarch64_dispatcher.c:108:22: error: ‘HWCAP_SVE’ undeclared (first use in this function); did you mean ‘HWCAP_SWP’?
  108 |         if (auxval & HWCAP_SVE)
      |                      ^~~~~~~~~
      |                      HWCAP_SWP
erasure_code/aarch64/ec_aarch64_dispatcher.c:110:22: error: ‘HWCAP_ASIMD’ undeclared (first use in this function)
  110 |         if (auxval & HWCAP_ASIMD)
      |                      ^~~~~~~~~~~
make[1]: *** [Makefile:3766: erasure_code/aarch64/ec_aarch64_dispatcher.lo] Error 1
make: *** [Makefile:2352: all] Error 2

It looks like there is a compiler error because "This file is for aarch64 only", but I am running an aarch64 kernel. How can that be?

from python-isal.

You are missing that the #error could be due to some ifdef, and confusion about flavors of aarch64.

from python-isal.

Hey there, I tried to upgrade homeassistant today, which seems to have a dependency on isal.

Please contact home assistent. They should not have added such a niche library without due dilligence. ISA-L is an assembly library, as such it only works on select systems. It does not work on Mac OS arm64 for instance, which is quite a substantial user base at this point. If home assistent claims to support BSD they should make sure they do not list dependencies that only support Windows, Macos and Linux.

That said, I don't mind PRs which enable installation on BSD platforms. It is just that I cannot test those, so I will not call it "support". Also if the home assistent developers want to support BSD and they want to use a particular dependency, the responsibility is on them to ensure that this dependency works on the supported platforms, or that fallback measures are taken so that it is not automatically installed on platforms for which there is no support.

from python-isal.

Home Assistant does not support FreeBSD. That's why I tried fixing it myself.

@tkundrat has no problem with FreeBSD but with Linux on his Raspberry Pi (which works for me without changes)

from python-isal.

Hey there, I tried to upgrade homeassistant today, which seems to have a dependency on isal.

Please contact home assistent. They should not have added such a niche library without due dilligence. ISA-L is an assembly library, as such it only works on select systems. It does not work on Mac OS arm64 for instance, which is quite a substantial user base at this point. If home assistent claims to support BSD they should make sure they do not list dependencies that only support Windows, Macos and Linux.

That said, I don't mind PRs which enable installation on BSD platforms. It is just that I cannot test those, so I will not call it "support". Also if the home assistent developers want to support BSD and they want to use a particular dependency, the responsibility is on them to ensure that this dependency works on the supported platforms, or that fallback measures are taken so that it is not automatically installed on platforms for which there is no support.

I looked it up, they only officially support Windows, macOS and Linux installations. So this should not be an issue.
I am running a Linux based installation (RaspberryPi OS/Debian) on my 64bit RasPi 4B, so isal should be supported

from python-isal.

Ah sorry, due to all the BSD work going on I made some wrong assumptions. Sorry for that.

That is quite strange. It should be able to find the wheel and install that. Could you try and see what happens when you install the wheel directly?

For python 3.12 it should be this one: https://files.pythonhosted.org/packages/ec/25/cf5a18c06a668ee63c33d21e3a902592c6a4418b383938f0973b93761c31/isal-1.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Simply do pip install pip install https://files.pythonhosted.org/packages/ec/25/cf5a18c06a668ee63c33d21e3a902592c6a4418b383938f0973b93761c31/isal-1.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Other wheels can be found here:
https://pypi.org/project/isal/#files

from python-isal.

Please also provide the output of python3 -c "import platform; print(platform.machine())"

from python-isal.

$ python3

Python 3.12.2 (main, Apr 19 2024, 11:23:53) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'aarch64'

from python-isal.

Have you tried installing the wheel directly?

from python-isal.

Agreed that HA should not have added this as a dependency :-)

from python-isal.

Have you tried installing the wheel directly?

Sorry, I did not see that message. Trying it out I get an error:

$ pip install https://files.pythonhosted.org/packages/ec/25/cf5a18c06a668ee63c33d21e3a902592c6a4418b383938f0973b93761c31/isal-1.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
ERROR: isal-1.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl is not a supported wheel on this platform.

from python-isal.

Also I read through the complete changelog and they added aiohttp-isal just in this new release homeassistant==2024.5.0 from 1st May. I checked, and until one version before (2024.4.4, the one I'm currently running) they used aiohttp-zlib-ng (and zlib-ng), so everything was fine for me.

from python-isal.

ERROR: isal-1.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl is not a supported wheel on this platform.

I am affraid I cannot help with that error. For some reason your platform is not "default" aarch64 that is supported by the PyPA wheelbuilding process. As a result I cannot offer any help. I seeHomeAssistent has merged a change now where ISA-L and zlib-ng will be used as fallback options, so in the coming releases you shouldn't have any issues anymore.

from python-isal.