basic lib doesn't work with Ampere ARM processors about ctransformers HOT 6 CLOSED

Thanks it's working
Is it possible to provide aarch64 pypi package with prebuild libctransformers.so for convenience?

from ctransformers.

I'm using GitHub Actions to build binaries but it doesn't support ARM processors currently.
I made it simpler to install from source now. You can just run the following command:

pip install ctransformers --no-binary ctransformers

from ctransformers.

Hi, for ARM processors it has to be built from source as precompiled libs are not available #9 (comment):

git clone --recurse-submodules https://github.com/marella/ctransformers
cd ctransformers
./scripts/build.sh

The compiled binary will be located at build/lib/libctransformers.so which can be used as:

llm = AutoModelForCausalLM.from_pretrained(..., lib='/path/to/ctransformers/build/lib/libctransformers.so')

from ctransformers.

It worked! Thanks for the help ^^

from ctransformers.

@marella thanks
As I saw it's for example possible with qemu here https://github.com/OpenNMT/CTranslate2/blob/50ce6a76920e8eef15a47997353101348871ec3d/.github/workflows/ci.yml#L163

from ctransformers.

Hi, just stumbled on this thread trying to get ctransformers to work on an RPi4. Quick notes:

• Firstly, a deprecation warning

pip install ctransformers --no-binary ctransformers
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453

• Secondly, this installation method results in a unicode error for me:

print(llm(formatted_prompt))
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 49: ordinal not in range(256)

• Lastly, manual installation from source work perfectly, however might it be a good idea to put this in the installation part of the readme, since I had to find it in a closed issue.

Overall love the library! 👍

from ctransformers.