CPU Execution about mt3 HOT 4 CLOSED

Did you have any luck with this? I'm also interested. I'm fairly new to GPU programming, but it seems like it would probably be better/easier to leverage the Metal GPU than to get rid of the GPU support entirely, no? i.e., switch from the Cuda functions to the Metal functions: https://github.com/ggerganov/llama.cpp/blob/master/ggml-metal.m

from mt3.

hey zane, i did end up getting it to work on mac m1, and then later in linux based docker using linux/arm64 machine. it was very involved. I had to build tensorflow-text, jax, and jaxlib from source for my python version (3.8) and machine aarch64. couldnt find a better way. the dockerfile is 2.1gb :(.

from mt3.

I also needed to make quite a few modifications to mt3 and t5x setup.py files and scripts in order to use different versions of libraries only available on python 3.8.

from mt3.

Would you mind sharing the Dockerfile? I’ve been piecing one together myself but was in the middle of dealing with the versioning hell that is all of Python, as you alluded.

from mt3.