bug: Missing Dependency when running about openllm HOT 14 CLOSED

Did you still run into this issue?

from openllm.

This is strange. Did you run into any issue when running containerize?

from openllm.

Here is the Docker Build ouput
`sudo openllm build flan-t5 --model-id google/flan-t5-xl
Packing 'flan-t5' into a Bento with kwargs={}...
Building Bento for LLM 'flan-t5'
Building Bento for LLM 'flan-t5'

██████╗ ██████╗ ███████╗███╗ ██╗██╗ ██╗ ███╗ ███╗
██╔═══██╗██╔══██╗██╔════╝████╗ ██║██║ ██║ ████╗ ████║
██║ ██║██████╔╝█████╗ ██╔██╗ ██║██║ ██║ ██╔████╔██║
██║ ██║██╔═══╝ ██╔══╝ ██║╚██╗██║██║ ██║ ██║╚██╔╝██║
╚██████╔╝██║ ███████╗██║ ╚████║███████╗███████╗██║ ╚═╝ ██║
╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═══╝╚══════╝╚══════╝╚═╝ ╚═╝

Successfully built Bento(tag="google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce").

Possible next steps:

• Push to BentoCloud with bentoml push:
  $ bentoml push google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce
• Containerize your Bento with bentoml containerize:
  $ bentoml containerize google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce
  Tip: To enable additional BentoML feature for 'containerize', use '--enable-features=FEATURE[,FEATURE]' [see 'bentoml containerize -h' for more advanced usage]

sudo bentoml containerize google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce
Building OCI-compliant image for google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce with docker

[+] Building 300.4s (19/19) FINISHED
=> [internal] load build definition from Dockerfile 0.5s
=> => transferring dockerfile: 3.36kB 0.0s
=> [internal] load .dockerignore 0.5s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04 0.5s
=> [base-container 1/14] FROM docker.io/nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04@sha256:3e1c8a9754bb530b0 40.2s
=> => resolve docker.io/nvidia/cuda:11.6.2-cudnn8-runtime-ubuntu20.04@sha256:3e1c8a9754bb530b01c21be43f65fcc4a74 0.0s
=> => sha256:3e1c8a9754bb530b01c21be43f65fcc4a74c9bec42f61183112e936dda5cbe43 743B / 743B 0.0s
=> => sha256:462ba0e2badc619e410d3a10f0471056553b71a9c48187920d5ec4b92031d713 2.42kB / 2.42kB 0.0s
=> => sha256:2e33cbfb80bf58ced106d63a60762db7d436deb6c2cc793c08cca42aa1f4f017 12.77kB / 12.77kB 0.0s
=> => sha256:ca1778b6935686ad781c27472c4668fc61ec3aeb85494f72deb1921892b9d39e 27.50MB / 27.50MB 0.4s
=> => sha256:65966f22ed3d88416d069fddf13902e152ae54b6c9a04e2787938f831014f9af 7.94MB / 7.94MB 0.5s
=> => sha256:ad6b7dd7c65f1dabb00d95a454994d3664d03efbd0d64ae34fef79a3ec25893a 23.61MB / 23.61MB 0.6s
=> => extracting sha256:ca1778b6935686ad781c27472c4668fc61ec3aeb85494f72deb1921892b9d39e 0.9s
=> => sha256:037a5c0ebcaab7c1b8cfc3c82745d94c7d394e874f8629a33609c9b46360d75f 184B / 184B 0.8s
=> => sha256:7921c0ffa124d1669cc033a479b9ef5a4817f57cb1a089caca43a4ef8b5ab6a1 1.12GB / 1.12GB 11.0s
=> => sha256:ea504443a0386bfe9f1f778e0318641268d763b0415beace78b81b49cee43262 6.89kB / 6.89kB 1.0s
=> => sha256:ca95f35017dce66c17b318dd49cf041bfe6b4705e84014031c4aa5058fa14d57 62.68kB / 62.68kB 0.9s
=> => sha256:651795e98cbbf0ffe858eb5000428947c028af902bb9ef0410a7da9b9488dc69 1.68kB / 1.68kB 1.0s
=> => sha256:b753b50441b1c0c7c235e859ba57459a755d421b828aaecd09b15f77f1c6701d 1.52kB / 1.52kB 1.3s
=> => sha256:6703542d8e6b9b8e8b19e4f5079960adab08b0a7026fa541bd72a11b51b8d588 727.16MB / 727.16MB 9.2s
=> => extracting sha256:65966f22ed3d88416d069fddf13902e152ae54b6c9a04e2787938f831014f9af 0.3s
=> => extracting sha256:ad6b7dd7c65f1dabb00d95a454994d3664d03efbd0d64ae34fef79a3ec25893a 0.6s
=> => extracting sha256:037a5c0ebcaab7c1b8cfc3c82745d94c7d394e874f8629a33609c9b46360d75f 0.0s
=> => extracting sha256:ea504443a0386bfe9f1f778e0318641268d763b0415beace78b81b49cee43262 0.0s
=> => extracting sha256:7921c0ffa124d1669cc033a479b9ef5a4817f57cb1a089caca43a4ef8b5ab6a1 16.3s
=> => extracting sha256:ca95f35017dce66c17b318dd49cf041bfe6b4705e84014031c4aa5058fa14d57 0.0s
=> => extracting sha256:651795e98cbbf0ffe858eb5000428947c028af902bb9ef0410a7da9b9488dc69 0.0s
=> => extracting sha256:b753b50441b1c0c7c235e859ba57459a755d421b828aaecd09b15f77f1c6701d 0.0s
=> => extracting sha256:6703542d8e6b9b8e8b19e4f5079960adab08b0a7026fa541bd72a11b51b8d588 10.7s
=> [internal] load build context 66.7s
=> => transferring context: 11.40GB 66.7s
=> [base-container 2/14] RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Pa 3.6s
=> [base-container 3/14] RUN --mount=type=cache,target=/var/lib/apt --mount=type=cache,target=/var/cache/apt s 26.5s
=> [base-container 4/14] RUN --mount=type=cache,target=/var/lib/apt --mount=type=cache,target=/var/cache/apt 55.7s
=> [base-container 5/14] RUN ln -sf /usr/bin/python3.9 /usr/bin/python3 && ln -sf /usr/bin/pip3.9 /usr/bin/ 0.4s
=> [base-container 6/14] RUN curl -O https://bootstrap.pypa.io/get-pip.py && python3 get-pip.py && rm - 8.9s
=> [base-container 7/14] RUN groupadd -g 1034 -o bentoml && useradd -m -u 1034 -g 1034 -o -r bentoml 0.5s
=> [base-container 8/14] RUN mkdir /home/bentoml/bento && chown bentoml:bentoml /home/bentoml/bento -R 0.3s
=> [base-container 9/14] WORKDIR /home/bentoml/bento 0.0s
=> [base-container 10/14] COPY --chown=bentoml:bentoml ./env/python ./env/python/ 0.0s
=> [base-container 11/14] RUN --mount=type=cache,target=/root/.cache/pip bash -euxo pipefail /home/bentoml/bent 84.3s
=> [base-container 12/14] COPY --chown=bentoml:bentoml . ./ 31.4s
=> [base-container 13/14] RUN rm -rf /var/lib/{apt,cache,log} 0.3s
=> [base-container 14/14] RUN chmod +x /home/bentoml/bento/env/docker/entrypoint.sh 0.4s
=> exporting to image 46.7s
=> => exporting layers 46.7s
=> => writing image sha256:64188b623d999137c14601d043c4f5fdd5d66416d5799070e9f8c4745b7c4e7a 0.0s
=> => naming to docker.io/library/google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce 0.0s
Successfully built Bento container for "google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce" with tag(s) "google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce"
To run your newly built Bento container, run:
docker run -it --rm -p 3000:3000 google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce serve`

from openllm.

You shouldn't run openllm with sudo first and foremost, try again without sudo

from openllm.

It’s not related to sudo.
Having same error without sudo

from openllm.

This seems like the weirdest bug to me. orjson is a core dependency and should already include during install openllm in the container?

Would you mind running bentoml containerize ... --opt progress=plain and dump all of the output to a text file and send it here? Thanks.

from openllm.

I discover a bug on main and I will release a new version patch version promptly

from openllm.

can you try with 0.1.9 to see if you still run into this issue?

from openllm.

Still the same issue

from openllm.

you need to provide bentoml containerize --opt no-cache to remove the previous cache layer. It will still use the previous cache hit afaik

from openllm.

Still the same issue.
We installed everything from scratch on another vm as well, doesn’t work

from openllm.

Hey there, I have identified the issue and will have a patch release soon. sorry for the trouble

from openllm.

v0.1.10 will address this issue. Thanks for spotting this!

from openllm.

I was having the same error. Upgraded to latest version (openllm, version 0.1.17). Build step with --overwrite and containerize step with --opt no-cache. It was solved.

from openllm.