Please report location when HuggingFaceTokenizer.newInstance fails with I/O error about djl HOT 11 CLOSED

@wnm3
When use use "intfloat/multilingual-e5-base", you are actually download the tokenizer from Huggingface HUB. It will save the tokenizer.json file in $HUGGINGFACE_HUB_CACHE directory (default ~/.cache/huggingface/hub). See: https://huggingface.co/docs/huggingface_hub/en/guides/manage-cache

Please make sure the cache folder is writeable. You can also manually set env var to point it to a desired directory. see example: https://github.com/deepjavalibrary/djl-serving/blob/master/serving/docker/Dockerfile#L44

from djl.

I noticed you are using DJL model zoo, any reason you manually create your tokenizer?

If you use our built-in TranslatorFactory, we will load the tokenizer from the model directory instead of download from Huggingface hub. See: https://github.com/deepjavalibrary/djl/blob/master/extensions/tokenizers/src/main/java/ai/djl/huggingface/translator/TextEmbeddingTranslatorFactory.java#L60-L64

from djl.

I'd been handed code that worked, so I didn't investigate alternatives ;^) -- the API you showed has more parameters. I'm happy to switch if you could provide a working example, as I'm not familiar with the input/output nor the model since we are passing a single string and map of options. Thanks in advance.

from djl.

I tried setting the env variables (the article showed HF_HUB_CACHE rather than HUGGINGFACE_HUB_CACHE) to a directory with rwx for all levels and still get the I/O error -- do I need to create subdirectories with all permissions? Still using the download...

default@37e5f86e600b:/opt/ol/wlp/output/defaultServer$ pwd
/opt/ol/wlp/output/defaultServer
default@37e5f86e600b:/opt/ol/wlp/output/defaultServer$ ls -l |grep data
drwxrwxrwx. 44    1000 1000       8192 Feb 14 06:33 data
default@37e5f86e600b:/opt/ol/wlp/output/defaultServer$ env |grep HUB
HF_HUB_CACHE=/opt/ol/wlp/output/defaultServer/data
HUGGINGFACE_HUB_CACHE=/opt/ol/wlp/output/defaultServer/data

from djl.

Here is an example take String as input and output a float[]: https://github.com/deepjavalibrary/djl-demo/blob/master/huggingface/nlp/src/main/java/com/examples/TextEmbedding.java

You can set Tokenizer parameters like this:

The huggingface tokenizer source code is here: https://github.com/huggingface/tokenizers/tree/main/tokenizers

from djl.

Will the API using newInstance I first asked about first check the .cache for the existence of the model before attempting to download it? If so, I can build the container with the model set up in ~/.cache...

from djl.

I don't recommend you setup ~/.cache directory. I's quite complicated and HF can change their folder structure in future.

Setting HF_HUB_CACHE should be the right solution. Can you check if you point HF_HUB_CACHE to a new folder, does the folder get created?

from djl.

@frankfliu thanks for your suggestions. I'm stuck because the newInstance for the intfloat/multilingual-e5-base calls JNI code that is attempting to save downloaded content to ~/.cache/huggingface/hub and I've tried environment variables to point it elsewhere but neither HF_HUB_CACHE nor HUGGINGFACE_HUB_CACHE seems to work. If the JNI code is in a github I can try to read to find what controls this location. I'm attempting to load the e5 model in a docker container and it fails with permissions issues for the home directory. Feel free to reach out directly if desired [email protected] / [email protected]

from djl.

The env variable I needed was HF_HOME -- it was in the Docker document you'd provided. Thank you.

from djl.

It's a bit confusion.

Based Huggingface website, HUGGINGFACE_HUB_CACHE is the old env var, Huggingface has changed it to HF_HUB_CACHE.

However, I checked their rust implementation, it seems only HF_HOME is implemented in their rust library: https://github.com/huggingface/hf-hub/blob/main/src/lib.rs#L195

from djl.

Thank you Frank. Ideally, the exception thrown could identify where it was attempting to write to, and possibly reference the env var's name ;^) I'll close this issue.

from djl.