Tiktoken 0.7.0 read-only system error in AWS Lamda about pr-agent HOT 5 OPEN

I tried using the env variable TIKTOKEN_CACHE_DIR pointing to /tmp since it seems it can be used to setup the
temporary cache dir in here with no success, it still creates the temp file in the read only filesystem.

from pr-agent.

We need the new Tiktoken for GPT-4o model, so this new requirement will stay.

You can always use a previous release of PR-Agent without the new Tiktoken, but this seems to be a problem of AWS lambda that should be solved, one way or another.
Maybe contact them, or try other workarounds.

If you do find a workaround, share and we will add it to the docs.

from pr-agent.

I did some debugging trying to find a workaround and the only one I found was setting TIKTOKEN_CACHE_DIR but that didn't work because litellm is overwriting that environment variable (see this PR BerriAI/litellm#1947) so I'm unable to pass a writable directory for AWS Lamda.

This issue is clearly related to this other issue in litellm: BerriAI/litellm#2607.

I'll also report it in that issue to see if there is any fix coming. Until this is fixed I guess AWS Lamda users will not be able to use the latest gpt-4o model with the pr agent.

from pr-agent.