Connection time out with OpenAI API about spacy-llm HOT 3 OPEN

Hi @pecto2020, this seems like you're experiencing connection issues with the OpenAI API. Under which circumstances does this happen?

from spacy-llm.

Hi @rmitsch , the issue arises when I process multiple texts, each around 4,000 tokens in size. I attempted to adjust the batch parameters in the configuration to address this, but I continued to encounter the same error. Currently, I've adapted my approach to process the text files individually, sending them to OpenAI one at a time for named entity recognition and relationship extraction. Added also sleep(5) after each iteration. Yet, the time request error persists. Additionally, I've noticed that sometimes the GPT's raw output presents labels in bullet points, as opposed to being comma-separated. It appears that when this formatting occurs, the likelihood of a time request error increases, though I'm not sure of a direct correlation (I see they are not correctly parsed by spacy-llm, so basically the information extracted by the model is lost with an empty json or I got time error request)

Is there anything else you'd like to know about the circumstances?

from spacy-llm.

the issue arises when I process multiple texts, each around 4,000 tokens in size. I attempted to adjust the batch parameters in the configuration to address this, but I continued to encounter the same error.

You're running into rate limitations on OpenAI's side. Unfortunately there isn't anything we can do about that. The batch size config won't make a difference here, but you'll want to experiment with interval and max_request_time.

Additionally, I've noticed that sometimes the GPT's raw output presents labels in bullet points, as opposed to being comma-separated. It appears that when this formatting occurs, the likelihood of a time request error increases, though I'm not sure of a direct correlation

I don't see a correlation here either. LLM output is not guaranteed to be consistent. Setting temperature to 0 is usually a good start, if you haven't done that yet.

from spacy-llm.