Goes on forever? about gpt-researcher HOT 3 CLOSED

To follow up with my previous comment: after hours of waiting, this is the error I get:

from gpt-researcher.

Another reason why this model takes up so much time and resources is that the model processes every character.

As you can see the queries: [“Sri Lanka…]. The model first looks up ‘[‘ and scrapes all the websites to understand the grammar behind ‘[‘. And then it does the same for the apostrophe, letter S and so on. I believe this is not how it should be done.

As you can see in the image, it is learning about the letter I in “Sri”.

from gpt-researcher.

HI @madiha1ahmed, good catch on youtube scraping, we will prevent it.

Regarding your issue, are you using gpt 4 or 3.5? the json produced from the generate_queries function was bad - a known issue when using gpt-3.5. You should simply run it again.
The avg time depends on many factors, computing power, internet access, latency, and openAI of course. However, longer than 10 minutes will indicate an issue.

Hope it helps

from gpt-researcher.