How to configure to use Tavily as Search Engine about gpt-researcher HOT 5 CLOSED

Similar issue, however i get rate limit. it should be using the Tavily key. I have it in the .env and in exported, seems to still use duck duck go

full error

RROR: Exception in ASGI application
Traceback (most recent call last):

Error: 400 Client Error: Bad Request for url: https://api.tavily.com/search

File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\duckduckgo_search\duckduckgo_search.py", line 46, in _get_url
raise DuckDuckGoSearchException("Ratelimit")
duckduckgo_search.exceptions.DuckDuckGoSearchException: Ratelimit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\uvicorn\protocols\websockets\wsproto_impl.py", line 234, in run_asgi
result = await self.app(self.scope, self.receive, self.send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\fastapi\applications.py", line 1106, in call
await super().call(scope, receive, send)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\starlette\applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\starlette\middleware\errors.py", line 149, in call
await self.app(scope, receive, send)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in call
raise exc
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in call
raise e
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in call
await self.app(scope, receive, send)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\starlette\routing.py", line 718, in call
await route.handle(scope, receive, send)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\starlette\routing.py", line 341, in handle
await self.app(scope, receive, send)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\starlette\routing.py", line 82, in app
await func(session)
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\fastapi\routing.py", line 325, in app
await dependant.call(**values)
File "C:\Users\chris\ai\gpt-researcher\backend\server.py", line 50, in websocket_endpoint
report = await manager.start_streaming(task, report_type, websocket)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\utils\websocket_manager.py", line 57, in start_streaming
report = await run_agent(task, report_type, websocket)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\utils\websocket_manager.py", line 75, in run_agent
report = await researcher.run()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\report_type\basic_report\basic_report.py", line 17, in run
await researcher.conduct_research()
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\master\agent.py", line 79, in conduct_research
self.context = await self.get_context_by_search(self.query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\master\agent.py", line 148, in get_context_by_search
context = await asyncio.gather(*[self.process_sub_query(sub_query) for sub_query in sub_queries])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\master\agent.py", line 162, in process_sub_query
scraped_sites = await self.scrape_sites_by_query(sub_query)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\master\agent.py", line 200, in scrape_sites_by_query
new_search_urls = await self.get_new_urls([url.get("href") for url in search_results])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\gpt_researcher\master\agent.py", line 200, in
new_search_urls = await self.get_new_urls([url.get("href") for url in search_results])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\duckduckgo_search\duckduckgo_search.py", line 91, in text
for i, result in enumerate(results, start=1):
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\duckduckgo_search\duckduckgo_search.py", line 142, in _text_api
resp = self._get_url("GET", "https://links.duckduckgo.com/d.js", params=payload)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chris\ai\gpt-researcher\env\Lib\site-packages\duckduckgo_search\duckduckgo_search.py", line 50, in _get_url
raise DuckDuckGoSearchException(f"_get_url() {url} {type(ex).name}: {ex}")
duckduckgo_search.exceptions.DuckDuckGoSearchException: _get_url() https://links.duckduckgo.com/d.js DuckDuckGoSearchException: Ratelimit

from gpt-researcher.

Ddgo is used as fallback if Tavily fails. Did you manage to validate that you can search queries with Tavily? If not, consider another search provider

from gpt-researcher.

@assafelovic I am getting this error when i am trying from my Mac.
Error: HTTPSConnectionPool(host='api.tavily.com', port=443): Max retries exceeded with url: /search (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)')))

Why it is throwing SSL certificate error. Note I am running inside VPN

from gpt-researcher.

Its working now.. Please close this issue.

from gpt-researcher.

Tavily is working. Thanks @assafelovic

from gpt-researcher.