Alternatives to You.com search API about storm HOT 14 OPEN

Yeah, I'll try to integrate more search APIs into storm.

from storm.

Hi, thanks for your interest!

We view this project as an example of a knowledge curation engine that serves as the intermediate layer between vase unstructured information and human. So, supporting different information sources is in our plan.

For the pointers you provide, are you willing to open a PR for integration? Happy to help merge it.

from storm.

Great, thank you!

from storm.

If we have plans and a to-do list, I'd like to claim some tasks to help.

from storm.

@LronDC Thank you for your interest in our project! We're currently working on an upcoming code release that will enhance the scalability of the project. We will keep you updated and soon share some potential tasks where the community can contribute. Stay tuned!

from storm.

@shaoyijia Please review this pull request #20.

1. Support DuckDuckGoSearchAPI and TavilySearchAPI as Alternatives to You.com.
2. When enabling TopicExpert to use DuckDuckGoSearchAPI or TavilySearchAPI, these APIs will return compelete contents instead of snippets as default.
3. One can setup the search API through editing these environments in secrets.toml:
   Set WEB_SEARCH_API as one of ['DuckDuckGoSearchAPI', 'TavilySearchAPI', 'YouSearchAPI'], using YouSearchAPI as default
   Setup You.com search API key by YDC_API_KEY=<your_youcom_api_key>
   Setup api.tavily.com search API key by TAVILY_API_KEY=<your_api_tavily_com_key>

from storm.

@shaoyijia Considering supporting different information sources, I recommend you to use our open-source project, i.e., QAnything. QAnything is a local knowledge base question-answering system designed to support a wide range of file formats and databases.

from storm.

sadly, we cannot access to You.com。

from storm.

@dl942702882 You.com offers free tier of api quota. It’s sufficient to write more than 25 articles locally.

from storm.

@dl942702882 , for switching to customized sources (before we support this officially), maybe you can check out what this PR (#20) tries to do?

from storm.

An update in this thread:

We just release the refactored code to make it easier to run/customize/develop the STORM engine. Now, search API, retrieval model integration in src/rm.py. The knowledge curation engine will directly consume the Information output by Retriever.

from storm.

@LronDC @songkq , we are now specifically interested in supporting:

1. Retrieval models that can retrieve information from customized source.
2. Search API that return academic sources, e.g., Semantic Scholar API

Contribution is highly appreciated if you are interested!

from storm.

@shaoyijia Hi, I'll support the Semantic Scholar API soon after the API key is obtained.

from storm.

Hi @songkq , thank you so much! I have a Semantic Scholar API so can also test it.

from storm.