Refind is a project to store my personnal documents in a Weaviate vector database and use OpenAI vectorizer module and generative search.

Here is a shell recording with asciinema showing the different answers with the same input question from :

1. the current ChatGPt version (using sgpt cli)
2. the generative search with Weaviate containing a sample document from the Enteprise Roadmap to SRE book

Create and provide your OpenAI API key :

export OPENAI_APIKEY="..."

Put full-text documents with less than 4097 tokens. If your prompt is 4000 tokens, your completion can be 97 tokens at most.

Let's use it :

# start the weaviate database
docker-compose up -d

# verify it is up and running
curl http://localhost:8080/v1/meta | jq .
docker-compose logs

# import data, DO IT ONCE
go run main.go import

# query data related to software
go run main.go query | jq .

# when done, gracefully shutdown
docker-compose down

In this example, Weaviate is returning software-related entries.

Weaviate operations :

# get the schema
curl -s http://localhost:8080/v1/schema | jq .

# get objects
curl -s http://localhost:8080/v1/objects | jq .

# get one class from the schema
curl -s http://localhost:8080/v1/schema/Chatbot | jq .

# delete a class
curl -s -XDELETE http://localhost:8080/v1/schema/Chatbot | jq .

The ChatGPT Retrieval plugin.

1. automatically add more metadata in weaviate when ingesting documents
2. automatically split documents bigger than the OpenAI token limit
3. add query string as a flag when running the query command
4. unit tests, let Copilot write them \o/

README.md