This repository utlizes LLM, or OpenAI ChatGPT API to create an app for question-answering for the contents in documents.

Download python libraries:

pip install -r requirements.txt

To vectorize documents for the question-answering system, use sentence transformers with example instructions here. Convert each section of the document into a single vector, and save to vectors.pkl. The original document can be saved as document.pkl with same order as their corresponding vectors.

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")

# document sections to encode
sections = [
    "Chapter 1 Lorem ipsum dolor sit amet, consectetur adipiscing elit...",
    "Chapter 2 Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua...z",
    "Chapter 3 Ut enim ad minim veniam, quis nostrud exercitation..."
]

# Sentences are encoded by calling model.encode()
embeddings = model.encode(sections)

In terminal or cmd run:

flask run

Question-Answering Application should open up in browser http://127.0.0.1:5000:

The local_llm_app.py script is compatible with Mistral-7B-Instruct-v0.1-GGUF. Download gguf model and set filepath in environment variable MODEL_PATH.

flask run local_llm_app