pixegami / langchain-rag-tutorial Goto Github PK

A simple Langchain RAG application.

Python 100.00%

langchain-rag-tutorial's Introduction

Langchain RAG Tutorial

Install dependencies

Do the following before installing the dependencies found in requirements.txt file because of current challenges installing onnxruntime through pip install onnxruntime.
- For MacOS users, a workaround is to first install onnxruntime dependency for chromadb using:
```
 conda install onnxruntime -c conda-forge
```
See this thread for additonal help if needed.
- For Windows users, follow the guide here to install the Microsoft C++ Build Tools. Be sure to follow through to the last step to set the enviroment variable path.
Now run this command to install dependenies in the requirements.txt file.

pip install -r requirements.txt

Install markdown depenendies with:

pip install "unstructured[md]"

Create database

Create the Chroma DB.

python create_database.py

Query the database

Query the Chroma DB.

python query_data.py "How does Alice meet the Mad Hatter?"

You'll also need to set up an OpenAI account (and set the OpenAI key in your environment variable) for this to work.

Here is a step-by-step tutorial video: RAG+Langchain Python Project: Easy AI/Chat For Your Docs.

langchain-rag-tutorial's People

Contributors

Stargazers

Watchers

Forkers

mdwoicke nazdik brotto alessandrozaccari radtonev illuminate7777 rndaorg mindxfuture duythanh90 hestie-s taner45 pragya5 18abhi89 phanibhushanksa imkaslam lancherba cmeals2507 chenchenick wangjiecamen internetuser2008 ash-cortez gpillai7 realsahilsaini ccaiccie radidam eletay rerm06 zac-t-b clger007 davidmonterocrespo24 kyojin11 jimmie-rissanen chakradharkowsik eshaan32 fwicaksono mahmoodbabalola thihaaung11 shivang98 alexzahar mcgidoug rin0o1 chawpohchia lsakole marco-commits genesis801 damamaimouna f1dave noumanjavaid96 haroldyoung isaacbabsky pragyapr06 maikthulhu wineny mnazmussakib serdalaslantas sreemaee21 seeli-rag yulinrong tuantranf fiyinfoluwa6 faisal-523 daavemitt grjus sandipd5 nina-tan fniesel cpepper96 d1ee7p2u9 fakhruddin90 hem pkboom grantad ai-rafique tushar-gupta11 autonomiczny nahyunkwon eddieayala9965 kingly123 mayso2025 phillipebarreto1 andrei2002petcu shadowsadi jsl5710 montymenon barkoczy howardchiang2 talshub chaitu328 knaarke juancarlosbaezpozos agrawalnishant cavecafe ghoshsrinjoy alwageed chiaweh2 amenalahassa muttfacejohnson ravindragr7 tonystarc97 tonytromp

langchain-rag-tutorial's Issues

Add packages versions in requirements.txt

I am having a hard time to solve some dependency issues with the needed packages. Can you also include the versions in the txt file?

creating database does not work anymore

apparently the import statements do not work anymore since langchain v0.2
https://python.langchain.com/v0.2/docs/versions/v0_2/

While running python3 create_database.py

I am getting following error on my mac

/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain/_api/module_import.py:92: LangChainDeprecationWarning: Importing DirectoryLoader from langchain.document_loaders is deprecated. Please replace deprecated imports:

from langchain.document_loaders import DirectoryLoader

with new imports of:

from langchain_community.document_loaders import DirectoryLoader
You can use the langchain cli to automatically upgrade many imports. Please see documentation here https://python.langchain.com/v0.2/docs/versions/v0_2/
warn_deprecated(
/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain/_api/module_import.py:92: LangChainDeprecationWarning: Importing OpenAIEmbeddings from langchain.embeddings is deprecated. Please replace deprecated imports:

from langchain.embeddings import OpenAIEmbeddings

with new imports of:

from langchain_community.embeddings import OpenAIEmbeddings
You can use the langchain cli to automatically upgrade many imports. Please see documentation here https://python.langchain.com/v0.2/docs/versions/v0_2/
warn_deprecated(
/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain/_api/module_import.py:92: LangChainDeprecationWarning: Importing Chroma from langchain.vectorstores is deprecated. Please replace deprecated imports:

from langchain.vectorstores import Chroma

with new imports of:

from langchain_community.vectorstores import Chroma
You can use the langchain cli to automatically upgrade many imports. Please see documentation here https://python.langchain.com/v0.2/docs/versions/v0_2/
warn_deprecated(
Error loading file data/books/alice_in_wonderland.md
Traceback (most recent call last):
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 60, in
main()
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 14, in main
generate_data_store()
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 18, in generate_data_store
documents = load_documents()
^^^^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 25, in load_documents
documents = loader.load()
^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain_community/document_loaders/directory.py", line 117, in load
return list(self.lazy_load())
(myenv) prismetrictech~$python3 create_database.py
/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain/_api/module_import.py:92: LangChainDeprecationWarning: Importing DirectoryLoader from langchain.document_loaders is deprecated. Please replace deprecated imports:

from langchain.document_loaders import DirectoryLoader

with new imports of:

from langchain_community.document_loaders import DirectoryLoader
You can use the langchain cli to automatically upgrade many imports. Please see documentation here https://python.langchain.com/v0.2/docs/versions/v0_2/
warn_deprecated(
/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain/_api/module_import.py:92: LangChainDeprecationWarning: Importing OpenAIEmbeddings from langchain.embeddings is deprecated. Please replace deprecated imports:

from langchain.embeddings import OpenAIEmbeddings

with new imports of:

from langchain_community.embeddings import OpenAIEmbeddings
You can use the langchain cli to automatically upgrade many imports. Please see documentation here https://python.langchain.com/v0.2/docs/versions/v0_2/
warn_deprecated(
/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain/_api/module_import.py:92: LangChainDeprecationWarning: Importing Chroma from langchain.vectorstores is deprecated. Please replace deprecated imports:

from langchain.vectorstores import Chroma

with new imports of:

from langchain_community.vectorstores import Chroma
You can use the langchain cli to automatically upgrade many imports. Please see documentation here https://python.langchain.com/v0.2/docs/versions/v0_2/
warn_deprecated(
Error loading file data/books/alice_in_wonderland.md
Traceback (most recent call last):
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 60, in
main()
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 14, in main
generate_data_store()
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 18, in generate_data_store
documents = load_documents()
^^^^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/create_database.py", line 25, in load_documents
documents = loader.load()
^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain_community/document_loaders/directory.py", line 117, in load
return list(self.lazy_load())
^^^^^^^^^^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain_community/document_loaders/directory.py", line 182, in lazy_load
yield from self._lazy_load_file(i, p, pbar)
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain_community/document_loaders/directory.py", line 220, in _lazy_load_file
raise e
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain_community/document_loaders/directory.py", line 210, in _lazy_load_file
for subdoc in loader.lazy_load():
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain_community/document_loaders/unstructured.py", line 88, in lazy_load
elements = self._get_elements()
^^^^^^^^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/langchain_community/document_loaders/unstructured.py", line 180, in _get_elements
return partition(filename=self.file_path, **self.unstructured_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/unstructured/partition/auto.py", line 414, in partition
_partition_md = _get_partition_with_extras("md")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/System/Volumes/Data/Storage/Technologies/LLM-AI/langchain-rag-tutorial-main/myenv/lib/python3.11/site-packages/unstructured/partition/auto.py", line 125, in _get_partition_with_extras
raise ImportError(
ImportError: partition_md is not available. Install the md dependencies with pip install "unstructured[md]"

issue when running the create_database.py

I am really new in this area. I just download your code and try to run it in the anaconda environment using PyCharm. But I met below error. I have installed all the packages mentioned in the requirements.txt. Could you please comment what happened here? Thanks.

Traceback (most recent call last):
File "C:\Users\PycharmProjects\rag_tutorial\create_database.py", line 1, in
from langchain.document_loaders import DirectoryLoader
File "C:\Users\AppData\Local\anaconda3\envs\pytorch\lib\site-packages\langchain_init_.py", line 8, in
from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
File "C:\Users\AppData\Local\anaconda3\envs\pytorch\lib\site-packages\langchain\agents_init_.py", line 2, in
from langchain.agents.agent import Agent
File "C:\Users\AppData\Local\anaconda3\envs\pytorch\lib\site-packages\langchain\agents\agent.py", line 2
from future import annotations
^
SyntaxError: future feature annotations is not defined

Process finished with exit code 1

Add reference to YouTube explanation of this code

Suggest to mention https://www.youtube.com/watch?v=tcqEUSNCn8I in the ReadMe

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.