Comments (3)
karthiksoman
commented on August 19, 2024
I see! Can you please share the exception raised while trying this Llama model?
The use of the official Llama model was motivated by the aim to conduct a comparative analysis with GPT models. I thought the performance comparison would be fair if we use official Llama model instead of using a quantized version of it.
from kg_rag.
namin
commented on August 19, 2024
$ python -m kg_rag.run_setup
Starting to set up KG-RAG ...
Did you update the config.yaml file with all necessary configurations (such as GPT .env path, vectorDB file paths, other file paths)? Enter Y or N: Y
Checking disease vectorDB ...
vectorDB already exists!
Do you want to install Llama model? Enter Y or N: Y
Did you update the config.yaml file with proper configuration for downloading Llama model? Enter Y or N: Y
Are you using official Llama model from Meta? Enter Y or N: Y
Did you get access to use the model? Enter Y or N: Y
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. If you see this, DO NOT PANIC! This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
Model is not downloaded! Make sure the above mentioned conditions are satisfied
Congratulations! Setup is completed.
and
$ python -m kg_rag.rag_based_generation.Llama.text_generation interactive
...
Press enter for Step 5 - LLM prompting
Prompting llama
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. If you see this, DO NOT PANIC! This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
Traceback (most recent call last):
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/scratch/namin/KG_RAG/kg_rag/rag_based_generation/Llama/text_generation.py", line 52, in <module>
main()
File "/scratch/namin/KG_RAG/kg_rag/rag_based_generation/Llama/text_generation.py", line 43, in main
interactive(question, vectorstore, node_context_df, embedding_function_for_context_retrieval, "llama")
File "/scratch/namin/KG_RAG/kg_rag/utility.py", line 344, in interactive
llm = llama_model(config_data["LLAMA_MODEL_NAME"], config_data["LLAMA_MODEL_BRANCH"], config_data["LLM_CACHE_DIR"], stream=True)
File "/scratch/namin/KG_RAG/kg_rag/utility.py", line 133, in llama_model
tokenizer = AutoTokenizer.from_pretrained(model_name,
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 736, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1854, in from_pretrained
return cls._from_pretrained(
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2017, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 128, in __init__
self.update_post_processor()
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 141, in update_post_processor
bos_token_id = self.bos_token_id
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1141, in bos_token_id
return self.convert_tokens_to_ids(self.bos_token)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 277, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 284, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1160, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 277, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 284, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1160, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 277, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 284, in _convert_token_to_id_with_added_voc
return self.unk_token_id
...
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1160, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 277, in convert_tokens_to_ids
return self._convert_token_to_id_with_added_voc(tokens)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 284, in _convert_token_to_id_with_added_voc
return self.unk_token_id
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1160, in unk_token_id
return self.convert_tokens_to_ids(self.unk_token)
File "/home/namin/mambaforge/envs/kg_rag2/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1040, in unk_token
return str(self._unk_token)
RecursionError: maximum recursion depth exceeded while getting the str of an object
from kg_rag.
karthiksoman
commented on August 19, 2024
Issue was with the tokenizer that was used to download the model. Apparently, PMC_Llama required 'LlamaTokenizer' and for meta Llama 'AutoTokenizer' was used. In addition, PMC_Llama required to configure an additional 'legacy' flag.
I have made those changes and it was successfully downloaded to my server. Please note the following:
-
run_setup.py is updated now. You can run it and it should download PMC_Llama (just follow the instructions which comes interactively).
-
Even though it downloads PMC_Llama, for me, it worked only on the prompt_based mode. In the rag based mode, no exceptions are raised, but it was just printing back the question and the context without generating any response. I presume, unlike vanilla Llama, this may require some specific way of prompting (this is a guess). I haven't explored much of PMC_Llama using KG-RAG. Please let me know how it goes for you.
-
I have now changed the command line arguments to make KG-RAG run flexible.
For interactive mode: -i True (if you dont give this, by default it will be non-interactive)
For choosing gpt model : -g gpt-4 (if you dont give this, by default it chooses gpt-35-turbo)
For running PMC_Lama : -m method-2 (If you dont give this, by default it chooses 'method-1' which is using 'AutoTokenizer')
So for example, if you want to run gpt-4 interactive mode:
python -m kg_rag.rag_based_generation.GPT.text_generation -i True -g gpt-4
All video demos in README are updated based on these changes and also I have cut a new release of KG-RAG to reflect these changes.
I am closing this issue, since this should address the download of PMC_Llama. Feel free to re-open it, if you hit any wall.
from kg_rag.
Related Issues (17)
- issues with key format HOT 10
- Disease_ With_ Relationship_ To_ genes.pickle file HOT 5
- Error in retrieving context HOT 8
- run time HOT 7
- meta-llama HOT 1
- errors when installing dependencies HOT 1
- Use other knowledge graphs HOT 3
- OSError: [Errno 101] Network is unreachable HOT 7
- Error in retrieving context for some diseases HOT 7
- No response from SPOKE api HOT 1
- vectorstore HOT 1
- ERROR: Could not build wheels for llama_cpp_python, which is required to install pyproject.toml-based projects HOT 1
- disease_nodes_db HOT 2
- Issues setting up KGRAG on macOS
- How to imply my own model
- The use of llama 3.1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kg_rag.