nisaaragharia / indian-lawyergpt Goto Github PK

Fine-Tuning Falcon-7B, LLAMA 2 with QLoRA to create an advanced AI model with a profound understanding of the Indian legal context.

Home Page: https://huggingface.co/nisaar

Python 0.08% Jupyter Notebook 99.92%

falcon fine-tuning gpt huggingface-transformers llms peft qlora large-language-models llama llama2

indian-lawyergpt's Introduction

Indian Law AI: Fine-Tuning Falcon-7B & LLAMA 2 Language Models

Welcome to our exciting project where we are adapting two cutting-edge language models, Falcon-7B & LLAMA 2, to become proficient in Indian law.

Overview

Our adventure began with a modest 150 Q&As on Indian law. Now, we're charging ahead with an impressive dataset of 3300 instructions! This AI legal project combines:

Falcon-7B & LLAMA 2: State-of-the-art language models, prepped and ready for legal training.
PEFT & QLoRA: The dream duo for memory-efficient and high-performance model fine-tuning.
Our Dataset: Comprehensive Indian law knowledge, spanning constitutional law, civil rights, and more!

Research Paper

Dataset Creation

Dive into our Dataset

Our dataset is designed with four key features: instruction, input, output, and prompt. Crafted to shape our models into AI law experts! Dataset on Hugging Face : https://huggingface.co/datasets/nisaar/Constitution_Of_India_Instruction_Set https://huggingface.co/datasets/nisaar/Articles_Constitution_3300_Instruction_Set https://huggingface.co/datasets/nisaar/LLAMA2_Legal_Dataset_4.4k_Instructions

Fine Tuning Process

Track the Progress

Get a front-row seat to the training progress with TensorBoard. Kickstart it, navigate to the provided localhost link, and witness the models learn:

indian-lawyergpt's People

Contributors

Stargazers

Watchers

indian-lawyergpt's Issues

How can I create own dataset

Facing issue to create a dataset with raw files like multiple pdfs text files or word files how can I directly use them if not how can I convert into json format

Data to finetune

Ways to generate instruct data to fine tune the model. Shall we have the feature to generate the instruction from raw data ?

Stop token during Inference

Why are the responses cut down in the middle?

Colab + Gradio

Hi,
I tried running it on Colab (15GB, Free) with a Gradio interface and the sessions just crashes due to memory issues. Is there a way to run in on colab unless I pay for the Pro version?

Hello,
First of all great work and thanks for sharing - you are democratizing AI to newbies. I tried with my own dataset formatted like Q-A pairs and ran the generated model to infer back on training data itself to find out how much it learn't. I didn't get good results in the sense that it used general knowledge to answer and not the QA I trained on. Any advice?

How is your evaluation on running the inference on training data? It would be interesting to know if there is a metrics which captures how much a model learn't from training data. Thanks,

Viba

how to deploy this chat bot model after pushing it to hugging face? i'm talking about qlora fine tuned model

Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit

Thanks for making this code available on GitHub.

Running this section of your code during inference I am getting the below error.

#change peft_model_id
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    load_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

peft_model_id = "chukypedro/falcon7b-gpt_model"
config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path,
    return_dict=True,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
tokenizer.pad_token = tokenizer.eos_token
model = PeftModel.from_pretrained(model, peft_model_id)

Error Traceback

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 11>:11                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py:485 in          │
│ from_pretrained                                                                                  │
│                                                                                                  │
│   482 │   │   │   │   class_ref, pretrained_model_name_or_path, **hub_kwargs, **kwargs           │
│   483 │   │   │   )                                                                              │
│   484 │   │   │   _ = hub_kwargs.pop("code_revision", None)                                      │
│ ❱ 485 │   │   │   return model_class.from_pretrained(                                            │
│   486 │   │   │   │   pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs,   │
│   487 │   │   │   )                                                                              │
│   488 │   │   elif type(config) in cls._model_mapping.keys():                                    │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:2815 in from_pretrained   │
│                                                                                                  │
│   2812 │   │   │   │   │   key: device_map[key] for key in device_map.keys() if key not in modu  │
│   2813 │   │   │   │   }                                                                         │
│   2814 │   │   │   │   if "cpu" in device_map_without_lm_head.values() or "disk" in device_map_  │
│ ❱ 2815 │   │   │   │   │   raise ValueError(                                                     │
│   2816 │   │   │   │   │   │   """                                                               │
│   2817 │   │   │   │   │   │   Some modules are dispatched on the CPU or the disk. Make sure yo  │
│   2818 │   │   │   │   │   │   the quantized model. If you want to dispatch the model on the CP  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: 
                        Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to 
fit
                        the quantized model. If you want to dispatch the model on the CPU or the disk while keeping
                        these modules in 32-bit, you need to set `load_in_8bit_fp32_cpu_offload=True` and pass a 
custom
                        `device_map` to `from_pretrained`. Check
                        https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-
cpu-and-gpu
                        for more details.