leehanchung / lora-instruct Goto Github PK

View Code? Open in Web Editor NEW

97.0 3.0 11.0 21.22 MB

Finetune Falcon, LLaMA, MPT, and RedPajama on consumer hardware using PEFT LoRA

License: Apache License 2.0

Python 58.59% Jupyter Notebook 40.67% Shell 0.74%

agi gpt llm lora nlp falcon llama mpt redpajama

lora-instruct's Introduction

🧑‍🏫🤏 LoRA-Instruct

This repository contains code for fine-tuning permissive open source LLMs using low-rank adaptation (LoRA).

Code is tested using Stanford Alpaca dataset.

Estimated training time for fine-tuning RedPajama-INCITE-Base-7B-v0.1 with a single RTX 3090 and Stanford Alpaca is ~12 hours.
Estimated training time for fine-tuning RedPajama-INCITE-Base-7B-v0.1 with RTX 3090 and RTX Titan and Stanford Alpaca is ~6.5 hours.
Currently only supports LoRA Instruct fine-tuning RedPajama-INCITE-Base-7B-v0.1.

Inspired by Alpaca-LoRA

Trained Models

Model	Runs	Training Time
LLaMA 3B	⬜
LLaMA 7B	⬜
RedPajama 3B	✅	1:44:14
RedPajama 7B	✅	3:09:58
MPT 3B	⬜
MPT 7B	⬜
Falcon 7B	✅

Training Hardware Spec

Ubuntu 20.04.1 LTS (WSL2)

Driver Version: 531.41
CUDA Version: 12.1
cuDNN version: 8.5.0

Local Setup

Install dependencies

poetry install

To fine-tune using NVidia 2000 series GPU or earlier, please comment out this line in finetune.py

model = prepare_model_for_int8_training(model)

Training (`finetune.py`)

This file contains a straightforward application of PEFT / LoRA to decoder only model, as well as some code related to prompt construction and tokenization.

Example usage:

python finetune.py \
    --base_model 'togethercomputer/RedPajama-INCITE-Base-7B-v0.1' \
    --output_dir './lora-redpajama'

Distributed Training with 🤗 Accelerate

We uses HuggingFace's accelerate library for distributed training. The following is an example for distributed training with two GPUs.

NOTE: please set the following environment variables

export WORLD_SIZE=2
export CUDA_VISIBLE_DEVICES=0,1

torchrun \
    --nproc_per_node=2 \
    --master_port=1234 \
    finetune.py \
    --base_model 'togethercomputer/RedPajama-INCITE-Base-7B-v0.1' \
    --output_dir './lora-redpajama'

References

lora-instruct's People

Contributors

Stargazers

Watchers

Forkers

rfsfreitas if001 teraoka-hiroshi nickydark1 mansurul11 zeuscsc amanda-dsouza techthiyanes wesley7137 zkyzq dimwap ash-hun

lora-instruct's Issues

How could we make predict using the trained lora weights

Can this codebase be applicable for finetuning larger models, e.g., falcon-40b?

Error message during training

Hi, I truncated "alpaca_data.json" to reduce training time and saved it in another file for training. Unfortunately, using this file gives me the error "pyarrow.lib.ArrowInvalid: JSON parse error: Column() changed from object to array in row 0".

Training on Colab possible?

Hi, I'd like to know if training on Colab is possible

Support for QLORA

Hello. Is there support for quantized lora finetuning of those llms?
Thanks

Falcon -7B training loss not reducing

Thanks for the wonderful code. While training Falcon-7B on Alpaca dataset, the training loss is not reducing. It used to work fine. Were there any recent changes?

Can you please release the inference code?

Thanks for the wonderful training code. Please release the inference code as well.

Support for mpt-30b

Hey @leehanchung

Does lora-instruct support fine tuning mpt-30b family of models out of the box?

MPTForCausalLM.forward() got an unexpected keyword argument 'inputs_embeds'

i 'm trying to finetune MPT-7b instruct, but getting this error.

File "/home/sulabh/new_gen_data/lora-instruct/finetune.py", line 383, in train
    trainer.train(resume_from_checkpoint=resume_from_checkpoint)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 1664, in train
    return inner_training_loop(
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 1940, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 2735, in training_step
    loss = self.compute_loss(model, inputs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/transformers/trainer.py", line 2767, in compute_loss
    outputs = model(**inputs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/peft/peft_model.py", line 678, in forward
    return self.base_model(
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/lora-instruct/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
TypeError: MPTForCausalLM.forward() got an unexpected keyword argument 'inputs_embeds'

Error message when training MPT-7B

Hi I got a message when I try to use Lora to train MPT-B, do you have any ideas to solve it?

ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (instruction in this case) have excessive nesting (inputs type list where type int is expected).