Comments (3)
@aksh555 Thanks for reporting! Yes this should be fixed easily and I will try to fix it in 1 week. You can also try to directly modify it into AutoModel
and then see if that works for your case! Thanks!
from lorahub.
@aksh555 Hi, thanks for your interest on our work! Yes definitely it should work well. First, you should train LoRA module based on your base model (e.g., Llama), then you can use https://github.com/sail-sg/lorahub/blob/main/example.py to do lorahub learning.
from lorahub.
Thanks @SivilTaram but the current algorithm.py supports only AutoModelForSeq2SeqLM
, so for using with Llama like model, the loading and dataset preparation needs to be changed?
from lorahub.
Related Issues (20)
- Is there any scalability issue? HOT 2
- About training examples HOT 1
- About LoraHub adapters HOT 3
- Request for Guidance on Reproducing Experiments for BigBenchHard HOT 8
- Bug Report - (batched option & tensor shape) HOT 4
- No single LoRA model can score on Disambiguation HOT 5
- Where is the instruction of each lora in lorahub? HOT 1
- Training Tasks and Data for these LoRA modules? HOT 8
- Evaluation using Glue Lorahub Modules HOT 4
- can lorahub be used in non llm tasks HOT 3
- Release of Code for Finetuning Lora modules of FLAN and evaluating the Finetuned Models. HOT 1
- FLAN datasets not available on Huggingface. HOT 10
- repaired an error in reproduce_bbh.py (shuffle seed) HOT 2
- train_model model load repaired HOT 4
- 404 HOT 2
- AttributeError: 'GenerationConfig' object has no attribute 'cache_implementation' HOT 4
- Downloading the FLAN-v2 dataset HOT 1
- How to evaluate glue lora?
- How to use local LoRA trained with train_model.py HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lorahub.