Comments (3)
We adopt left-padding to make the model be compatible with both training and generation since the padding-side must be left during generation.
from llama-factory.
Will this effect performance on chat model? Just saw many other frameworks all using padding right
from llama-factory.
We did not conduct any experiments to validate it. But I thought it would not affect the model's performance.
from llama-factory.
Related Issues (20)
- 不知道什么时候会更新苹果的openELM HOT 1
- chatglm3 llama3-8b 工具微调无法生效问题 HOT 4
- 使用llama-factory导出的微调模型使用transformers加载没生效 HOT 1
- ORPO dataset hiyouga/DPO-En-Zh-20k有问题 HOT 5
- chatglm3-6b工具微调失效
- SFT zero2 zero3下loss不一致 HOT 4
- chatglm3-6b工具微调失效 HOT 2
- 单机双卡4090微调Qwen1.5-7B-Chat爆显存 HOT 4
- 能否支持苹果2024年4月24日开源的模型 HOT 1
- ppo 训练过程中 报 RuntimeError: 'weight' must be 2-D HOT 1
- 关于llama3base版本的评测 HOT 5
- 关于如何使用DPO微调Llama3-8b HOT 1
- quantize.sh 量化Llama3-8b 显存不足 HOT 2
- Could you please share some tips with your rich experience? HOT 1
- Support for RLAIF methods HOT 1
- G
- adapter_config.json
- 评估集上loss、学习率为0 HOT 1
- windows下baichuan2训练异常 HOT 2
- sft后更换数据集继续sft,应该如何进行? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llama-factory.