Comments (1)
65B is difficult to train so the exact date is not clear.
We plan to first release 13B models, which should come before next Mar 14. And then 30B models. According to my experience, training 30B models will require around 2 weeks so it is expected to be released before Jun.
65B model currently cannot be fitted into single A100 so we need some tricks, for example, model parallel, but there is no efficient implementation of LLaMA. So we cannot determine a exact date.
Thanks.
from pandallm.
Related Issues (20)
- 感觉模型比原始的LlaMa模型大了很多,是什么原因 HOT 1
- 推理超参数 HOT 1
- 预训练和微调阶段使用的是全量微调还是lora? HOT 1
- 词表没扩充过是吗?还是32,000吧? HOT 1
- linly预训练阶段和panda类似,指令微调阶段使用了500万条,为什么效果比panda差? HOT 1
- 请问有使用的examples吗 HOT 1
- 权重转换脚本不支持来自 Facebook 的原版权重 HOT 2
- 请问如何在COIG数据集上继续训练 HOT 8
- 如何将COIG数据集处理为instruction tuning的格式 HOT 1
- 单卡A100训练7B模型OOM HOT 12
- 商用7b模型,直接进行推理说话毫无逻辑,难道不是开箱即用的嘛? HOT 4
- 能否提供一下模型量化的方法?其他在mac上进行推理测试 HOT 1
- 如果要重新训练panda,请问数据集应该以什么结构存储呢 HOT 5
- 由于训练数据的预处理存在不确定性,请问可以进一步分享处理代码吗 HOT 2
- 关于推理的问题 HOT 5
- RuntimeError: The size of tensor a (32000) must match the size of tensor b (32001) at non-singleton dimension 0 HOT 2
- Removed.
- Panda-LLaMA2-13B的模型链接不对吧? HOT 3
- 在线体验需要访问密码?
- 在线体验 pandallm.ai/,该链接无法打开
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandallm.