CamelBell (驼铃) is be a Chinese Language Tuning project based on LoRA. CamelBell is belongs to Project Luotuo(骆驼), an open sourced Chinese-LLM project created by 冷子昂 @ 商汤科技 & 陈启源 @ 华中师范大学 & 李鲁鲁 @ 商汤科技

CamelBell is NOT an official product of SenseTime

[2023-3-24] We've just released Evaluation code, tuning Chinese LLM with very few data on GLM-6B via LoRA, try here

Check our evaluation code here .

We have tuned a Chinese model based on ChatGLM-6B

The training code was baed on ChatGLM-Tuning. However, the original code of ChatGLM-Tuning is still in building and not support Chinese Tuning. We modified part of training code.

Our training code is in cleaning, if you are in hurry, check the ChatGLM-Tuning project and try to debug the part of Tokenizer and INT4/INT8 switcher stuff.

This is an inbuilding project, we plan to finish at least 3 demo LoRA models in this project

A. The model A will tuned on very few instruction (only around 80 questions, see in developer_instruction.json), the model has been released now

B. The model B, we plan to do something interesting. 李鲁鲁 plan to write a script, selecting a character in a movie/ a book/ or history, query thousands of QA data from OpenAI api. and tuning GLM into a character chat bot.

C. The model C, find some specific domain QA data.

inbuilding project

• release evaluation code
• release model A
• write data scipt for model B
• collecting data for model B
• release model B
• collecting data for model C
• release model C
• clean and release training code
• refactor GLM code into standard HuggingFace pipeline

Detailed Sponsorship and Balance see in Sponsorship_and_balance.md

Top 3 Sponsors(爸爸) until 3/24, this table in sub-repo may delay than the major Luotuo

骆驼原本是我们的一个作业项目，我们原本计划训练到1.0为止。但是社区的热情超过了我们的想象。如果您愿意赞助我们的项目，可以

扫描这个二维码

并且加这个支付宝账号，留下您的姓名

项目的资金流向将被公开，所有的资金将被用于数据的标注，训练算力的购买或者后续周边产品的发放。数据和算力的捐献也会一同总结在sponsorship的表格中。备用链接 二维码 , 支付宝账号

This was originally an exercise project for us, and we originally planned to train until version 1.0. However, the enthusiasm of the community exceeded our expectations. If you are willing to sponsor our project, you can scan this QR code and add this Alipay account, leaving your name.

All funds will be used for data annotation, purchase of training computing power, or distribution of subsequent peripheral products.

Please cite the repo if you use the data or code in this repo.

@misc{alpaca,
  author={Ziang Leng, Qiyuan Chen and Cheng Li},
  title = {Luotuo: An Instruction-following Chinese Language, LoRA tuning on LLaMA model},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/LC1332/Chinese-alpaca-lora}},
}