fudandisc / disc-finllm Goto Github PK

技术报告中给出的示例模板都是英文的，但数据集都是中文的

点击Demo链接时反馈 “101.206.185.250 拒绝了我们的连接请求。”

First I appreciate your terrific work and the decision to open source it!

I tested the cases below by your online demo, and found the answers were not right.

1. 一个借贷公司向甲借款50000元，年利率为8%（复利），借款期为3年，计算3年后需要归还的金额。
2. 假设你最近买了一只股票，但是现在它的价格下跌了20％，你需要股价上涨多少百分比才能抵消这个损失？
3. 我在银行存了一笔固定存款，一年后利息是按1%的年利率计算的，如果我选择每月领取利息，那么我最终可以得到比一次性领取利息更多的金额吗？

Please take a look at the cases and see if there is anything wrong with the model's performance.

我是一个NLP的入门学生，看到您的项目，非常感兴趣。我在尝试复现该项目的时候发现项目需要的数据并未公开，请问能申请获得该数据的访问使用权限么？
非常感谢！

运行demo：
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation.utils import GenerationConfig
from peft import PeftModel, PeftConfig

model_path = "自己的模型路径"
model = AutoModelForCausalLM.from_pretrained(
model_path, torch_dtype=torch.float16, device_map="auto", trust_remote_code=True
)
model.generation_config = GenerationConfig.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(
model_path, use_fast=False, trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, lora_path
)

messages = [
{"role": "user", "content": "请解释一下什么是银行不良资产？"},
]
response = model.chat(tokenizer, messages)
print(response)

报错：
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like baichuan-inc/Baichuan-13B-Chat is not the path to a directory containing a file named modeling_baichuan.py.

请问是哪里搞错了吗？模型已经下载到本地，是需要改config.json文件的模型路径吗？

怎么在微调阶段引入新的领域知识？垂直领域的新知识似乎很难通过微调让模型学习

期待公开完整数据集^_^

demo中按照下述方式就可以回答问题。服务内部是如何根据用户的问题，决策调用哪个LORA模型呢？这个决策是基于规则还是基于模型呢？ 对这个问题不清楚，希望详细解答一下，多谢~

messages = [
    {"role": "user", "content": "xxxx 问题？"},
]
response = model.chat(tokenizer, messages)

请问计划什么时候公开金融SFT数据集～

RT

我使用 FinGPTV3 在中文上进行使用全是乱码，可以提供下代码吗？
感谢

此项目所用的数据总量是496.000KB吗

下载的参数效果和网站demo差别非常大
自己部署模型的结果：

网站demo结果

Traceback (most recent call last):
File "/data/FinAi_Mapping_Knowledge/yangcehao/download.py", line 30, in
model = AutoModelForCausalLM.from_pretrained("Go4miii/DISC-FinLLM", device_map="auto", trust_remote_code=True)
File "/data/FinAi_Mapping_Knowledge/anaconda3/envs/sjj_env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 557, in from_pretrained
cls.register(config.class, model_class, exist_ok=True)
File "/data/FinAi_Mapping_Knowledge/anaconda3/envs/sjj_env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 583, in register
raise ValueError(
ValueError: The model class you are passing has a config_class attribute that is not consistent with the config class you passed (model has <class 'transformers_modules.baichuan-inc.Baichuan-13B-Chat.e580bc803f3f4f6b42ddccd0730739c057c7b54c.configuration_baichuan.BaichuanConfig'> and you passed <class 'transformers_modules.Go4miii.DISC-FinLLM.8e235cb9e81f952702b0b90c5f3292a1089b0dba.configuration_baichuan.BaichuanConfig'>. Fix one of those so they match!

直接采用huggingface的代码读入的。感谢解答

如题，在huggingface上看到train_loss这个图片，这个跑到1000步突然就降了很多是为啥呢？

我直接尝试从huggingface上拉取报了如上错误

尝试autoeval , 评估chatglm2-6b,baichuan-7b 相同的错误如下：
1.chatglm2-6b:
python3 autoeval.py --model chatglm2-6b --lora_path ../../models/chatglm2-6b --eval_data all --device cuda:0
cuda:0
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:09<00:00, 1.31s/it]
Traceback (most recent call last):
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/peft/config.py", line 184, in _get_peft_type
config_file = hf_hub_download(
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
validate_repo_id(arg_value)
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '../../models/chatglm2-6b'. Use repo_type argument if needed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/wts/DISC-FinLLM/eval/evaluator/autoeval.py", line 45, in
llm = model_lists.get(model_name)(device, lora_path)
File "/home/wts/DISC-FinLLM/eval/evaluator/finllm.py", line 33, in init
self.model = PeftModel.from_pretrained(self.model, peft_model_id)
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/peft/peft_model.py", line 324, in from_pretrained
PeftConfig._get_peft_type(
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/peft/config.py", line 190, in _get_peft_type
raise ValueError(f"Can't find '{CONFIG_NAME}' at '{model_id}'")
ValueError: Can't find 'adapter_config.json' at '../../models/chatglm2-6b'
2.baichuan-7b
python3 autoeval.py --model baichuan-7b --lora_path ../../models/baichuan-7b --eval_data all --device cuda:0
cuda:0
Traceback (most recent call last):
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/peft/config.py", line 184, in _get_peft_type
config_file = hf_hub_download(
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
validate_repo_id(arg_value)
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '../../models/baichuan-7b'. Use repo_type argument if needed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/wts/DISC-FinLLM/eval/evaluator/autoeval.py", line 45, in
llm = model_lists.get(model_name)(device, lora_path)
File "/home/wts/DISC-FinLLM/eval/evaluator/finllm.py", line 136, in init
self.model = PeftModel.from_pretrained(self.model, peft_model_id)
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/peft/peft_model.py", line 324, in from_pretrained
PeftConfig._get_peft_type(
File "/home/wts/anaconda3/envs/DISC-FinLLM/lib/python3.10/site-packages/peft/config.py", line 190, in _get_peft_type
raise ValueError(f"Can't find '{CONFIG_NAME}' at '{model_id}'")
ValueError: Can't find 'adapter_config.json' at '../../models/baichuan-7b'

I was very inspired by this project.
Is it possible to change the corresponding LoRa weights in different financial scenarios, and what are the advantages of setting multiple LoRa in this way compared with a single one?

Since there are 4 experts adaptors using lora SFT in the paper, the next question is why not try MOE like Mixtral-8X7B?

请问，对于Calculation(expression)，是进行了两次推理吗？还是在推理过程中中断，调用工具，在拼接回来？

According to the readme of Baichuan-13B github, online quantized need load the whole fp16 LLM into Memory, which cost 26GB memory, but consider OS, Desktop and other necessary program also need memory. There should be at least 64GB Memory for online quantize.

Baichuan-13B provided a pre-8bit-quantized model on huggingface, which can directly loaded into GPU such as 3090 (24GB). However, FinLLM have no such pre-8bit-quantized model.

As a poor student, I will be very appreciate if you can provide a pre-8bit-quantized model.

Waiting for reply,
sincerely

As the title, do you have detailed date when to opensource the sft data?