Comments (7)
Epoch 1/50
2000/2000 [==============================] - 193s 97ms/step - loss: 0.0078
Evaluation: 100%|█████████████████████████████████████████| 1159/1159 [00:32<00:00, 35.58it/s]
[val] f1: 0.90232, p: 0.92398 r: 0.88164
这是我本地跑的,你是用同样的数据集吗?也没有改动代码?
from bert4torch.
如果是使用你自己的数据集,不传入steps_per_epoch或者设置为None,表示跑一个完整的epoch,然后检查下数据处理部分代码,看看结果咋样?
from bert4torch.
我用的人民日报数据集,也没动过代码,在colab上跑的
from bert4torch.
我在colab上是可以跑起来的,如下截图,请检查你是否成功加载了预训练模型,或者先在本机调试下看看
from bert4torch.
我感觉是BERT版本原因,作者大大使用的是哪个版本的BERT,我换了huggingface里的bert-base-chinese版本第一轮的还是0,第二轮以后开始慢慢提升,但是收敛速度很慢,到17轮F1到了93多
from bert4torch.
我感觉是BERT版本原因,作者大大使用的是哪个版本的BERT,我换了huggingface里的bert-base-chinese版本第一轮的还是0,第二轮以后开始慢慢提升,但是收敛速度很慢,到17轮F1到了93多
您好,是的,因为你用的bert-base-chinese的权重的key和本框架没有完全对齐,导致Laynorm层的权重和偏置未成功加载,可以使用转化脚本修改一下,本项目使用到的部分数据集下载链接,直接加载的checkpoint清单
from bert4torch.
from bert4torch.
Related Issues (20)
- tokenizer_encode_config和tokenizer_decode_config问题 HOT 3
- 预训练模型加载 HOT 5
- DataLoader的num_workers设置为大于0时出错 HOT 3
- 请问怎么调整最后的输出结果的小数点位数,目前最后的测试结果是两位,在哪里可以调整一下啊? HOT 7
- 请问在用bart bert等模型生成标题时,如果修改标题的最大长度? HOT 3
- 显示每个epoch的已用时间和预估剩余时间 HOT 3
- chinese-xlnet-mid载入 HOT 1
- XLnet分类报错 HOT 4
- simbert微调问题 HOT 2
- LLM:chatGLM2推理加速 HOT 1
- bert4torch版本0.2.8升级到0.3.4问题 HOT 2
- 分类算法sentence句子编码的时候,没理解到mask处理逻辑 HOT 1
- Trainer.compile与torch2.0新加的model.compile重名 HOT 3
- bert4torch中,使用单卡多GPU训练,使用accelerate后,不能使用梯度剪裁。 HOT 3
- bert4torch/examples/llm /task_chatglm2_lora.py中的config_path = dir_path + 'bert4torch_config.json' 在哪里? HOT 3
- callback变红 HOT 3
- self.random_sample和 self.beam_search返回值tensor未统一转化numpy,导致两函数同时调用报错 HOT 2
- 加载Yi-6B进行lora微调报错 HOT 1
- 请求大佬写个NER指针网络的预测代码 HOT 4
- Text2Vec加载BAAI_bge-large-zh-v1.5进行相似度计算得到的结果有误 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bert4torch.