Comments (6)
使用sklearn计算f1-score
欢迎指正
import os
from sklearn.metrics import f1_score, precision_score, accuracy_score, recall_score
EVAL_DIR = "eval"
def txt_result_analysis(path):
with open(path, mode="r", encoding="utf-8") as answer_file:
source_label = [i.rstrip() for i in answer_file if i.strip()]
label_dict = {}
for one in source_label:
i, j = one.split("\t")
label_dict[i] = j
return label_dict
def official_f1(average="macro"):
"""使用sklearn包计算f1-score"""
source_label_dict = txt_result_analysis("./eval/answer_keys.txt")
proposed_label_dict = txt_result_analysis("./eval/proposed_answers.txt")
source_label, proposed_label = [], []
for key in source_label_dict:
value1 = source_label_dict[key]
value2 = proposed_label_dict[key]
source_label.append(value1)
proposed_label.append(value2)
"""
micro: 计算所有的TP TN FP FN,然后计算F1-score
f1_score_ = f1_score(source_label, proposed_label, average="micro")
macro: 计算每一个类别的F1-score,然后对所有的F1-scre求平均值
f1_score_ = f1_score(source_label, proposed_label, average="macro")
"""
f1_score_ = f1_score(source_label, proposed_label, average=average)
return f1_score_
if __name__ == "__main__":
print("macro-averaged F1 = {}%".format(official_f1() * 100))
from r-bert.
你好:
请教一下,为什么执行了python3 main.py --do_train --do_eval之后,result.txt 文件是空的,请问这是什么原因呢?谢谢
请问您的问题现在解决了嘛
from r-bert.
I also have the same problem,result.txt is empty,and I haven't deal with it.
from r-bert.
我也有这个问题
from r-bert.
from r-bert.
除了安装Perl外,还需要将 eval/answer_keys.txt 中的标签改为你自己验证集数据的标签。还需要更改 semeval2010_task8_scorer-v1.2.pl 中的部分内容,挺麻烦的,还不如用 sklearn 计算。
from r-bert.
Related Issues (16)
- Question about F1 results HOT 2
- The F1-Score decrease after the last update. HOT 10
- Mul-BERT 在 the SemEval 2010 Task 8 dataset 到达 90.72 (Macro-F1)方法非常简单,
- The question is that the information of e1 and e2 is obtained in predict.py, but predict.py does not seem to be used in train .
- Training on a custom dataset
- Custom dataset with different relations
- 运行predict.py出现错误:FileNotFoundError: [Errno 2] No such file or directory: './model\\training_args.bin'
- How can i train my own dataset? HOT 4
- CUDA out of memory HOT 2
- How can I retrain the model from the past trained model ? HOT 2
- Some model files might be missing... HOT 5
- Segmentation fault (core dumped) HOT 1
- how can i train the R-bert in chinese dataset? HOT 1
- 看到效果达到89.69%的关系抽取源码实现,推荐一波 https://github.com/DongPoLI/EC-BERT HOT 1
- FClayer for two entities
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from r-bert.