Code Monkey home page Code Monkey logo

Comments (6)

lixianglong1205 avatar lixianglong1205 commented on June 12, 2024 3

使用sklearn计算f1-score
欢迎指正

import os
from sklearn.metrics import f1_score, precision_score, accuracy_score, recall_score

EVAL_DIR = "eval"


def txt_result_analysis(path):
    with open(path, mode="r", encoding="utf-8") as answer_file:
        source_label = [i.rstrip() for i in answer_file if i.strip()]
    label_dict = {}
    for one in source_label:
        i, j = one.split("\t")
        label_dict[i] = j

    return label_dict


def official_f1(average="macro"):
    """使用sklearn包计算f1-score"""
    source_label_dict = txt_result_analysis("./eval/answer_keys.txt")
    proposed_label_dict = txt_result_analysis("./eval/proposed_answers.txt")
    source_label, proposed_label = [], []
    for key in source_label_dict:
        value1 = source_label_dict[key]
        value2 = proposed_label_dict[key]
        source_label.append(value1)
        proposed_label.append(value2)
    """
    micro: 计算所有的TP TN FP FN,然后计算F1-score
    f1_score_ = f1_score(source_label, proposed_label, average="micro")
    macro: 计算每一个类别的F1-score,然后对所有的F1-scre求平均值
    f1_score_ = f1_score(source_label, proposed_label, average="macro")
    """
    f1_score_ = f1_score(source_label, proposed_label, average=average)
    return f1_score_


if __name__ == "__main__":
    print("macro-averaged F1 = {}%".format(official_f1() * 100))


from r-bert.

Pvvv avatar Pvvv commented on June 12, 2024

你好:
请教一下,为什么执行了python3 main.py --do_train --do_eval之后,result.txt 文件是空的,请问这是什么原因呢?谢谢

请问您的问题现在解决了嘛

from r-bert.

hialoha avatar hialoha commented on June 12, 2024

I also have the same problem,result.txt is empty,and I haven't deal with it.

from r-bert.

Claymore715 avatar Claymore715 commented on June 12, 2024

我也有这个问题

from r-bert.

Pvvv avatar Pvvv commented on June 12, 2024

from r-bert.

wjx-git avatar wjx-git commented on June 12, 2024

除了安装Perl外,还需要将 eval/answer_keys.txt 中的标签改为你自己验证集数据的标签。还需要更改 semeval2010_task8_scorer-v1.2.pl 中的部分内容,挺麻烦的,还不如用 sklearn 计算。

from r-bert.

Related Issues (16)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.