Code Monkey home page Code Monkey logo

pytorch_bert_intent_classification_and_slot_filling's Introduction

pytorch_bert_intent_classification_and_slot_filling

基于pytorch的中文意图识别和槽位填充

说明

基本思路就是:分类+序列标注(命名实体识别)同时训练。
使用的预训练模型:hugging face上的chinese-bert-wwm-ext
如果要重新训练,将config.py里面的load_model设置为False
依赖:

pytorch==1.6+
transformers==4.5.0

运行指令:

python main.py

可在config.py里面修改相关的参数,训练、验证、测试、还有预测。

结果

意图识别accuracy:0.9767441860465116
precision:0.9767441860465116
recall:0.9767441860465116
f1:0.9767441860465116
              precision    recall  f1-score   support

           0       1.00      0.94      0.97        16
           2       1.00      1.00      1.00         1
           3       1.00      1.00      1.00         4
           4       1.00      1.00      1.00        16
           5       0.00      0.00      0.00         1
           6       1.00      1.00      1.00        22
           7       0.84      0.89      0.86        18
           8       0.98      0.95      0.96        57
           9       1.00      1.00      1.00         2
          10       0.00      0.00      0.00         0
          11       0.00      0.00      0.00         1
          12       0.98      0.99      0.99       327
          13       1.00      1.00      1.00         1
          14       1.00      1.00      1.00         3
          15       1.00      1.00      1.00         1
          17       1.00      1.00      1.00         4
          18       1.00      0.80      0.89         5
          19       1.00      1.00      1.00        14
          21       0.00      0.00      0.00         1
          22       1.00      1.00      1.00        13
          23       1.00      1.00      1.00         9

    accuracy                           0.98       516
   macro avg       0.80      0.79      0.79       516
weighted avg       0.97      0.98      0.97       516

槽位填充accuracy:0.9366942909760589
precision:0.8052708638360175
recall:0.8461538461538461
f1:0.8252063015753938
                   precision    recall  f1-score   support

             Dest       1.00      1.00      1.00         7
              Src       1.00      0.86      0.92         7
             area       1.00      0.25      0.40         4
           artist       0.89      1.00      0.94         8
       artistRole       1.00      1.00      1.00         2
           author       1.00      1.00      1.00        13
         category       0.73      0.90      0.81        42
             code       0.71      0.83      0.77         6
          content       0.89      0.94      0.91        17
    datetime_date       0.72      0.95      0.82        19
    datetime_time       0.58      0.64      0.61        11
         dishName       0.84      0.88      0.86        74
        dishNamet       0.00      0.00      0.00         1
          dynasty       1.00      1.00      1.00        11
      endLoc_area       0.00      0.00      0.00         2
      endLoc_city       0.96      1.00      0.98        43
       endLoc_poi       0.62      0.73      0.67        11
  endLoc_province       0.00      0.00      0.00         1
          episode       1.00      1.00      1.00         1
             film       0.00      0.00      0.00         1
       ingredient       0.53      0.62      0.57        16
          keyword       0.88      0.88      0.88        25
    location_area       0.00      0.00      0.00         2
    location_city       0.40      1.00      0.57         4
     location_poi       0.36      0.57      0.44         7
location_province       0.00      0.00      0.00         3
             name       0.80      0.88      0.84       182
       popularity       0.00      0.00      0.00         5
       queryField       1.00      1.00      1.00         2
     questionWord       0.00      0.00      0.00         1
         receiver       1.00      1.00      1.00         4
         relIssue       0.00      0.00      0.00         1
       scoreDescr       0.00      0.00      0.00         1
             song       0.86      0.80      0.83        15
   startDate_date       0.93      0.93      0.93        15
   startDate_time       0.00      0.00      0.00         1
    startLoc_area       0.00      0.00      0.00         1
    startLoc_city       0.95      0.97      0.96        38
     startLoc_poi       0.00      0.00      0.00         1
         subfocus       0.00      0.00      0.00         1
              tag       0.40      0.40      0.40         5
           target       1.00      1.00      1.00        12
     teleOperator       0.00      0.00      0.00         1
          theatre       0.50      0.50      0.50         2
        timeDescr       0.00      0.00      0.00         2
        tvchannel       0.74      0.81      0.77        21
        yesterday       0.00      0.00      0.00         1

        micro avg       0.81      0.85      0.83       650
        macro avg       0.52      0.54      0.52       650
     weighted avg       0.79      0.85      0.81       650

=================================
打开相机这
意图LAUNCH
槽位: [('name', '相', 2, 2)]
=================================
=================================
国际象棋开局
意图QUERY
槽位: [('name', '国际象棋', 0, 3)]
=================================
=================================
打开淘宝购物
意图LAUNCH
槽位: [('name', '淘宝', 2, 3)]
=================================
=================================
搜狗
意图LAUNCH
槽位: []
=================================
=================================
打开uc浏览器
意图LAUNCH
槽位: [('name', 'uc浏', 2, 4)]
=================================
=================================
帮我打开人人
意图LAUNCH
槽位: []
=================================
=================================
打开酷狗并随机播放
意图LAUNCH
槽位: [('name', '酷狗', 2, 3)]
=================================
=================================
赶集
意图LAUNCH
槽位: []
=================================
=================================
从合肥到上海可以到哪坐车意图QUERY
槽位: [('Src', '合肥', 1, 2), ('Dest', '上海', 4, 5)]
=================================
=================================
从台州到金华的汽车意图QUERY
槽位: [('Src', '台州', 1, 2), ('Dest', '金华', 4, 5)]
=================================
=================================
从西安到石嘴山的汽车票意图QUERY
槽位: [('Src', '西安', 1, 2), ('Dest', '石嘴山', 4, 6)]
=================================

补充

上述实验只是基于自己的思路所做的,有些同学需要了解相关知识,这里附上:
意图识别和槽位填充综述
利用bert进行意图识别和槽位填充

pytorch_bert_intent_classification_and_slot_filling's People

Contributors

taishan1994 avatar wuhaitao93 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

pytorch_bert_intent_classification_and_slot_filling's Issues

数据集的问题

如果数据集存在角色的话,比如医生患者对话中,识别意图时应该怎么构建数据集呢?是要把角色加到text的前面吗
形如:
{
"sentence_id": "1",
"speaker": "医生",
"sentence": "什么时间开始发烧的?",
"dialogue_act": "Request-Symptom"
},
{
"sentence_id": "2",
"speaker": "患者",
"sentence": "上午",
"dialogue_act": "Inform-Symptom"
},
{
"sentence_id": "3",
"speaker": "医生",
"sentence": "一共烧了几次?",
"dialogue_act": "Request-Symptom"
},
{
"sentence_id": "4",
"speaker": "患者",
"sentence": "中午吃了药一直就烧呢,还有点拉肚子",
"dialogue_act": "Inform-Symptom"
},

关于数据集

您好,我想问一下这个模型的数据集时公用数据集吗? 还是自建数据集

关于推理置信度问题

首先感谢您的工程。我有一个问题,我在做实体识别时,如果故意输入完全不应该识别到实体的文字(如输入xxxxkkdkdd),此时还是会识别到某个实体,这样就会导致无法信任推理结果。因此是否有能引入实体识别置信度,根据置信度来决定此次识别的准确度

作者你好,我在尝试运行你的程序时出现了以下错误,请问应该如何解决,太头疼了,在网上找了很多方法都没用,我使用的是transformers==4.5.0,tokenizers==0.10.3

OSError: Can't load tokenizer for '../../model_hub/chinese-bert-wwm-ext/'. Make sure that:

  • '../../model_hub/chinese-bert-wwm-ext/' is a correct model identifier listed on 'https://huggingface.co/models'

  • or '../../model_hub/chinese-bert-wwm-ext/' is the correct path to a directory containing relevant tokenizer files

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.