hanlard / transformer-based-pretrained-model-for-event-extraction Goto Github PK

View Code? Open in Web Editor NEW

156.0 2.0 25.0 25.77 MB

使用基于Transformer的预训练模型在ACE2005数据集上进行事件抽取任务

Python 100.00%

event-extraction transformer bert ace2005 muti-task

transformer-based-pretrained-model-for-event-extraction's Introduction

Transformer-based-pretrained-model-for-event-extraction

使用BERT/OpenAI-GPT2/ALBERT/XLM/Roberta/XLNet/Ctrl/DistilBert/TransfoXL等预训练语言模型在ace2005数据集上进行事件抽取任务。

代码在nlpcl-lab / bert-event-extraction框架上修改，使用transformers包和CRF模型替换了原项目的模型构建部分。

模型整体采用序列标注的方式，未使用任何辅助信息。先用crf做触发词识别，再根据触发词识别结果再用crf进行论元识别。

预训练模型选用xlm-roberta-large时，trigger-f1=0.72; argument-f1=0.45。argument提升了0.05。

说明：当前是按照一个事件类型使用一个CRF进行识别论元，这会造成一定程度的数据稀疏问题，可以在consts.py中修改，将多种事件类型合并为一个CRF识别

trigger classification

P=0.677 R=0.754 F1=0.713

argument classification

P=0.588 R=0.384 F1=0.464

trigger identification

P=0.723 R=0.805 F1=0.762

argument identification

P=0.617 R=0.403 F1=0.488

超参如下

==================== 超参 ====================

可选预训练模型：

PreTrainModel = ['Bert_large', 'Gpt', 'Gpt2', 'Ctrl', 'TransfoXL',

'Xlnet_base', 'Xlnet_large', 'XLM', 'DistilBert_base', 'DistilBert_large',

'Roberta_base', 'Roberta_large', 'XLMRoberta_base', 'XLMRoberta_large',

'ALBERT-base-v1', 'ALBERT-large-v1', 'ALBERT-xlarge-v1', 'ALBERT-xxlarge-v1',

'ALBERT-base-v2', 'ALBERT-large-v2', 'ALBERT-xlarge-v2', 'ALBERT-xxlarge-v2']

       early_stop = 5
               lr = 1e-05
               l2 = 1e-05
         n_epochs = 50
           logdir = logdir
         trainset = data/train.json
           devset = data/dev.json
          testset = data/test.json
       LOSS_alpha = 1.0        
   PreTrain_Model = XLMRoberta_large
       model_path = /Transformer-based-pretrained-model-for-event-extraction-master/save_model/latest_model.pt
       batch_size = 16

运行

在LDC网站获取ACE2005数据集,企业和学校购买后方可获取(LDC2006T06)： https://catalog.ldc.upenn.edu/byyear#2005
按照 https://github.com/nlpcl-lab/ace2005-preprocessing 的方法，将ACE2005数据处理为json格式的train/dev/test后放入\data文件夹，处理后的格式应该和\data中sample.json一致
安装依赖环境
训练评估：python DataLoadAndTrain.py --LOSS_alpha=1 --lr=1e-5 --l2=1e-5 --early_stop=5 --PreTrain_Model="XLMRoberta_large" --batch_size=16

我的邮箱：[email protected]

我的知乎主页：https://www.zhihu.com/people/zhang-han-32-13-81

transformer-based-pretrained-model-for-event-extraction's People

Contributors

Stargazers

Watchers

transformer-based-pretrained-model-for-event-extraction's Issues

触发词用CRF识别为什么不用BIO_tagging的方式？

您好，论元的特征是由text emb + trigger emb+event emb表示的
然后输入到CRF中标注
但论元的CRF没有BIO的形式，直接就是纯纯的28类标签？

Segmentation fault (core dumped)

python DataLoadAndTrain.py --LOSS_alpha=1 --lr=1e-5 --l2=1e-5 --early_stop=5 --PreTrain_Model="Gpt2" --batch_size=16
2023-03-01 10:52:17.583697: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib/:/usr/local/nvidia/lib64/
2023-03-01 10:52:17.583823: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib/:/usr/local/nvidia/lib64/
2023-03-01 10:52:17.583840: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
==================== 超参 ====================
PreTrainModel = ['Bert_large', 'Gpt', 'Gpt2', 'Ctrl', 'TransfoXL', 'Xlnet_base', 'Xlnet_large', 'XLM', 'DistilBert_base', 'DistilBert_large', 'Roberta_base', 'Roberta_large', 'XLMRoberta_base', 'XLMRoberta_large', 'ALBERT-base-v1', 'ALBERT-large-v1', 'ALBERT-xlarge-v1', 'ALBERT-xxlarge-v1', 'ALBERT-base-v2', 'ALBERT-large-v2', 'ALBERT-xlarge-v2', 'ALBERT-xxlarge-v2']
early_stop = 5
lr = 1e-05
l2 = 1e-05
n_epochs = 50
logdir = logdir
trainset = data/train.json
devset = data/dev.json
testset = data/test.json
LOSS_alpha = 1.0
telegram_bot_token =
telegram_chat_id =
PreTrain_Model = Gpt2
model_path = Transformer-based-pretrained-model-for-event-extraction-master/save_model/latest_model.pt
batch_size = 16
Segmentation fault (core dumped)
(base) root@dl-230215085122e58-pod-jupyter-575dd95b75-wct2x:~/Transformer-based-pretrained-model-for-event-extraction#
为什么会出现这个问题呢

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 3.00 GiB total capacity; 2.13 GiB already allocated; 3.55 MiB free; 2.18 GiB reserved in total by PyTorch)

触发词分类评估结果是在触发词检测正确基础上显示的吗

thank you

你好，能否共享一下数据集

你好，想复现一下实验，但无奈没有渠道获取数据集，恳请作者能共享一下，我的邮箱是[email protected],谢谢

请问我在LCD用学校邮箱注册之后，显示需要管理员同意，这需要多长时间呢

Your organization administrator(s) has been notified of your registration. When he or she accepts you as a member of the organization, your organizations' corpora will be available to you.
一直显示这样

DMCNN实验结果

您好，请问您复现的DMCNN效果如何呢

Transformer-based-pretrained-model-for-event-extraction/migration_model/train.py报错

Traceback (most recent call last):
File "/Users//NLP/Transformer-based-pretrained-model-for-event-extraction/migration_model/train.py", line 13, in
from model import Net
File "/Users//NLP/Transformer-based-pretrained-model-for-event-extraction/migration_model/model.py", line 16, in
from DataLoadAndTrain import idx2trigger, argument2idx
File "/Users//NLP/Transformer-based-pretrained-model-for-event-extraction/migration_model/DataLoadAndTrain.py", line 12, in
from model import Net
ImportError: cannot import name 'Net'
这写的也没问题啊,怎么会报错呢??

请问如何获取

你好，在做事件抽取研究，请问如何获取ACE2005数据集呢？哪里可以下载或者申请吗？感谢

请问如何获取ace2005数据集呢？

你好，请问如何获取ace2005数据集呢？哪里可以下载或者申请吗？

open domain/close domain？

请问这个事件抽取是open domain还是close domain？

论元在触发词正确的基础上做是不是有点问题

OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

请问出现在跑这个代码的时候出现如下问题要怎么解决呢
OSError: Unable to load weights from pytorch checkpoint file. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.