akeepers / lear Goto Github PK

Hi~in your paper, you show us a picture which indicates the non-effective attention mechanism of previous models:

I am wondering whether you can show us a picture like the above picture to demonstrate the effectiveness of LEAR's attention mechanism? Thank you~

could you release the hyper parameters for the few shot mode or could you tell the ways you instruction

Hi～，最近做看 BERT-MRC 和 你的 LAER，有点好奇：
1）论文上的效果是取最高的分数吗，还是做几次实验，然后取平均值。
2）seed 设置为 42 效果会好很多吗，对比其他 seed 来说

I want to ask where I can find this file——first_label_file

就是processed下面，我看ner的已经有了，方便上传一下吗，或者给个sample？麻烦你了

您好！
在看了您的论文之后，想试着在自己的数据集上看看效果，参数设置涉及到label的那块有些模糊不清，不知道您方便回答一下吗？
如果可以的话，给个具体的实例最好，麻烦啦！
就是以zh_msra为例，能否给一个使用您提出的模型时，配置参数该如何设置呢

您好！
在看了您的论文之后，想试着在自己的数据集上看看效果，参数设置涉及到label的那块有些模糊不清，不知道您方便知道一下吗？
如果可以的话，给个具体的实例最好，麻烦啦！
就是以zh_msra为例，能否给一个使用您提出的模型时，配置参数该如何设置呢

可以参考您进行ner时候传的参数吗
python run_ner.py --task_type sequence_classification --task_save_name FLAT_NER --data_dir ./data/data/ner --data_name zh_msra --model_name bert_ner --model_name_or_path bert-base-cased --output_dir ./model --do_lower_case False --result_dir ./model/result --first_label_file ./data/data/ner/zh_msra/processed/label_map.json --overwrite_output_dir TRUE --train_set ./data/data/ner/zh_msra/processed/train.json --dev_set ./data/data/ner/zh_msra/processed/dev.json --test_set ./data/data/ner/zh_msra/processed/test.json

Hello, I find there are many variables which have a prefix 'first'. For example, 'first_label'. Can you explain the meaning of 'first' here?

such as ontonote5, conll03, mitmovie etc.
plz
and transformers version 3.9.2 i could not find.the official reach the 3.5.1 and then 4.0.0

@Akeepers
hi，我这边在尝试使用你的代码，我发现模型这部分我不是太get到啥意思？

上图中的四个模型分别是指？期待回复

参数有点混乱，能否给一下run_ner的参数示例？

Hi, LEAR is an excellent work. I have encountered some problems with the loss when running the code. The loss is oscillating without convergence. I wonder if you have any ideas about it. Thank you.

Regards,
Chuck

老师你好，最近再复现这篇论文的代码，数据集按照老版本的HMEAE进行数据切分后还是不匹配，请问您那里有老版本的HMEAE备份吗，有的话能发给我一份吗

Hi~

1. 想问你在多卡条件下跑过事件检测的代码嘛？我用4张卡跑，报了下面的错误：

Training:   0%|                                                                                                                                                               | 0/1159 [00:22<?, ?it/s]
Traceback (most recent call last):
  File "run_trigger_extraction.py", line 405, in <module>
    main()
  File "run_trigger_extraction.py", line 378, in main
    train(args, model, processor)
  File "run_trigger_extraction.py", line 243, in train
    pred_sub_heads, pred_sub_tails = model(data, add_label_info=add_label_info)
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
    output.reraise()
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/yc21/project/LEAR/models/model_event.py", line 635, in forward
    fused_results = self.label_fusing_layer(
  File "/home/yc21/software/anaconda3/envs/lear/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/yc21/project/LEAR/utils/model_utils.py", line 320, in forward
    return self.get_fused_feature_with_attn(token_embs, label_embs, input_mask, label_input_mask, return_scores=return_scores)
  File "/home/yc21/project/LEAR/utils/model_utils.py", line 504, in get_fused_feature_with_attn
    scores = torch.matmul(token_feature_fc, label_feature_t).view(
RuntimeError: shape '[4, 48, 33, -1]' is invalid for input of size 160512

2. 我用一张卡跑，设置batchsize=2, gradient_accumulation_step=16, 能够跑通，但是得到的结果是train上的loss收敛到了6，dev的f1一直是0。我不知道这只是我的个人问题，还是有其他人也存在这个问题。有童鞋跑出相应的结果了么？

谢谢大家~

作者你好，
我想问一下下面这个参数的问题！

这个里边的参数和--task_layer_lr这个参数是一个么，为什么我在命令行输入的--task_layer_lr参数，在上边图片标注的跟我输入的不一样。

同学你好，

我根据论文，使用以下参数运行
CUDA_VISIBLE_DEVICES=0 python run_ner.py --task_type sequence_classification --task_save_name SERS --data_dir ./data/ner --data_name zh_onto4 --model_name SERS --model_name_or_path ../bert_models/chinese-roberta-wwm-ext-large --output_dir ./zh_onto4_models/bert_large --do_lower_case False --result_dir ./zh_onto4_models/results --first_label_file ./data/ner/zh_onto4/processed/label_map.json --overwrite_output_dir True --train_set ./data/ner/zh_onto4/processed/train.json --dev_set ./data/ner/zh_onto4/processed/dev.json --test_set ./data/ner/zh_onto4/processed/test.json --is_chinese True --max_seq_length 128 --per_gpu_train_batch_size 32 --gradient_accumulation_steps 1 --num_train_epochs 5 --learning_rate 8e-6 --task_layer_lr 10 --label_str_file ./data/ner/zh_onto4/processed/label_annotation.txt --span_decode_strategy v5

在Ontonote4 Chinese上得到 f1: 82.32，与论文上报告的82.95有一些差距，请问这个数值是合理的吗，还是我有运行参数设置不对？

期待您的回复！
叶德铭