kosugi11037 / bert-int Goto Github PK

start training...
+++++++++++
Epoch: 0
+++++++++++
train ent1s num: 4500 train ent2s num: 4500 for_Candidate_ent1s num: 15000 for_candidate_ent2s num: 15000
D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn_reduction.py:46: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
Traceback (most recent call last):
File "D:/实验室相关的内容/bert-int/bert-int-master/basic_bert_unit/main.py", line 75, in
main()
File "D:/实验室相关的内容/bert-int/bert-int-master/basic_bert_unit/main.py", line 70, in main
train(Model,Criterion,Optimizer,Train_gene,train_ill,test_ill,ent2data)
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\train_func.py", line 115, in train
for_candidate_ent2s,entid2data,Train_gene.index2entity)
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\train_func.py", line 44, in generate_candidate_dict
temp_emb = entlist2emb(Model,train_ent1s[i:i+batch_size],entid2data,CUDA_NUM).cpu().tolist()
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\train_func.py", line 26, in entlist2emb
batch_emb = Model(batch_token_ids,batch_mask_ids)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\Basic_Bert_Unit_model.py", line 20, in forward
x = self.bert_model(input_ids = batch_word_list,attention_mask = attention_mask)#token_type_ids =token_type_ids
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 627, in forward
head_mask=head_mask)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 348, in forward
layer_outputs = layer_module(hidden_states, attention_mask, head_mask[i])
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 326, in forward
attention_outputs = self.attention(hidden_states, attention_mask, head_mask)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 283, in forward
self_outputs = self.self(input_tensor, attention_mask, head_mask)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 211, in forward
attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
RuntimeError: cublas runtime error : the GPU program failed to execute at C:/w/1/s/windows/pytorch/aten/src/THC/THCBlas.cu:450

您好，我阅读代码后没有发现计算Neighboring relation mask matrix的相关内容，请问这个矩阵是在哪计算的呢？

作者您好，在Basic_Bert_Unit_model在训练时存在显存不够的情况(12G)，想问您一下您的训练环境是什么样的,谢谢！

Hi,

I trained the model from scratch keeping all parameters same. I am getting hits@1 to be ~0.87. What kind of changes would be required to get the results mentioned in your paper.

I was trying to train bert-int from scratch and ran out of gpu memory. How much do I need min to work with it.

尊敬的作者，您好：
您的这项工作是非常伟大的，我学到了很多。
其中关于DWY100K数据集，请问您的输入是什么样的？它是否还有descriptions？
非常期待并感谢您的回复！

There is some pre-processing over the DBP15K dataset, could you share the scripts you used to make the data format you have including how you extracted descriptions.

您好，请问在论文这一节中，attribute-value pairs应该怎么理解？attribute和value的embedding分别是如何获取的？

Can you tell me about the description of DWY100k?

您好，根据您在百度网盘中提供的预训练模型无法正常运行。
在运行get_entity_embedding.py 的时候出现了如下报错：

作者您好，有个问题没明白想请教一下，在basic-bert中训练时用的F.pairwise_distance(pos_emb1,pos_emb2,p=1,keepdim=True)#L1 distance，而test中用的是cos_sim_mat_generate(emb1,emb2,batch_size,cuda_num=CUDA_NUM)，为什么没用相同的相似度进行衡量呢？还有一个问题是跑了5个epoch后，test的hit @ 1: 0.09190 hit @10 : 0.10610 hit @ 25: 0.10990 hit @ 50: 0.11743 指标一直是0.0x，而且epochloss也一直是震荡状态（从3千到1万2千之间），请问作者有没有遇到过这样的问题，还是我哪个地方搞错了，盼望回复，谢谢~

Hi,

Thanks for the great work in KG alignment. As the limited memory of my GPUs and the speed of CPUs, could you upload the pre-trained BERT unit models?

Cheers,
Yu

您好，我想问下一下2016-10-des_dict是如何提取的？我看了https://wiki.dbpedia.org/downloads-2016-10也没找到是如何提取的？

If I keep the description size to 0 and only consider entity names, ther's still a good performance. Why arent they part of descriptions by default, maybe concatenating name with description would be beneficial?

(bert-int) root@61febcfea76c:/workspace/bert-int/interaction_model# python interaction_model.py
----------------interaction model--------------------
GPU num 0
train_ill num: 4500 /test_ill num:10500 / train_ill & test_ill num: 0
Traceback (most recent call last):
File "interaction_model.py", line 49, in
main()
File "interaction_model.py", line 22, in main
att_features = pickle.load(open(ATTRIBUTEVIEW_SIMILARITY_FEATURE_PATH,'rb')) #attribute-view interaction similarity feature
FileNotFoundError: [Errno 2] No such file or directory: '../data/dbp15k/ja_en/attribute_similarity_feature.pkl'