bert-int's Issues
result = self.forward(*input, **kwargs) File "/data/my_project/nlp_text/nlp_models/bert-int-master/basic_bert_unit/Basic_Bert_Unit_model.py", line 20, in forward cls_vec = sequence_output[:,0]
n Batch_TrainData_Generator, train ill num: 4500
In Batch_TrainData_Generator, ent_ids1 num: 15000
In Batch_TrainData_Generator, ent_ids2 num: 15000
start training...
+++++++++++
Epoch: 0
+++++++++++
train ent1s num: 4500 train ent2s num: 4500 for_Candidate_ent1s num: 15000 for_candidate_ent2s num: 15000
Traceback (most recent call last):
File "main.py", line 75, in <module>
main()
File "main.py", line 70, in main
train(Model,Criterion,Optimizer,Train_gene,train_ill,test_ill,ent2data)
File "/data/my_project/nlp_text/nlp_models/bert-int-master/basic_bert_unit/train_func.py", line 115, in train
for_candidate_ent2s,entid2data,Train_gene.index2entity)
File "/data/my_project/nlp_text/nlp_models/bert-int-master/basic_bert_unit/train_func.py", line 44, in generate_candidate_dict
temp_emb = entlist2emb(Model,train_ent1s[i:i+batch_size],entid2data,CUDA_NUM).cpu().tolist()
File "/data/my_project/nlp_text/nlp_models/bert-int-master/basic_bert_unit/train_func.py", line 26, in entlist2emb
batch_emb = Model(batch_token_ids,batch_mask_ids)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/data/my_project/nlp_text/nlp_models/bert-int-master/basic_bert_unit/Basic_Bert_Unit_model.py", line 20, in forward
cls_vec = sequence_output[:,0]
TypeError: string indices must be integers
运行basic_bert_unit时遇到的问题
start training...
+++++++++++
Epoch: 0
+++++++++++
train ent1s num: 4500 train ent2s num: 4500 for_Candidate_ent1s num: 15000 for_candidate_ent2s num: 15000
D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn_reduction.py:46: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
Traceback (most recent call last):
File "D:/实验室相关的内容/bert-int/bert-int-master/basic_bert_unit/main.py", line 75, in
main()
File "D:/实验室相关的内容/bert-int/bert-int-master/basic_bert_unit/main.py", line 70, in main
train(Model,Criterion,Optimizer,Train_gene,train_ill,test_ill,ent2data)
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\train_func.py", line 115, in train
for_candidate_ent2s,entid2data,Train_gene.index2entity)
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\train_func.py", line 44, in generate_candidate_dict
temp_emb = entlist2emb(Model,train_ent1s[i:i+batch_size],entid2data,CUDA_NUM).cpu().tolist()
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\train_func.py", line 26, in entlist2emb
batch_emb = Model(batch_token_ids,batch_mask_ids)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\实验室相关的内容\bert-int\bert-int-master\basic_bert_unit\Basic_Bert_Unit_model.py", line 20, in forward
x = self.bert_model(input_ids = batch_word_list,attention_mask = attention_mask)#token_type_ids =token_type_ids
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 627, in forward
head_mask=head_mask)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 348, in forward
layer_outputs = layer_module(hidden_states, attention_mask, head_mask[i])
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 326, in forward
attention_outputs = self.attention(hidden_states, attention_mask, head_mask)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 283, in forward
self_outputs = self.self(input_tensor, attention_mask, head_mask)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda\envs\bert-int3\lib\site-packages\transformers\modeling_bert.py", line 211, in forward
attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
RuntimeError: cublas runtime error : the GPU program failed to execute at C:/w/1/s/windows/pytorch/aten/src/THC/THCBlas.cu:450
Neighboring relation mask matrix的计算问题
您好,我阅读代码后没有发现计算Neighboring relation mask matrix
的相关内容,请问这个矩阵是在哪计算的呢?
Basic_Bert_Unit_model 训练问题
作者您好,在Basic_Bert_Unit_model在训练时存在显存不够的情况(12G),想问您一下您的训练环境是什么样的,谢谢!
Results aren't matching the paper results
Hi,
I trained the model from scratch keeping all parameters same. I am getting hits@1 to be ~0.87. What kind of changes would be required to get the results mentioned in your paper.
What are hardware requirements for training bert-int from scratch with/without cuda?
I was trying to train bert-int from scratch and ran out of gpu memory. How much do I need min to work with it.
关于DWY100K数据集
尊敬的作者,您好:
您的这项工作是非常伟大的,我学到了很多。
其中关于DWY100K数据集,请问您的输入是什么样的?它是否还有descriptions?
非常期待并感谢您的回复!
Why aggregation operation by RBF?
Hi! You're work inspired me a lot. I have a question about the feature extraction in the chapter "Neighbor-view Interactions".
Would you mind tell me the reason for the usage of RBF function to generate features from maximum similarity vectors, or what inspired you to propose the aggregation method. I spent some time to try other aggregation methods. However, the accuracy was lower than yours.
Thanks for your reading. Looking forward to your reply!
Could you share data conversion scripts / rules you used?
There is some pre-processing over the DBP15K dataset, could you share the scripts you used to make the data format you have including how you extracted descriptions.
Attribute-view Interactions的问题
您好,请问在论文这一节中,attribute-value pairs应该怎么理解?attribute和value的embedding分别是如何获取的?
Can you tell me about the description of DWY100k?
Can you tell me about the description of DWY100k?
提供的预训练模型无法正常使用。
Basic_Bert_Unit_model train-test相似度以及epoch-loss
作者您好,有个问题没明白想请教一下,在basic-bert中训练时用的F.pairwise_distance(pos_emb1,pos_emb2,p=1,keepdim=True)#L1 distance,而test中用的是cos_sim_mat_generate(emb1,emb2,batch_size,cuda_num=CUDA_NUM),为什么没用相同的相似度进行衡量呢?还有一个问题是跑了5个epoch后,test的hit @ 1: 0.09190 hit @10 : 0.10610 hit @ 25: 0.10990 hit @ 50: 0.11743 指标一直是0.0x,而且epochloss也一直是震荡状态(从3千到1万2千之间),请问作者有没有遇到过这样的问题,还是我哪个地方搞错了,盼望回复,谢谢~
Upload pre-trained BERT unit models
Hi,
Thanks for the great work in KG alignment. As the limited memory of my GPUs and the speed of CPUs, could you upload the pre-trained BERT unit models?
Cheers,
Yu
Description data是如何提取的
您好,我想问下一下2016-10-des_dict是如何提取的?我看了https://wiki.dbpedia.org/downloads-2016-10也没找到是如何提取的?
Why aren't names considered in the descriptions as part of Mbert input?
If I keep the description size to 0 and only consider entity names, ther's still a good performance. Why arent they part of descriptions by default, maybe concatenating name with description would be beneficial?
运行interaction_model.py遇到问题,没有"../data/dbp15k/ja_en/attribute_similarity_feature.pkl"文件
(bert-int) root@61febcfea76c:/workspace/bert-int/interaction_model# python interaction_model.py
----------------interaction model--------------------
GPU num 0
train_ill num: 4500 /test_ill num:10500 / train_ill & test_ill num: 0
Traceback (most recent call last):
File "interaction_model.py", line 49, in
main()
File "interaction_model.py", line 22, in main
att_features = pickle.load(open(ATTRIBUTEVIEW_SIMILARITY_FEATURE_PATH,'rb')) #attribute-view interaction similarity feature
FileNotFoundError: [Errno 2] No such file or directory: '../data/dbp15k/ja_en/attribute_similarity_feature.pkl'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.