zjunlp / relphormer Goto Github PK
View Code? Open in Web Editor NEW[Neurocomputing 2023] Relational Graph Transformer for Knowledge Graph Representation
License: MIT License
[Neurocomputing 2023] Relational Graph Transformer for Knowledge Graph Representation
License: MIT License
I have a question about examples.
example file is basically made up of dictionary and there is a key 'en_id'. I thought the 'en_id' corresponds with 'en'. The relation id and entities are matched, on the other hand the head or tail entities and their id are not matched in example file. Also i don't understand that the length of the fb15k-237 based on the entity2id file is 14541. But i added a line "print(num_entities)" on the processor.py file to check the length of the fb15k-237 is 14951. Could you explain?
我在问答部分的环境配置中遇到了一些问题,请问是否有关于问答任务环境配置的具体内容,尤其是关于kge的版本及所需要的相关内容的版本
Hello, I am interested in your paper. I would like to ask you about the approximate time for this code to run in each task. How much GPU memory is consumed?
Hello,
Do you have a conda environment file for this project to share? I am having a lot of problems trying to resolve conflicts and many sorts of errors due to that.
It'd be great if you can share that!
Thanks in advance,
Hi, another error [Errno 2] No such file or directory: 'dataset/FB15k-237/masked_head_neighbor.txt' occurred after I solved the dependency issue.Actually, there is no such file in dataset.So, I wonder if the project you upload is complete and can be run, if yes, could you tell about how to run it.Thanks!
在运行fb15k.sh文件时出现以下问题,请问这个问题是因为什么呀?该如何解决呜呜呜
问题如下:
Traceback (most recent call last):
File "main.py", line 149, in
main()
File "main.py", line 77, in main
parser = _setup_parser()
File "main.py", line 54, in _setup_parser
model_class = _import_class(f"models.{temp_args.model_class}") # 通过 _import_class 函数,从 'models' 模块导入指定类,类名由 temp_args.model_class 提供
File "main.py", line 24, in import_class
class = getattr(module, class_name)
AttributeError: module 'models' has no attribute 'Relphormer'
Hello.. I was trying to run your repository for UMLS.
I am facing few issues:
bash scripts/pretrain_umls.sh
is not saming the best model. The variable path
desn't have the location to save.Can you please help me with the it and let me know the best way to run your repo for UMLS. My goal is to try it for link prediction for UMLS. Thank you for your repo.
你好,最近在运行您的代码,Epoch0的features后一半数据是没有lables的,所以Epoch 0 进行到50%的时候,出现KeyError: 'label',查看了代码后发现是data_module.py的DataCollatorForSeq2Seq类的__init__方法中features处理问题,当lables不存在的时候产生错误,我在运行KNN-KG的时候遇见了同样的问题,请问有什么解决方案吗,错误提示如下:
File "C:\Users\admin\Desktop\code\Relphormer-main1\Relphormer-main\pretrain\data\data_module.py", line 88, in
label = [feature.pop("label") for feature in features]
KeyError: 'label'
Epoch 0: 50%|█████ | 10280/20558 [41:37<41:37, 4.12it/s, loss=6.37, v_num=3, hits10=0.000, hits1=0.000]
您好,针对之前提出的问题,我仍有一些疑问,首先,对您的文章中首先第一步应该是进行初始化向量,是采用BERT模型进行初始化向量,那具体是怎么获得实体和关系的embedding呢,是将之前的文本描述内容输出BERT得到向量就得到了实体和关系的embedding吗,那为什么一定要使用这个实体描述呢
其次,针对您的模型中, 在上下文子图采样环节,最终以序列的形式输入模型中,在图示中我注意到这个序列是一个一维的,那如果一个实体对应不同的关系,一维的这个序列不久只能表示一条线上的吗,另外这一部分还提到了A,如果我没有理解错误的话,您这个A相当于是一个实体-关系矩阵,反应是否有关系,但是这部分内容是怎样使用的呢,序列输入Transformer,矩阵是用在结构增强的部分吗,其三,在结构增强部分中出现的偏转,具体是指什么呢,为什么要有偏转,又有何意义,出现的fstructure是指什么,希望您能详细讲述一下这个偏转具体是什么含义,我没看懂呜呜呜,不好意思问题有些多,是最近在整理论文的时候发现还有些地方没看懂抱歉,希望收到您的详细回答呜呜呜
as the title
Hi, thanks for sharing the source code! Could you please provide a requirement.txt about library and their versions .I had a big trouble when running the project for the dependency issues. Thank a lot.
Hi! The paper: "Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queries" was published as a KDD'22 paper, which proposes 'Kgtransformer'.
It seems that Triple2seq implemented in your code is similar as the mask mechanism in 'Kgtransformer'. For Link Prediction, Sampling neighbors of center/query entity, then mask the query and recover, delete the nodes about origin query triplet information to avoid leakage.
both follow the pre-training LM(BERT) for semantic text, then fine-tuning workflow.
The loss targets for Relphormer and Kgtransformer both include MKM loss and contextual loss(similar formula).
I am looking forward to your help to explain what are the main differences/advances between Relphormer and Kgtransformer?
Thanks a lot for your answering.
您好,最近在阅读您的论文,注意到论文中关于hitter模型在WN18RR数据集上的MRR指标为48.5,这不同于 HittER原文中的50.3,请问是什么原因呢,是进行了重新的实验吗?
Hello!
Can you please point out where in the code the process of triple2seq which is supposed to happen during each epoch, happening? I searched the repo, but I can't seem to find it.
Thanks a lot.
I love how Relphormer utilizes text in addition to graph information. I would like to train a model using Relphormer that reproduces your results for the recommendation task.
However, I am hitting an error.
Here is a notebook that reproduces the problem: https://colab.research.google.com/drive/1PHGALZo6AkimU5jEq0I__lnoJaWxUhgg?usp=sharing
(Run in a T4 GPU runtime.)
It fails at this step:
trainer.fit(lit_model, datamodule=data)
with the following stake trace:
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
[<ipython-input-7-aea1f0bf8188>](https://localhost:8080/#) in <cell line: 14>()
12 )
13
---> 14 trainer.fit(lit_model, datamodule=data)
15
16 # Explore lit_model and its hidden states or proceed with further actions as needed
10 frames
[/content/Relphormer/data/processor.py](https://localhost:8080/#) in _create_examples(self, lines, set_type, data_dir, args)
435 '''
436 if not args.pretrain:
--> 437 with open(os.path.join(data_dir, "masked_head_neighbor.txt"), 'r') as file:
438 masked_head_neighbor = json.load(file)
439 with open(os.path.join(data_dir, "masked_tail_neighbor.txt"), 'r') as file:
FileNotFoundError: [Errno 2] No such file or directory: '/content/Relphormer/dataset/fb15k-237/masked_head_neighbor.txt'
I dug into where this masked_head_neighbor.txt file is loaded and written. This is where I am confused.
if not args.pretrain:
with open(os.path.join(data_dir, "masked_head_neighbor.txt"), 'r') as file:
masked_head_neighbor = json.load(file)
with open(os.path.join(data_dir, "masked_tail_neighbor.txt"), 'r') as file:
masked_tail_neighbor = json.load(file)
print(f'\n \t Not pre-training stage, masked subgraphs loaded.')
else:
masked_head_neighbor = []
masked_tail_neighbor = []
However, even if I set args.pretrain = None
, it still appears to be trying to read the "masked_head_neighbor.txt"
file.
So, I have two questions:
masked_head_neighbor.txt
file, how do I create it?Thanks for all your great work on this project. I'm really looking forward to getting it working.
您好,在您的数据集中存在text long部分,请问在链接预测任务中该文件是必要的吗?如果必要该文件有何作用,同时我注意到关于KG BERT的数据集中均存在该文件,是否用到BERT均需要该文件?那该文件有何要求吗,是否text中只需要包含实体即可?期待收到您的回答
In the create_neighbor.py, the paper generated the graph structure sequence using triples from the train, valid, and test datasets. I worried this might cause test dataset leakage in the training process. Is it more reasonable to only use the training dataset in the training process?
And another question: in your comparison experiment, you cited the HittER model. However, when I read the codes, the HittER model only used the graph structure nodes from the training dataset. So, is the comparison fair with the HittER model in your experiment? Not only HittER, in another work's code(IMF: Interactive Multimodal Fusion Model for Link Prediction) using graph structure for KG, also only uses the training dataset.
Looking forward to your reply. Thx!
Hi, i'm having a problem while training fb15k237 using default parameters. Any suggestions? thanks
Can't pickle local object 'get_linear_schedule_with_warmup..lr_lambda'
File "/Relphormer/main.py", line 126, in main
trainer.fit(lit_model, datamodule=data)
File "/Relphormer/main.py", line 139, in
main()
AttributeError: Can't pickle local object 'get_linear_schedule_with_warmup..lr_lambda'
I'm experiencing exactly the same thing with Relphormer that I faced when trying to get HittER running in Colab and Lightning studio: microsoft/HittER#11
What's your recommendation on how we can get this working?
In pre-training of umls with default parameter, the result is as followed:
{'Test/hits1': 0.00202020202020202,
'Test/hits10': 0.06868686868686869,
'Test/hits20': 0.13198653198653199,
'Test/hits3': 0.017508417508417508,
'Test/mean_rank': 70.0895622895623,
'Test/mrr': 0.034388057056645285}
In pre-training of WN18RR with default parameter, the result is as followed:
{'Test/hits1': 3.552610835907126e-05,
'Test/hits10': 0.00027088657623791835,
'Test/hits20': 0.0005639769702002563,
'Test/hits3': 7.993374380791033e-05,
'Test/mean_rank': 20417.76962206882,
'Test/mrr': 0.00030194088791004475}
您好!请问:
I saw that in the Triple2Seq section, it is mentioned that the neighbors are triple-level, but in the preprocessing code.
Relphormer/dataset/create_neighbor.py
Line 98 in 67a2aab
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.