liuweiping2020 / krlpapers Goto Github PK

KRL: knowledge representation learning. KE: knowledge embedding.

Contributed by Shulin Cao and Xu Han.

We release OpenKE, an open source toolkit for KRL/KE. This repository provides a standard KRL/KE training and testing framework. Currently, the implemented models in OpenKE include TransE, TransH, TransR, TransD, RESCAL, DistMult, ComplEx and HolE.

1. Representation Learning: A Review and New Perspectives. Yoshua Bengio, Aaron Courville, and Pascal Vincent. TPAMI 2013. paper

2. Knowledge Representation Learning: A Review. (In Chinese) Zhiyuan Liu, Maosong Sun, Yankai Lin, Ruobing Xie. 计算机研究与发展 2016. paper

3. A Review of Relational Machine Learning for Knowledge Graphs. Maximilian Nickel, Kevin Murphy, Volker Tresp, Evgeniy Gabrilovich. Proceedings of the IEEE 2016. paper

4. Knowledge Graph Embedding: A Survey of Approaches and Applications. Quan Wang, Zhendong Mao, Bin Wang, Li Guo. TKDE 2017. paper

1. RESCAL: A Three-Way Model for Collective Learning on Multi-Relational Data. Nickel Maximilian, Tresp Volker, Kriegel Hans-Peter. ICML 2011. paper code

   RESCAL is a tensor factorization approach to knowledge representation learning, which is able to perform collective learning via the latent components of the factorization.

2. SE: Learning Structured Embeddings of Knowledge Bases. Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio. AAAI 2011. paper

   SE assumes that the head and tail entities are similar in a relation-dependent subspace, where each relation is represented by two different matrices.

3. LFM: A Latent Factor Model for Highly Multi-relational Data. Rodolphe Jenatton, Nicolas L. Roux, Antoine Bordes, Guillaume R. Obozinski. NIPS 2012. paper

   LFM is based on a bilinear structure, which captures variouts orders of interaction of the data, and also shares sparse latent factors across different relations.

4. NTN: Reasoning With Neural Tensor Networks for Knowledge Base Completion. Richard Socher, Danqi Chen, Christopher D. Manning, Andrew Ng. NIPS 2013. paper

   NTN is a neural network which allows mediated interaction of entity vectors via a tensor. NTN might be the most expressive model to date, but it is not sufficiently simple and efficient to handle large-scale KGs.

5. TransE: Translating Embeddings for Modeling Multi-relational Data. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko. NIPS 2013. paper code

   TransE is the first model to introduce translation-based embedding, which interprets relations as the translations operating on entities.

6. TransH: Knowledge Graph Embedding by Translating on Hyperplanes. Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen. AAAI 2014. paper code

   To preserve the mapping propertities of 1-N/N-1/N-N relations, TransH inperprets a relation as a translating operation on a hyperplane. In addition, TransH proposes "bern.", a strategy of constructing negative labels.

7. TransR & CTransR: Learning Entity and Relation Embeddings for Knowledge Graph Completion. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, Xuan Zhu. AAAI 2015. paper KB2E OpenKE

   An entity may have multiple aspects and various relations may focus on different aspects of entites. TransR first projects entities from entity space to corresponding relation space and then builds translations between projected entities. CTransR extends TransR by clustering diverse head-tail entity pairs into groups and learning distinct relation vectors for each group, which is the initial exploration for modeling internal correlations within each relation type.

8. TransD: Knowledge Graph Embedding via Dynamic Mapping Matrix. Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, Jun Zhao. ACL 2015. paper KB2E OpenKE

   TransD constructs a dynamic mapping matrix for each entity-relation pair by considering the diversity of entities and relations simultaneously. Compared with TransR/CTransR, TransD has fewer parameters and has no matrix vector multiplication.

9. TransA: An Adaptive Approach for Knowledge Graph Embedding. Han Xiao, Minlie Huang, Hao Yu, Xiaoyan Zhu. arXiv 2015. paper

   Applying elliptial equipotential hypersurfaces and weighting specific feature dimensions for a relation, TransA can model complex entities and relations.

10. KG2E: Learning to Represent Knowledge Graphs with Gaussian Embedding. Shizhu He, Kang Liu, Guoliang Ji and Jun Zhao. CIKM 2015. paper code

    Different entities and relations may contain different certainties, which represent the confidence for indicating the semantic when scoring a triple. KG2E represents each entity/relation by a Gaussion distribution, where the mean denotes its position and the covariance presents its certainty.

11. DistMult: Embedding Entities and Relations for Learning and Inference in Knowledge Bases. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng. ICLR 2015. paper code

    DistMult is based on the bilinear model where each relation is represented by a diagonal rather than a full matrix. DistMult enjoys the same scalable property as TransE and it achieves superior performance over TransE.

12. PTransE: Modeling Relation Paths for Representation Learning of Knowledge Bases. Yankai Lin, Zhiyuan Liu, Huanbo Luan, Maosong Sun, Siwei Rao, Song Liu. EMNLP 2015. paper code

    Multi-step relation paths contain rich inference patterns between entities. PTransE considers relation paths as translations between entities and designs an excellent algorithm to measure the reliablity of relation paths. Experiment shows PTransE achieves outstanding improvements on KBC and RE tasks.

13. RTransE: Composing Relationships with Translations. Alberto García-Durán, Antoine Bordes, Nicolas Usunier. EMNLP 2015. paper

    RTransE learns to explicitly model composition of relationships via the addition of their corresponding translations vectors. In addition, the experiments include a new evaluation protocal, in which the model answers questions related to compositions of relations directly.

14. ManifoldE: From One Point to A Manifold: Knowledge Graph Embedding For Precise Link Prediction. Han Xiao, Minlie Huang and Xiaoyan Zhu. IJCAI 2016. paper

    ManifoldE expands point-wise modeling in the translation-based principle to manifold-wise modeling, thus overcoming the issue of over-strict geometric form and achieving remarkable improvements for precise link prediction.

15. TransG: A Generative Mixture Model for Knowledge Graph Embedding. Han Xiao, Minlie Huang, Xiaoyan Zhu. ACL 2016. paper code

    A relation in knowledge graph may have different meanings revealed by the associated entity pairs. TransG generates multiple translation components for a relation via a Bayesian non-parametric infinite mixture model.

16. ComplEx: Complex Embeddings for Simple Link Prediction. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier and Guillaume Bouchard. ICML 2016. paper code OpenKE

    ComplEx extends DistMult by introducing complex-valued embeddings so as to better model asymmetric relations. It is proved that HolE is subsumed by ComplEx as a special case.

17. ComplEx extension: Knowledge Graph Completion via Complex Tensor Factorization. Théo Trouillon, Christopher R. Dance, Johannes Welbl, Sebastian Riedel, Éric Gaussier, Guillaume Bouchard. JMLR 2017. paper code OpenKE

18. HolE: Holographic Embeddings of Knowledge Graphs. Maximilian Nickel, Lorenzo Rosasco, Tomaso A. Poggio. AAAI 2016. paper code OpenKE

    HolE employs circular correlations to create compositional representations. HolE can capture rich interactions but simultaneously remains efficient to compute.

19. KR-EAR: Knowledge Representation Learning with Entities, Attributes and Relations. Yankai Lin, Zhiyuan Liu, Maosong Sun. IJCAI 2016. paper code

    Existing KG-relations can be divided into attributes and relations, which exhibit rather distinct characteristics. KG-EAR is a KR model with entities, attributes and relations, which encodes the correlations between entity descriptions.

20. TranSparse: Knowledge Graph Completion with Adaptive Sparse Transfer Matrix. Guoliang Ji, Kang Liu, Shizhu He, Jun Zhao. AAAI 2016. paper code

    The entities and relations in a knowledge graph are heterogeneous and unbalanced. To overcome the heterogeneity, TranSparse uses sparse matrices to model the relations. To deal with the problem of imbalance of relations, each relation has two separate sparse transfer matrices for head and tail entity.

21. TKRL: Representation Learning of Knowledge Graphs with Hierarchical Types. Ruobing Xie, Zhiyuan Liu, Maosong Sun. IJCAI 2016. paper code

    Entities should have multiple representations in different types. TKRL is the first attempt to capture the hierarchical types information, which is significant to KRL.

22. TEKE: Text-Enhanced Representation Learning for Knowledge Graph. Zhigang Wang, Juan-Zi Li. IJCAI 2016. paper

    TEKE incorporates the rich textual content information to expand the semantic structure of the knowledge graph. Thus, each relation is enabled to own different representations for different head and tail entities to better handle 1-N/N-1/N-N relations. TEKE handle the problems of low performance on 1-N/N-1/N-N1 relations and KG sparseness.

23. STransE: A Novel Embedding Model of Entities and Relationships in Knowledge Bases. Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu and Mark Johnson. NAACL-HLT 2016. paper code

    STransE is a simple combination of the SE and TransE model, using two projection matrices and one translation vector to represent each relation. STransE produces competitive results on link prediction evaluations.

24. GAKE: Graph Aware Knowledge Embedding. Jun Feng, Minlie Huang, Yang Yang, Xiaoyan Zhu. COLING 2016. paper code

    Regarding a knowledge base as a directed graph rather than independent triples, GAKE utilizes graph context (neighbor/path/edge context) to learn knowledge representions. Furthermore, GAKE designs an attention mechanism to learn representitive powers of different subjects.

25. DKRL: Representation Learning of Knowledge Graphs with Entity Descriptions. Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, Maosong Sun. AAAI 2016. paper code

    DKRL takes advantages of entity descriptions to learn knowledge representations. Outstanding performances under the zero-shot setting indicate that DKRL is capable of building representations for novel entities according to their descriptions.

26. ProPPR: Learning First-Order Logic Embeddings via Matrix Factorization. William Yang Wang, William W. Cohen. IJCAI 2016. paper

    ProPPR is the first foraml study to investigate the problem of learning low-dimensional first-order logic embeddings from scratch, while scaling formula embeddings based probabilistic logic reasoning to large knowledge graphs.

27. SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions. Han Xiao, Minlie Huang, Lian Meng, Xiaoyan Zhu. AAAI 2017. paper

    SSP models the strong correlatons between triples and the textual correlations by performing the embedding process in a sementic improvements against the state-of-the-art baselines.

28. ProjE: Embedding Projection for Knowledge Graph Completion. Baoxu Shi, Tim Weninger. AAAI 2017. paper code

    ProjE views the KGC task as a ranking problem and projects candidate-entities onto a vector representing a combined embedding of the known parts of an input triple. Besides, ProjE optimizes a ranking loss of the list of candidate-entities collectively. ProjE can be viewed as a simplified version of NTN.

29. ANALOGY: Analogical Inference for Multi-relational Embeddings. Hanxiao Liu, Yuexin Wu, Yiming Yang. ICML 2017. paper code

    Analogical inference is of greate use to knowledge base completion. ANALOGY models analogical structure in knowledge embedding. In addition, it is proved that DistMult, HolE and ComplEx are special cases of ANALOGY.

30. IKRL: Image-embodied Knowledge Representation Learning. Ruobing Xie, Zhiyuan Liu, Tat-Seng Chua, Huan-Bo Luan, Maosong Sun. IJCAI 2017. paper code

    IKRL is the first attemp to combine images with knowledge graphs for KRL. Its promising performances indicate the significance of visual information for KRL.

31. RUGE: Knowledge Graph Embedding with Iterative Guidance from Soft Rules. Shu Guo, Quan Wang, Lihong Wang, Bin Wang, Li Guo. AAAI 2018. paper code

    RUGE is the first work that models interactions between embedding learning and logical inference in a principled framework. It enables an embedding model to learn simultaneously from labeled triples, unlabeled triples and soft rules in an iterative manner.