• Marcos Paulo Silva Gôlo (ICMC/USP) | [email protected]
• Marcelo Isaias de Moraes Junior (ICMC/USP) | [email protected]
• Rudinei Goularte (UFMS) | [email protected]
• Ricardo Marcacini (ICMC/USP) | [email protected]

If you use any part of this code or research in your research, please cite it using the following BibTex entry

@article{ref:Golo2024,
  title={Unsupervised Heterogeneous Graph Neural Networks for One-Class Tasks: Exploring Early Fusion Operators},
  author={G{\^o}lo, Marcos Paulo Silva and de Moraes Junior, Marcelo Isaias and Goularte, Rudinei and Marcacini, Ricardo Marcondes},
  journal={Journal on Interactive Systems},
  volume={15},
  number={1},
  pages={517--529},
  year={2024}
}

Heterogeneous graphs are an essential structure that models real-world data through different types of nodes and relationships between them, including multimodality, which comprises different types of data such as text, image, and audio. Graph Neural Networks (GNNs) are a prominent graph representation learning method that takes advantage of the graph structure and its attributes that, when applied to the multimodal heterogeneous graph, learn a unique semantic space for the different modalities. Consequently, it allows multimodal fusion through simple operators such as sum, average, or multiplication, generating unified representations considering the supplementary and complementarity relationships between the modalities. In multimodal heterogeneous graphs, the labeling process tends to be even more costly due to the multiple modalities analyzed, in addition to the imbalance of classes inherent to some applications. In order to overcome these problems in applications that comprise a class of interest, One-Class Learning (OCL) is used. Given the lack of studies on multimodal early fusion in heterogeneous graphs for OCL tasks, we proposed a method based on unsupervised GNN for heterogeneous graphs and evaluated different early fusion operators. In this paper, we extend another work by evaluating the behavior of the main GNN convolutions in the method. We highlight that using operators such as subtraction, multiplication, and average were the best early fusion operators. In addition, GNN layers that do not use an attention mechanism performed better. In this way, we argue for heterogeneous graph neural networks in multimodal using early fusion simple operators instead of well-often-used concatenation and less complex convolutions.

• Fake News : https://github.com/GoloMarcos/FKTC.git

• Recommender Systems : https://github.com/GoloMarcos/One-Class-Recommendation-GNNLinkPrediciton.git

• Music : : https://github.com/AngeloMendes/Unsupervised-Heterogeneous-GraphNeural-Network-for-Hit-Song-Prediction-through-One-Class-Learning.git

• Event : https://github.com/joaopedromattos/GNEE