Self-Supervised Learning with Graph Representations Explained for Experts

Self-supervised learning with graph representations refers to a class of learning methods where representations of nodes, edges, or entire graphs are learned through pretext tasks without external labels.

Formally, one considers a graph \( G = (V, E) \) with node set \( V \) and edge set \( E \). The goal is to learn an embedding function \( f_ heta: V \cup E \rightarrow \mathbb{R}^d \) that captures structural and semantic properties.

Typical self-supervised tasks include:

• Contrastive learning, where positive pairs (e.g., nodes and their neighbors) are distinguished from negative pairs.
• Reconstruction tasks, where parts of the graph (e.g., edges or attributes) are predicted.
• Masking of node features and their prediction.

Graph Neural Networks (GNNs) such as GCN, GAT, or GraphSAGE serve as encoders that aggregate local neighborhood information.

Definition:
“Self-supervised learning with graph representations is a learning paradigm that uses graph structures and attributes to learn meaningful embeddings for nodes, edges, or entire graphs through pretext tasks without external labels.”

Source:
Hu, W., et al. (2020). Strategies for Pre-training Graph Neural Networks. Proceedings of the 8th International Conference on Learning Representations (ICLR).