Self-supervised learning with graphs refers to a paradigm in which Graph Neural Networks (GNNs) are trained without external labels by exploiting intrinsic structures and properties of graphs.
Formally, this can be achieved by minimizing a self-defined loss function \( \mathcal{L}_{ssl} \) based on pretext tasks, e.g.:
\[ \min_ heta \mathcal{L}_{ssl} = \mathbb{E}_{(v,u) \sim D}[\ell(f_ heta(v), f_ heta(u))] \]
where \( f_ heta \) is the representation function parameterized by \( heta\) and \( \ell \) is a distance or similarity function between the representations of nodes \(v\) and \(u\).
Contrastive methods such as Deep Graph Infomax (DGI) maximize the mutual information between global and local representations to produce meaningful embeddings.
Other approaches use masking strategies (e.g., Graph Masked Autoencoders) or prediction of contextual information within the graph.
These methods enable the use of large unlabeled graph data and improve performance on downstream tasks through more robust and generalizable feature representations.
Definition:
“Self-supervised learning with graphs is a learning paradigm in which Graph Neural Networks are trained via intrinsic pretext tasks to extract meaningful representations from unlabeled graph data.”
Source:
Velickovic, P., et al. (2019). Deep Graph Infomax. International Conference on Learning Representations (ICLR).