Self-supervised learning with contrastive methods uses the idea that a model learns to place similar data points closer together and dissimilar ones further apart in the feature space.
Typically, data pairs are generated, e.g., different views of the same image, called positive pairs, as well as random other images as negative pairs.
The model is trained so that the representations of the positive pairs are maximized and those of the negative pairs are minimized.
Popular algorithms are SimCLR, MoCo, or BYOL, which use different strategies for generating data pairs and stabilizing training.
This method enables learning robust and generalizable features that can be used for many downstream tasks without major adjustments.