Self-Supervised Learning (SSL) refers to a learning paradigm in which a model learns representations from input data by solving automatically generated tasks (pretext tasks), without relying on manually annotated labels.
Formally, a function f: X → Z is learned that produces a representation z = f(x) for an input data distribution p(x) by minimizing a loss function L(f(x), g(x)), where g(x) is target information derived automatically from the data.
Typical pretext tasks include, for example, predicting context information (e.g., neighboring pixels, missing words), solving jigsaw puzzles for images, or contrastive learning objectives that define positive and negative pairs.
SSL forms a bridge between supervised and unsupervised learning methods and enables efficient use of large amounts of unlabeled data.
Modern models such as SimCLR, MoCo, or BERT are based on self-supervised learning and have significantly improved performance in many areas of AI.
Definition:
“Self-Supervised Learning is a learning paradigm in which models solve automatically generated tasks from input data to learn useful representations without relying on manually labeled data.”
Source:
Jing, L. & Tian, Y. (2020). Self-Supervised Visual Feature Learning with Deep Neural Networks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.