Self-supervised learning (SSL) refers to a learning paradigm in which models are trained by solving prediction or reconstruction tasks on unlabeled data, without explicit labels.
Typical methods include contrastive learning, masked model prediction (e.g., masked language models like BERT), and context prediction.
SSL aims to learn meaningful, generalizable representations that can be fine-tuned for downstream tasks.
The method addresses the challenge of label scarcity and improves learning efficiency on large, heterogeneous datasets.
Definition:
“Self-supervised learning is a paradigm where the system learns representations from unlabeled data by solving pretext tasks that generate supervisory signals from the data itself.”
Source:
Jing, L., & Tian, Y. (2020). Self-supervised visual feature learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11), 4037-4058.