Meta-learning refers to methods in which a learning algorithm is trained on a distribution of tasks to develop a learning strategy that enables rapid adaptation to new tasks.
Formally, one considers a task distribution p(T), where for each task T there are training and test data available. The goal is to learn a meta-parameter \( \theta \) that allows quick adaptation to a new task \( T_i \) through small updates.
A well-known method is Model-Agnostic Meta-Learning (MAML) (Finn et al., 2017), which works by optimizing the following objective:
\[ \min_{\theta} \sum_{T_i \sim p(T)} \mathcal{L}_{T_i}(U(\theta, D^{train}_{T_i})) \]
Here, \( U(\theta, D^{train}_{T_i}) \) is an update step (e.g., a gradient step) on the training data of task \( T_i \), and \( \mathcal{L}_{T_i} \) is the loss function of the task.
Meta-learning thus enables efficient learning with few examples (Few-Shot Learning) and is closely related to transfer learning and automatic algorithm design.
Definition:
“Meta-learning is the process by which a learning algorithm is trained to improve its learning ability so that it can solve new tasks quickly and efficiently with few examples.”
Source:
Finn, C., Abbeel, P., & Levine, S. (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Proceedings of the 34th International Conference on Machine Learning (ICML).