Hyperparameter optimization refers to the process of tuning the parameters of a model that are not learned during training but must be set in advance.
These hyperparameters significantly influence the performance and behavior of models, such as learning rate, number of layers, or regularization strength.
Typical methods for optimization are grid search, random search, or Bayesian optimization.
In this process, a search space is defined and systematically searched for combinations that improve the model on validation data.
Efficient hyperparameter optimization is crucial to best adapt models to data and avoid overfitting.