Bayesian optimization is a method for global optimization of expensive, black-box functions where derivatives are not available.
It combines a probabilistic surrogate model, usually a Gaussian Process (GP), with an acquisition function that determines which input points to select next to evaluate the function.
The surrogate model approximates the unknown objective function and provides uncertainty estimates that are used to maximize information gain.
The acquisition function balances exploration (exploring unknown areas) and exploitation (using known good areas), e.g., through Expected Improvement or Upper Confidence Bound.
Bayesian optimization is particularly suitable for applications such as hyperparameter tuning of machine learning models or experimental design processes where evaluations are expensive.