Bayesian optimization is a method for efficient optimization of expensive or complex functions where evaluations are costly.
It uses a probabilistic model, usually a Gaussian process, to estimate the unknown objective and simultaneously quantify the uncertainty of the predictions.
Uncertainty quantification is crucial to achieve a balance between exploration (exploring unknown areas) and exploitation (utilizing known good solutions).
Typical acquisition functions such as Expected Improvement (EI) or Upper Confidence Bound (UCB) use these uncertainties to selectively choose new points that are promising or contribute to reducing uncertainty.
This method is frequently used in hyperparameter optimization, robotics, and materials science, where evaluation is expensive or time-consuming.