Stochastic optimization methods

Optimization problems arising in practice involve random model parameters. For the computation of robust optimal solutions, i.e., optimal solutions being insensitive with respect to random parameter variations, appropriate deterministic substitute problems are needed. Based on the probability distribution of the random data, and using decision theoretical concepts, optimisation problems under stochastic uncertainty are converted into appropriate deterministic substitute problems. Due to the occurring probabilities and expectations, hence, multiple integrals, approximative solution techniques must be applied. Several deterministic and stochastic approximation methods are provided: Taylor expansion methods, regression and response surface methods (RSM), probability inequalities, First order reliability methods (FORM), convex approximation/deterministic descent directions/efficient points, stochastic gradient methods, stochastic approximation procedures, differentiation formulas for probabilities and expectations. The mathematical properties of the approximative problems and iterative solution procedures (e.g. rate of convergence) are described.