Gradient-based algorithms

Using optimization algorithms with these black box models is a challenging problem for at least two reasons. First, some of those models can take significant CPU computation and second, derivatives for gradient based algorithms usually cannot be accurately estimated because most of these black box models introduce noise (small sensitivity of some variables, termination criteria in the algorithms, etc). 

Snyman, J.A. (2005). Practical Mathematical Optimization. An Introduction to Basic Optimization Theory ami Classical and New Gradient-Based Algorithms, Springer, New York, 257 p. 

Pressure or density programming is the most popular of the gradient techniques in SFC. Density is the important parameter with respect to retention but pressure is the physical property which is directly monitored by SFC instruments. If enough experimental density-volume-temperature data are available for the mobile phase then a computer-based algorithm can be used to generate specific density programs. Such data are available for only a few mobile phases, such as carbon dioxide and the n-... 

An evolutionary algorithm is included in the current release of Frontline Systems Premium Excel Solver (for current information, see www.frontsys.com). It is invoked by choosing Standard Evolutionary from the Solver dropdown list in the Solver Parameters Dialog Box. The other nonlinear solver is Standard GRG Nonlinear, which is the GRG2 solver described in Section 8.7. As discussed there, GRG2 is a gradient-based local solver, which will find the nearest local solution to its starting point. The evolutionary solver is much less likely to stop at a local minimum, as we illustrate shortly. 

Although this example results in fairly extreme conditions, it does point the need to alter the gradient based search to avoid well dewatering and algorithmic failure. Because the search trajectory of the optimization algorithm and the size and shape of the feasible region cannot be predicted, additional search procedures must be included for all unconfined aquifers. 

A neural network is typically trained by variations of gradient descent-based algorithms, trying to minimize an error function [77]. It is important that additional validation data be left untouched during ANN training, so as to have an objective measure of the model s generalization ability [78],... 

Conventional gradient base optimisation techniques are not effective to deal with objective functions with multiple local minima and can be trapped in local minima. Particle swam optimisation (PSO) is a recently developed optimisation technique that can cope with multiple local minima. This paper proposes using PSO and stacked neural networks to find the optimal control policy for batch processes. A standard PSO algorithm and three new PSO algorithms with local search were developed. In order to enhance the reliability of the obtained optimal control policy, an additional term is added to the optimisation objective function to penalise wide model prediction confidence bormds. 

All the four PSO algorithms can find the global optimal solutions whereas the gradient based optimisation algorithm from the MATLAB Optimisation Toolbox, fminunc, fails to find the global optimal solutions when the initial values are not close to the global optimal solutions. 

---