Optimization steepest ascent technique

The most popular optimization techniques are Newton-Raphson optimization, steepest ascent optimization, steepest descent optimization. Simplex optimization. Genetic Algorithm optimization, simulated annealing. - Variable reduction and - variable selection are also among the optimization techniques. 

It is well known in computer science that steepest ascent is a poor optimization technique for landscapes with many optima. To improve the search, simulated annealing includes an effective temperature that determines the ability of a walker to overcome energy barriers (Kirkpat-... 

The method of steepest ascent and the simplex search can handle only one criterion, while the resportse surface methods allow simultaneous mapping of several responses. Response surface modelling can therefore be used to optimize several responses simultaneously. The problem of multiple responses is elegantly handled by PLS modelling. This technique is discussed in Chapter 17. 

Using a direct search technique on the performance index and the steepest ascent method, Seinfeld and Kumar (1968) reported computational results on non-linear distributed systems. Computational results were also reported by Paynter et al. (1969). Both the gradient and the accelerated gradient methods were used and reported (Beveridge and Schechter, 1970 Wilde, 1964). All the reported computational results were carried out through discretization. However, the property of hyperbolic systems makes them solvable without discretization. This property was first used by Chang and Bankoff (1969). The method of characteristics (Lapidus, I962a,b) was used to synthesize the optimal control laws of the hyperbolic systems. 

If the goal of the experimentation has been to optimize something, the next step after analysing the results of a 2 or a fractional 2 design is to try to make improvement using knowledge provided by the analysis. The most common technique is the method of steepest ascent, also called the gradient (path) method. 

The basic difficulty with the steepest descent method is that it is too sensitive to the scaling of/(x), so that convergence is very slow and what amounts to oscillation in the x space can easily occur. For these reasons steepest descent or ascent is not a very effective optimization technique. Fortunately, conjugate gradient methods are much faster and more accurate. 

---