Steepest ascent technique

One important use of experimental designs is to achieve optimum operating conditions of industrial processes. For a discussion of this application, see Box and Wilson (1951). This paper is extraordinarily rich in response surface concepts. What is the steepest ascent technique discussed in this paper What models are assumed, and what experimental designs are used ... 

One strategy that has often been used is to proceed along the path of steepest ascent until a maximum is reached. Then another search is made. A path of steepest ascent is determined and followed until another maximum is reached. This is continued until the climber thinks he is in the vicinity of the global maximum. To aid in reaching the maximum, the technique of using three points to estimate a quadratic surface, as was done previously, may be used. 

It is easy to imagine how the method of steepest ascents is generalized to multidimensional studies. It is, however, difficult to portray these cases graphically. They are the most important applications for the method and it is in such large studies that the technique is most advantageous. 

It is well known in computer science that steepest ascent is a poor optimization technique for landscapes with many optima. To improve the search, simulated annealing includes an effective temperature that determines the ability of a walker to overcome energy barriers (Kirkpat-... 

The most popular optimization techniques are Newton-Raphson optimization, steepest ascent optimization, steepest descent optimization. Simplex optimization. Genetic Algorithm optimization, simulated annealing. - Variable reduction and - variable selection are also among the optimization techniques. 

The method of steepest ascent and the simplex search can handle only one criterion, while the resportse surface methods allow simultaneous mapping of several responses. Response surface modelling can therefore be used to optimize several responses simultaneously. The problem of multiple responses is elegantly handled by PLS modelling. This technique is discussed in Chapter 17. 

In the preceding chapters it was discussed how a near-optimum experimental domain can be found by the method of steepest ascent or by a simplex search. However, these methods cannot be used to efficiently locate the optimum conditions. For this, response surface modelling is a far more efficient technique. 

There is, however, one application when the overall desirability D can be rather safely used, namely as a tool in conjunction with response surface modelling. In this context, it can be used to explore the joint modelling of several responses so that a near-optimum region can be located by simulations against the response surface models. The search for conditions which incrase D can be effected either by simplex techniques or by the method of steepest ascent. For the steepest ascent, a linear model for D is first determined from the experiments in the design used to establish the response surface models. The settings which increase D can be translated back into the individual responses by using the response surface models. Thus, it is possible to establish immediately whether the simulated reponse values correspond to suitable experimental conditions. Such results must, of course, be verified by experimental runs. 

Using a direct search technique on the performance index and the steepest ascent method, Seinfeld and Kumar (1968) reported computational results on non-linear distributed systems. Computational results were also reported by Paynter et al. (1969). Both the gradient and the accelerated gradient methods were used and reported (Beveridge and Schechter, 1970 Wilde, 1964). All the reported computational results were carried out through discretization. However, the property of hyperbolic systems makes them solvable without discretization. This property was first used by Chang and Bankoff (1969). The method of characteristics (Lapidus, I962a,b) was used to synthesize the optimal control laws of the hyperbolic systems. 

If the goal of the experimentation has been to optimize something, the next step after analysing the results of a 2 or a fractional 2 design is to try to make improvement using knowledge provided by the analysis. The most common technique is the method of steepest ascent, also called the gradient (path) method. 

The basic difficulty with the steepest descent method is that it is too sensitive to the scaling of/(x), so that convergence is very slow and what amounts to oscillation in the x space can easily occur. For these reasons steepest descent or ascent is not a very effective optimization technique. Fortunately, conjugate gradient methods are much faster and more accurate. 

---