Steepest ascent Principles

The method of steepest ascent was the first method by which multivariate experiments could be designed with a view to achieving a systematic improvement of the result. It was described by Box and Wilson[l] as early as 1951, and has been much used over the years, especially in industrial experimentation. The underlying principles are simple as can be seen in the following geometric illustration. 

A linear response surface is shown in Fig. 10.1. It is seen that the best (highest) response within the domain is at the limit. It is also seen that an increased response can be expected outside the explored domain, in a direction which describes the steepest path upwards along the response surface. When this direction is known, it is possible to run a series of experiments in which the settings of the variables are adjusted to follow the path of the steepest ascent, see Fig 10.2. 

The experiments are carried out in this direction as long as they lead to improvements. This will cease to be the case when the experiments have reached a domain where the response surface is no longer monotonous, e.g. the path of the steepest ascent has passed over a rising ridge. Fig 10.3a, or has reached a domain where the surface is saddle-shaped. Fig. 10,3b. 

When this happens, there are four possibilities (1) A. satisfactory result has been obtained, and further investigations are unnecessary. (2) A new direction of the steepest ascent is determined and the investigation is continued in this direction. Fig. 10.4. (3) The turning point is probably close to the optimum conditions and to locate the optimum more precisely, a second order response surface model is established to map the optimum domain. (4) The improvements obtained are not significant enough, and the investigation is abandoned. 

The method will thus not permit a detailed localization of the optimum conditions. It is a method by which it is possible to rapidly arrive at a near-optimum experimental domain. 

---

Special attention is paid to replication of trials when applying the method of steepest ascent. Trials with best response values are in principle replicated only, although it is not pointless to replicate all trials. 

Figure 5-3 The top part of the figure shows the isolines of the misfit functional map and the steepest descent path of the iterative solutions in the space of model parameters. The bottom part presents a magnified element of this map with just one iteration step shown, from iteration (n. — 1) to iteration number ti. According to the line search principle, the direction of the steepest ascent at iteration number n must be perpendicular to the misfit isoline at the minimum point along the previous direction of the steepest descent. Therefore, many steps may be required to reach the global minimum, because every subsequent steepest descent direction is perpendicular to the previous one, similar to the path of experienced slalom skiers.

If the design was for a second-order model and examination of the contour plots or canonical analysis (see below) showed that the optimum probably lay well outside the experimental domain, then the direction for exploration would no longer be a straight line, as for the steepest ascent method. In fact, the "direction of steepest ascent" changes continually and lies on a curve called the optimum path. The calculations for determining it are complex, but with a suitable computer program the principle and graphical interpretation become easy. 

---