Direction of the steepest ascent

It is realized that the direction of the steepest ascent is related to the slope of the response surface. Thus, to determine this direction, it is necessary first to determine the slope of the response surface by means of a linear approximation of the response surface model. 

an initial screening experiment can be used to determine the model. 

How the experimental variables, Xj, should be changed, Xj + AXj, to give a maximum increase, Ay, of the response can be explained as follows.  

This is a scalar product of the vectors b and Ax, which also can be written 

The scalar product has its maximum valued when cos((A) = 1, i.e. the angle is zero, which is obtained when the vectors are parallel. 

---

The well-known Box-Wilson optimization method (Box and Wilson [1951] Box [1954, 1957] Box and Draper [1969]) is based on a linear model (Fig. 5.6). For a selected start hyperplane, in the given case an area A0(xi,x2), described by a polynomial of first order, with the starting point yb, the gradient grad[y0] is estimated. Then one moves to the next area in direction of the steepest ascent (the gradient) by a step width of h, in general... 

This result comes from the simple fact that, if we minimize a functional along some direction, described by a parametric line, the direction of the steepest ascent must be perpendicular to this line at the minimum point on the line (see Figure 5-3) otherwise we would still not be reaching the minimum along this line. A formal proof of this result was presented in Chapter 4 for the linear operator A, when we discussed the minimal residual method for the linear inverse problem solution (formula (4.51)). 

Figure 5-3 The top part of the figure shows the isolines of the misfit functional map and the steepest descent path of the iterative solutions in the space of model parameters. The bottom part presents a magnified element of this map with just one iteration step shown, from iteration (n. — 1) to iteration number ti. According to the line search principle, the direction of the steepest ascent at iteration number n must be perpendicular to the misfit isoline at the minimum point along the previous direction of the steepest descent. Therefore, many steps may be required to reach the global minimum, because every subsequent steepest descent direction is perpendicular to the previous one, similar to the path of experienced slalom skiers.

However, the dirc ctions of ascent l(m j arc selected in a different way. On the first step we use the direction " of the steepest ascent ... 

Therefore, according to (7.6), the direction of the steepest ascent is equal to... 

Note that we can give the same physical interpretation to every subsequent iteration in the iterative scheme (7.26). According to formulae (7.12) and (7.43), the direction of the steepest ascent I (p ) on each iteration can be computed using migration of the residual field (Q — gr (C)], which is the difference between the predicted field on the n-th iteration, g , and the observed gravity field gr ... 

When this happens, there are four possibilities (1) A. satisfactory result has been obtained, and further investigations are unnecessary. (2) A new direction of the steepest ascent is determined and the investigation is continued in this direction. Fig. 10.4. (3) The turning point is probably close to the optimum conditions and to locate the optimum more precisely, a second order response surface model is established to map the optimum domain. (4) The improvements obtained are not significant enough, and the investigation is abandoned. 

Fig.10.5 A series of experiments is defined by a successive displacement along the direction of the steepest ascent.

However, it is not to be taken for granted that the experiments in Table 10.3 really are located on the path of steepest ascent if the response surface model should be expressed in the natural variables. The slopes in different directions of the response surface are not invariant when the variables are transformed. When the coded variables, Xj, are translated back to the natural variables, u, the step of variation of the natural variables, Sj, will intervene and this may change the direction of the steepest ascent path. The direction is invariant to a change of variables, only if the steps of variation are equal for all variables. If there are different units of measurement, e.g. " C, h, equivalents of reagents, etc., they are likely to be different. 

U. Different directions of the steepest ascent path due to scaling of the variahles... 

The direction of the steepest ascent in this case will be... 

From this follows that the vector, Au, describing the variation of Mi and M2 in the direction of the steepest ascent will be... 

The increments of each natural variable, j, is determined by the response surface slopes, fej, in the space spanned by the x, variables (x-space), and the step of variation, Su used to scale the natural variables to unit variation in the x-space. From this it is seen, that the directions of the steepest ascent in the x-space and the u-space are equal, (Ax parallel to Au), only if the scaling factors, Smj, are all equal. 

The method can be used directly after an initial screening. The effects of the variables as determined from the screening experiment can be used to define the direction of the steepest ascent. This allows of a stepwise approach. 

To determine the direction of the steepest ascent, all estimated slopes along the variable axes are used. This means that even variables with a minor influence are used for maximum profit. We cannot be sure that variables associated with small linear coefficients are without any influence. If they have a small influence, this is picked up by the model parameter, and the method of steepest ascent will pay account for even such small influences. 

The method is sensitive to the range of variation explored in the initial design. The direction of the steepest ascent will depend on the scaling of the variables. 

The linear model, used to define the direction of the steepest ascent, is a local model, which is valid only in the explored domain. When experiments are run along the direction of the steepest ascent, this is actually an extrapolation from the known... 

In practice, this means that the series of simplex experiments will describe a zig-zag path along the direction of the steepest ascent. The decision in which direction to move is made on the basis of the last simplex run. Hence, the method does not involve any drastic extrapolations. The movement upwards will be adjusted to follow the direction of the steepest ascent, even if the slopes of the response surface is changed when we move away from the initial experimental domain. 

It is well known from calculus that the direction of the steepest ascent on a response surface is given by the gradient vector, i.e. the vector of p>artial derivatives with respect to the design variables at a given point. The basic idea has been presented in section 3.2, and now we shall present the technical details. 

Sequential strategies of optimization are based on an initial design of an experiment followed by a sequence of further measurements in the direction of the steepest ascent or descent. That is, no quantitative relationship between factors and responses is evaluated, but the response surface is searched along an optimal (invisible) path. The two strategies are exemplified in Figure 4.1. 

---