Objective function contours

Figure 2.41. Objective function contour plot using SIMUSOLV.

Circular feasible region with objective function contours and the constraint. 

Circular objective function contours and linear inequality constraint. 

In Figure 8.12, this direction (the dashed line) points to the center of the circular objective function contours at (0.5, 2.5). In step 4, the line search moves along dc... 

Circular objective function contours with a nonlinear inequality constraint. 

Each of the function-minimization procedures involves some assessment of the objective function contour in parameter space. This contour is easily visualized in three dimensions, as shown in Figure 19.1(a), but the large number of adjustable parameters used in typical regressions makes graphical visualization cumbersome if not impossible. Numerical methods, such as those described in this chapter, are therefore required. 

Figure 19.6 Operating window for a 2 X 2 optimization problem. The dashed lines are objective function contours, increasing from left to right. The maximum profit occurs where the profit line intersects the constraints at vertex D.

Secondly, the linearized inverse problem is, as well as known, ill-posed because it involves the solution of a Fredholm integral equation of the first kind. The solution must be regularized to yield a stable and physically plausible solution. In this apphcation, the classical smoothness constraint on the solution [8], does not allow to recover the discontinuities of the original object function. In our case, we have considered notches at the smface of the half-space conductive media. So, notche shapes involve abrupt contours. This strong local correlation between pixels in each layer of the half conductive media suggests to represent the contrast function (the object function) by a piecewise continuous function. According to previous works that we have aheady presented [14], we 2584... 

Figure 8.1 Contours of the objective function in the vicinity of the optimum. Potential problems with overstepping are shown for a tivo-parameter problem.

Two variable problems can be plotted as shown in Figure 1.17. The values of the objective function are shown as contour lines, as on a map, which are slices through the three-dimensional model of the function. Seeking the optimum of such a function can be... 

The effectiveness of the above described computational procedure was tested by generating an analytical ("ideal") data curve by calculating the isocyanate concentration as a function of time assuming rate constants of k = 1.0/min and k2 = 1.0 A/mol/min and initial concentrations of blocked isocyanate and hydroxyl of 1.0 M. The objective function, for various values of k. and k, for this "ideal" data was calculated and a contour plot for constant values of F was generated and is shown in Figure 2. 

Instead of a formal development of conditions that define a local optimum, we present a more intuitive kinematic illustration. Consider the contour plot of the objective function fix), given in Fig. 3-54, as a smooth valley in space of the variables X and x2. For the contour plot of this unconstrained problem Min/(x), consider a ball rolling in this valley to the lowest point offix), denoted by x. This point is at least a local minimum and is defined by a point with a zero gradient and at least nonnegative curvature in all (nonzero) directions p. We use the first-derivative (gradient) vector Vf(x) and second-derivative (Hessian) matrix V /(x) to state the necessary first- and second-order conditions for unconstrained optimality ... 

This problem is shown in Figure 4.5. The feasible region is defined by linear constraints with a finite number of comer points. The objective function, being nonlinear, has contours (the concentric circles, level sets) of constant value that are not parallel lines, as would occur if it were linear. The minimum value of/corresponds to the contour of lowest value having at least one point in common with the feasible region, that is, at xx = 2, x2 = 3. This is not an extreme point of the feasible set, although it is a boundary point. For linear programs the minimum is always at an extreme point, as shown in Chapter 7. 

Neither of the problems illustrated in Figures 4.5 and 4.6 had more than one optimum. It is easy, however, to construct nonlinear programs in which local optima occur. For example, if the objective function / had two minima and at least one was interior to the feasible region, then the constrained problem would have two local minima. Contours of such a function are shown in Figure 4.7. Note that the minimum at the boundary point x1 = 3, x2 = 2 is the global minimum at / = 3 the feasible local minimum in the interior of the constraints is at / = 4. 

Geometry of a quadratic objective function of two independent variables—elliptical contours. If the eigenvalues are equal, then the contours are circles. 

Figure 4.12 corresponds to objective functions in well-posed optimization problems. In Table 4.2, cases 1 and 2 correspond to contours of /(x) that are concentric circles, but such functions rarely occur in practice. Elliptical contours such as correspond to cases 3 and 4 are most likely for well-behaved functions. Cases 5 to 10 correspond to degenerate problems, those in which no finite maximum or minimum or perhaps nonunique optima appear. 

For well-posed quadratic objective functions the contours always form a convex region for more general nonlinear functions, they do not (see tlje next section for an example). It is helpful to construct contour plots to assist in analyzing the performance of multivariable optimization techniques when applied to problems of two or three dimensions. Most computer libraries have contour plotting routines to generate the desired figures. 

Figure 4.16 illustrates the character of ffx) if the objective function is a function of a single variable. Usually we are concerned with finding the minimum or maximum of a multivariable function fix)- The problem can be interpreted geometrically as finding the point in an -dimension space at which the function has an extremum. Examine Figure 4.17 in which the contours of a function of two variables are displayed. 

Figure 4.17b shows contours for the objective function in this example. Note that the global minimum can only be identified by evaluating/(x) for all the local minima. 

First, let us consider the perfectly scaled quadratic objective function /(x) = x + x, whose contours are concentric circles as shown in Figure 6.6. Suppose we calculate the gradient at the point xT = [2 2]... 

Contours of constant value of the objective function/are defined by the linear equation... 

The geometry of this problem is shown in Figure 8.11. The linear equality constraint is a straight line, and the contours of constant objective function values are circles centered at the origin. From a geometric point of view, the problem is to find the point on the line that is closest to the origin at x = 0, y = 0. The solution to the problem is at x = 2, y = 2, where the objective function value is 8. 

The feasible region and some contours of the objective function are shown in Figure 8.12. The goal is to find the feasible point that is closest to the point (0.5, 2.5), which is (1.5, 1.5). 

Results of the optimization. Figure El2.2b illustrates contours of the objective function for the plug flow model the objective function (/) was optimized by the GRG... 

Contours (the heavy lines) for the objective function of extraction process. Points 1, 2, 3, and 4 indicate the progress of the reduced-gradient method toward the optimum (point 4). 

As shown in Fig. 1.2, to solve this problem we need only analytical geometry. The constraints (1.29) restrict the solution to a convex polyhedron in the positive quadrant of the coordinate system. Any point of this region satisfies the inequalities (1.29), and hence corresponds to a feasible vector or feasible solution. The function (1.30) to be maximized is represented by its contour lines. For a particular value of z there exists a feasible solution if and only if the contour line intersects the region. Increasing the value of z the contour line moves upward, and the optimal solution is a vertex of the polyhedron (vertex C in this example), unless the contour line will include an entire segment of the boundary. In any case, however, the problem can be solved by evaluating and comparing the objective function at the vertices of the polyhedron. 

Fig. 1.2. Feasible region and contour lines of the objective function...

If the j-th column is in the basis then zj cj = 0 follows, but an entry of the last row of the simplex tableau may vanish also for a column that is not in the basis. If this situation occures in the optimal simplex tableau then the linear programming problem has several optimal basic solutions. In our preliminary example this may happen when contour lines of the objective function are parallel to a segment of the boundary of the feasible region. 

As the convergence ratio measures the reduction of the error at every step (llx +i x ll — Pita x ll for a linear rate), the relevant SD value can be arbitrarily close to 1 when k is large (Figure 12). In other words, because the n lengths of the elliptical axes belonging to the contours of the function are proportional to the eigenvalue reciprocals, the convergence rate of SD is slowed as the contours of the objective function become more eccentric. Thus, the SD search vectors may in some cases exhibit very inefficient paths toward a solution (see final section for a numerical example). 

Several features of the optimization problem are apparent in Figure 19.1(a). The model is nonlinear with respect to parameters nevertheless, the objective function is well behaved near the solution where it can be approximated by a quadratic function. The contours projected onto the base of the plot have an elliptical shape. The major axis of the ellipse does not lie along either axis. 

Figure 19.1 The objective function, equation (19.3), for an RC circuit as a function of parallel resistor and capacitor values. The circuit parameters were Ri = 1 Q, and Ti = 1 s. The synthetic data were calculated for frequencies ranging from 1 to 10 Hz at a spacing of 10 points per decade, and the noise was determined by machine precision. The objective function at the set parameter values was found to be equal to zero a) 3-D perspective drawing of the contour surface b) 2-D representation of the contour surface.

The Method of Steepest Descent tends to be quite efficient far from the solution, but convergence can be painfully slow near the solution. Slow convergence is likely where die contours are attenuated and banana-shaped, i.e., where the method tends to change direction often with minimal changes in objective function value. 

Figure 18.18 illustrates the shift of the position of the optimum experimental conditions when Pr is replaced by the Pr X Y objective function for the optimization. The maximum production rate is at point A, while Pr x Y reaches its maximum at point B. The contour lines clearly show that the production rate is hardly lower at the new optimum. On the other hand, the recovery yield is improved when the experimental conditions are shifted from point A to point B. The surface determined by Pr x Y exhibits a well defined maximum, which makes the numerical path toward optimization stable. 

---