Unconstrained problem

Local Minimum Point for Unconstrained Problems Consider the following unconstrained optimization problem ... 

The calculations begin with given values for the independent variables u and exit with the (constrained) derivatives of the objective function with respec t to them. Use the routine described above for the unconstrained problem where a succession of quadratic fits is used to move toward the optimal point for an unconstrained problem. This approach is a form or the generahzed reduced gradient (GRG) approach to optimizing, one of the better ways to cany out optimization numerically. 

Instead of a formal development of conditions that define a local optimum, we present a more intuitive kinematic illustration. Consider the contour plot of the objective function fix), given in Fig. 3-54, as a smooth valley in space of the variables X and x2. For the contour plot of this unconstrained problem Min/(x), consider a ball rolling in this valley to the lowest point offix), denoted by x. This point is at least a local minimum and is defined by a point with a zero gradient and at least nonnegative curvature in all (nonzero) directions p. We use the first-derivative (gradient) vector Vf(x) and second-derivative (Hessian) matrix V /(x) to state the necessary first- and second-order conditions for unconstrained optimality ... 

Banga et al. [in State of the Art in Global Optimization, C. Floudas and P. Pardalos (eds.), Kluwer, Dordrecht, p. 563 (1996)]. All these methods require only objective function values for unconstrained minimization. Associated with these methods are numerous studies on a wide range of process problems. Moreover, many of these methods include heuristics that prevent premature termination (e.g., directional flexibility in the complex search as well as random restarts and direction generation). To illustrate these methods, Fig. 3-58 illustrates the performance of a pattern search method as well as a random search method on an unconstrained problem. 

Some unconstrained problems inherently involve only one variable... 

In nonlinear programming problems, optimal solutions need not occur at vertices and can occur at points with positive degrees of freedom. It is possible to have no active constraints at a solution, for example in unconstrained problems. We consider nonlinear problems with constraints in Chapter 8. 

Solution. This problem is illustrated graphically in Figure E8.1a. Its feasible region is a circle of radius one. Contours of the linear objective xx + x2 are lines parallel to the one in the figure. The contour of lowest value that contacts the circle touches it at the point x = (—0.707, —0.707), which is the global minimum. You can solve this problem analytically as an unconstrained problem by substituting for xx or x2 by using the constraint. 

The essential idea of a penalty method of nonlinear programming is to transform a constrained problem into a sequence of unconstrained problems. 

GRG algorithms use a basic descent algorithm described below for unconstrained problems. We state the steps here ... 

Find the optimum of P with respect to xx and x2 (an unconstrained problem), noting that jc and x are functions of r. 

This weighted sum of absolute values in e(x) was also discussed in Section 8.4 as a way of measuring constraint violations in an exact penalty function. We proceed as we did in that section, eliminating the nonsmooth absolute value function by introducing positive and negative deviation variables dpt and dnt and converting this nonsmooth unconstrained problem into an equivalent smooth constrained problem, which is... 

Because software to find local solutions of NLP problems has become so efficient and widely available, multistart methods, which attempt to find a global optimum by starting the search from many starting points, have also become more effective. As discussed briefly in Section 8.10, using different starting points is a common and easy way to explore the possibility of local optima. This section considers multistart methods for unconstrained problems without discrete variables that use randomly chosen starting points, as described in Rinnooy Kan and Timmer (1987, 1989) and more recently in Locatelli and Schoen (1999). We consider only unconstrained problems, but constraints can be incorporated by including them in a penalty function (see Section 8.4). 

Step 2 Formulation of the unconstrained problem. Applying previous results, the (y — x) vector of the objective function is modified as follows ... 

Let us apply the sequential processing of the constraints to the system defined previously in Example 5.1. We will process one equation at a time starting from the unconstrained problem. 

A key idea in developing necessary and sufficient optimality conditions for nonlinear constrained optimization problems is to transform them into unconstrained problems and apply the optimality conditions discussed in Section 3.1 for the determination of the stationary points of the unconstrained function. One such transformation involves the introduction of an auxiliary function, called the Lagrange function L(x,A, p), defined as... 

The transformed unconstrained problem then becomes to find the stationary points of the Lagrange function... 

An alternative to the method of Lagrange multipliers for imposing the necessary constraints is sketched below. It derives a lower dimensional unconstrained problem from the original constrained problem by using an orthogonal basis for the null space of the constraint matrix. This method is well suited to the potentially rank-deficient problem at hand, where steps may be taken to... 

The ZDT test problems are two objective problems framed by Zitzler et al. [Deb (2001)]. ZDT test problems are unconstrained problems. 

In this section we give a brief review of the mathematics involved in solving the linear least-squares problems, with or without linear constraints, as encountered in this chapter. More details about the unconstrained problem can be found in any text on linear regression, e.g. Draper and Smith (1981), or Press, et al. (1986). More details about the constrained problem can be found in Lawson and Hanson (1974). 

There is no restriction on z, so the least-squares solution to this unconstrained problem is, following equation 7.86,... 

It is obvious that in general, the charge distribution that solves (9) is not q = — x. which is the solution to the unconstrained problem Jq = —x, but contains an additional term that corresponds to a correction to account for the constraint of overall charge conservation. We note that this solution has only been published previously in the ES-I- model of Streitz and Mintmire [10] and was later rederived by Bultinck and Carbd-Dorca [54]. 

---