Gradient-Based Nonlinear Programming

For continuous variable optimization we consider (3-84) without discrete variable y. The general NLP problem (3-85) is presented here  

We consider first methods that find only local solutions to nonconvex problems, as more difficult (and expensive) search procedures are required to find a global solution. Local methods are currently very 

Instead of a formal development of conditions that define a local optimum, we present a more intuitive kinematic illustration. Consider the contour plot of the objective function fix), given in Fig. 3-54, as a smooth valley in space of the variables X and x2. For the contour plot of this unconstrained problem Min/(x), consider a ball rolling in this valley to the lowest point offix), denoted by x. This point is at least a local minimum and is defined by a point with a zero gradient and at least nonnegative curvature in all (nonzero) directions p. We use the first-derivative (gradient) vector Vf(x) and second-derivative (Hessian) matrix V /(x) to state the necessary first- and second-order conditions for unconstrained optimality  

These necessary conditions for local optimality can be strengthened to sufficient conditions by making the inequality in (3-87) strict (i.e., positive curvature in all directions). Equivalently, the sufficient (necessary) curvature conditions can be stated as follows V /(x ) has all positive (nonnegative) eigenvalues and is therefore defined as a positive (semidefinite) definite matrix. 

Now consider the imposition of inequality [g(x) 0] and equality constraints 7i(x) = 0] in Fig. 3-55. Continuing the kinematic interpretation, the inequality constraints g(x) 0 act as fences in the valley, and equality constraints h(x) = 0 act as rails. Consider now a ball, constrained on a rail and within fences, to roll to its lowest point. This stationary point occurs when the normal forces exerted by the fences [- Vg(x )] and rails [- V/i(x )] on the ball are balanced by the force of gravity [— Vfix )]. This condition can be stated by the following Karush-Kuhn-Tucker (KKT) necessary conditions for constrained optimality  

---

The nonlinear programming problem based on objective function (/), model equations (b)-(g), and inequality constraints (was solved using the generalized reduced gradient method presented in Chapter 8. See Setalvad and coworkers (1989) for details on the parameter values used in the optimization calculations, the results of which are presented here. 

Saber, H. M. and A. Ravindran. 1996. A partitioning gradient based (PGB) algorithm for solving nonlinear goal programming problem. Computers and Operations Research. 23 141-152. 

---