Constrained problems

Constrained Derivatives—Equality Constrained Problems Consider minimizing the objective function F written in terms of n variables z and subject to m equahty constraints h z) = 0, or... 

Equality Constrained Problems—Lagrange Multipliers Form a scalar function, called the Lagrange func tion, by adding each of the equality constraints multiplied by an arbitrary iTuiltipher to the objective func tion. 

V L is equal to the constrained derivatives for the problem, which should be zero at the solution to the problem. Also, these stationarity conditions very neatly provide the necessaiy conditions for optimality of an equality-constrained problem. 

Equality- and Inequality-Constrained Problems—Kuhn-Tucker Multipliers Next a point is tested to see if it is an optimum one when there are inequality constraints. The problem is... 

Inequality Constrained Problems To solve inequality constrained problems, a strategy is needed that can decide which of the inequality constraints should be treated as equalities. Once that question is decided, a GRG type of approach can be used to solve the resulting equality constrained problem. Solving can be split into two phases phase 1, where the go is to find a point that is feasible with respec t to the inequality constraints and phase 2, where one seeks the optimum while maintaining feasibility. Phase 1 is often accomphshed by ignoring the objective function and using instead... 

Alternatively p can be seen as a Lagrange multiplier introduced to solve the constrained problem minimize 0prior(x) subject to (/>ml(x) be equal to some... 

There are two general types of optimization problem constrained and unconstrained. Constraints are restrictions placed on the system by physical limitations or perhaps by simple practicality (e.g., economic considerations). In unconstrained optimization problems there are no restrictions. For a given pharmaceutical system one might wish to make the hardest tablet possible. The constrained problem, on the other hand, would be stated make the hardest tablet possible, but it must disintegrate in less than 15 minutes. 

Problem 4.1 is nonlinear if one or more of the functions/, gv...,gm are nonlinear. It is unconstrained if there are no constraint functions g, and no bounds on the jc,., and it is bound-constrained if only the xt are bounded. In linearly constrained problems all constraint functions g, are linear, and the objective/is nonlinear. There are special NLP algorithms and software for unconstrained and bound-constrained problems, and we describe these in Chapters 6 and 8. Methods and software for solving constrained NLPs use many ideas from the unconstrained case. Most modem software can handle nonlinear constraints, and is especially efficient on linearly constrained problems. A linearly constrained problem with a quadratic objective is called a quadratic program (QP). Special methods exist for solving QPs, and these iare often faster than general purpose optimization procedures. 

Neither of the problems illustrated in Figures 4.5 and 4.6 had more than one optimum. It is easy, however, to construct nonlinear programs in which local optima occur. For example, if the objective function / had two minima and at least one was interior to the feasible region, then the constrained problem would have two local minima. Contours of such a function are shown in Figure 4.7. Note that the minimum at the boundary point x1 = 3, x2 = 2 is the global minimum at / = 3 the feasible local minimum in the interior of the constraints is at / = 4. 

The KTC are closely related to the classical Lagrange multiplier results for equality constrained problems. Form the Lagrangian... 

The essential idea of a penalty method of nonlinear programming is to transform a constrained problem into a sequence of unconstrained problems. 

Transformation of a constrained problem to an unconstrained equivalent problem. The contours of the unconstrained penalty function are shown for different values of r. 

GRG Probably most robust of all three methods Versatile—especially good for unconstrained or linearly constrained problems but also works well for nonlinear constraints Once it reaches a feasible solution it remains feasible and then can be stopped at any stage with an improved solution Needs to satisfy equalities at each step of the algorithm... 

Lasdon, L. S. and A. D. Waren. Generalized Reduced Gradient Software for Linearly and Nonlinearly Constrained Problems. Design and Implementation of Optimization Software, H. J. Greenberg, ed., Sijthoff and Noordhoff, Holland (1978), pp. 363-397. 

This weighted sum of absolute values in e(x) was also discussed in Section 8.4 as a way of measuring constraint violations in an exact penalty function. We proceed as we did in that section, eliminating the nonsmooth absolute value function by introducing positive and negative deviation variables dpt and dnt and converting this nonsmooth unconstrained problem into an equivalent smooth constrained problem, which is... 

The method developed for linear constraints is extended to nonlinearly constrained problems. It is based on the idea that the nonlinear constraints linear Taylor series expansion around an estimation of the solution (xi, ut). In general, measurement values are used as initial estimations for the measured process variables. The following linear system of equations is obtained ... 

Lasdon, L. S., and Waren, A. D. (1978). Generalized reduced gradient software for linearly and nonlinearly constrained problems. In Design and Implementation of Optimisation Software (H. Greenberg, ed.), p. 335. Sijthoff, Holland. 

To define the two-stage algorithm we will follow the work of Valko and Vadja (1987), which basically decouples the two problems. Starting from the definition of the general EVM problem in Eq. (9.22), the vectors zi,..., %u minimize the constrained problem at fixed 0, if and only if each Zj, j = 1,2,..., M, is the solution to... 

Obviously, the trivial solution v,=0 (/ = L > n) does not fit our needs and we must search for solutions as a constrained problem in which the solution vector is of constant, yet arbitrary, length. In other words, we become interested in the vector with some criterion of best direction regardless of its magnitude, which we may conveniently take as unity. Let us lump the C/ coefficients into the m x n matrix A and the n coefficients Vj into the vector x , hence... 

Neuman, C. P., and Sen, A., A suboptimal control algorithm for constrained problems using cubic splines, Automatica 9, 601-613 (1973). 

The augmented Lagrange multiplier algorithm finds the energy minimum of the constrained problem with an iterative, three-step procedure ... 

Remark 2 Gordon s theorem has been frequently used in the derivation of optimality conditions of nonlinearly constrained problems. 

Remark 1 The implications of transforming the constrained problem (3.3) into finding the stationary points of the Lagrange function are two-fold (i) the number of variables has increased from n (i.e. the x variables) to n + m + p (i.e. the jc, A and /z variables) and (ii) we need to establish the relation between problem (3.3) and the minimization of the Lagrange function with respect to x for fixed values of the lagrange multipliers. This will be discussed in the duality theory chapter. Note also that we need to identify which of the stationary points of the Lagrange function correspond to the minimum of (3.3). 

Consider the following bilinearly (quadratically) constrained problem... 

---