Nonlinear programming problem Constrained optimization

In this section we present, under the assumption of differentiability, the first-order necessary optimality conditions for the constrained nonlinear programming problem (3.3) as well as the corresponding geometric necessary optimality conditions. 

Constrained Optimization When constraints exist and cannot be eliminated in an optimization problem, more general methods must be employed than those described above, because the unconstrained optimum may correspond to unrealistic values of the operating variables. The general form of a nonlinear programming problem allows for a nonlinear objective function and nonlinear constraints, or... 

Find (a feasible) X which maximizes/mini-mizes the objective function f(X) subject to the given constraints g(X) < > = b. Optimization problems, in which at least one of the functions among objectives and constrains is nonlinear, define a nonlinear optimization problem. This type of problem is the most general one, and all other problems can be considered as special cases of the nonlinear programming problem (Rao 2009). 

For the numerical solution of optimal control problems, there are basically two well-established approaches, the indirect approach, e. g., via the solution of multipoint boxmdary-value problems based on the necessary conditions of optimal control theory, and the direct approach via the solution of constrained nonlinear programming problems based on discretizations of the control and/or the state variables. The application of an indirect method is not advisable if the equations are too complicated or a moderate accuracy of the numerical solution is commensurate with the model accuracy. Therefore, the easier-to-handle direct approach has been chosen here. Direct collocation methods, see, e. g., Stryk [6], as well as direct multiple shooting methods, see, e. g., Bock and Plitt [1], belong to this approach. In view of forthcoming large scale problems, we will focus here on the direct multiple shooting method, since only the control variables have to be discretized for this method. This leads to lower dimensional nonlinear programming problems. 

Most practical multivariable problems include constraints, which must be treated using enhancements of unconstrained optimization algorithms. The next two sections describe two classes of constrained optimization techniques that are used extensively in the process industries. When constraints are an important part of an optimization problem, constrained techniques must be employed, because an unconstrained method might produce an optimum that violates the constraints, leading to unrealistic values of the process variables. The general form of an optimization problem includes a nonlinear objective function (profit) and nonlinear constraints and is called a nonlinear programming problem. 

A quadratic programming problem minimizes a quadratic function of n variables subject to m linear inequality or equality constraints. A convex QP is the simplest form of a nonlinear programming problem with inequality constraints. A number of practical optimization problems are naturally posed as a QP problem, such as constrained least squares and some model predictive control problems. 

Linear and nonlinear programming solvers have been interfaced to spreadsheet software for desktop computers. The spreadsheet has become a popular user interface for entering and manipulating numeric data. Spreadsheet software increasingly incorporates analytic tools that are accessible from the spreadsheet interface and permit access to external databases. For example, Microsoft Excel incorporates an optimization-based routine called Solver that operates on the values and formulas of a spreadsheet model. Current versions (4.0 and later) include LP and NLP solvers and mixed integer programming (MIP) capability for both linear and nonlinear problems. The user specifies a set of cell addresses to be independently adjusted (the decision variables), a set of formula cells whose values are to be constrained (the constraints), and a formula cell designated as the optimization objective. 

In addition to a wide variety of problem types, there are three common types of constrained optimization problems that are typically of interest linear programs (LPs), quadratic programs (QPs), and nonlinear programs (NLPs). 

The problem of portfolio selection is easily expressed numerically as a constrained optimization maximize economic criterion subject to constraint on available capital. This is a form of the knapsack problem, which can be formulated as a mixed-integer linear program (MILP), as long as the project sizes are fixed. (If not, then it becomes a mixed-integer nonlinear program.) In practice, numerical methods are very rarely used for portfolio selection, as many of the strategic factors considered are difficult to quantify and relate to the economic objective function. 

Chapter 13 illustrates the problem of constrained optimization by introducing the active set methods. Successive linear programming (SLP), projection, reduced direction search, SQP methods are described, implemented, and adopted to solve several practical examples of constrained linear/nonlinear optimization, including the solution of the Maratos effect. 

A great number of studies indicated that quadratic approximation methods, which are characterized by solving a sequence of quadratic subproblems recursively belong to the most efficient and reliable nonlinear programming algorithms presently available. This method combines the most efficient characteristics of different optimization techniques (see e.g, [19]). For equality constrained problems, the general nonlinear constrained optimization problem can be formulated by an... 

---