Unconstrained program

Optimization of Unconstrained Olnective Assume the objective Func tion F is a function of independent variables i = r. A computer program, given the values for the independent variables, can calculate F and its derivatives with respect to each Uj. Assume that F is well approximated as an as-yet-unknown quadratic function in u. 

Constrained Optimization When constraints exist and cannot be eliminated in an optimization problem, more general methods must be employed than those described above, since the unconstrained optimum may correspond to unrealistic values of the operating variables. The general form of a nonhuear programming problem allows for a nonlinear objec tive function and nonlinear constraints, or... 

Nonlinear Programming The most general case for optimization occurs when both the objective function and constraints are nonlinear, a case referred to as nonlinear programming. While the idea behind the search methods used for unconstrained multivariable problems are applicable, the presence of constraints complicates the solution procedure. 

Gill, P.E. and W. Murray, "Newton-type Methods for Unconstrained and Linearly Constrained Optimization", Mathematical Programming, 7,311-350 (1974). 

Problem 4.1 is nonlinear if one or more of the functions/, gv...,gm are nonlinear. It is unconstrained if there are no constraint functions g, and no bounds on the jc,., and it is bound-constrained if only the xt are bounded. In linearly constrained problems all constraint functions g, are linear, and the objective/is nonlinear. There are special NLP algorithms and software for unconstrained and bound-constrained problems, and we describe these in Chapters 6 and 8. Methods and software for solving constrained NLPs use many ideas from the unconstrained case. Most modem software can handle nonlinear constraints, and is especially efficient on linearly constrained problems. A linearly constrained problem with a quadratic objective is called a quadratic program (QP). Special methods exist for solving QPs, and these iare often faster than general purpose optimization procedures. 

In nonlinear programming problems, optimal solutions need not occur at vertices and can occur at points with positive degrees of freedom. It is possible to have no active constraints at a solution, for example in unconstrained problems. We consider nonlinear problems with constraints in Chapter 8. 

The essential idea of a penalty method of nonlinear programming is to transform a constrained problem into a sequence of unconstrained problems. 

Unconstrained u(k) A is varied using a one-dimensional search (external to the MPC program) to find a good response that satisfies the input constraints in step 2. 

Dunn, J. C., and Bertsekas, D., Efficient dynamic programming implementations of Newton s method for unconstrained optimal control problems, J. Opt. Tkeo. Applies. 63(1), 23 (1989). 

Equations (4.40) and (4.41) are easily implemented in an existing least-squares program and give both the constrained and the unconstrained results in a single refinement cycle. However, the method fails if the unconstrained refinement corresponds to a singular matrix, as would be the case, for example, if all population parameters, including those of the core functions, were to be refined in addition to the scale factor k. 

It is easy to modify the program to solve the problem assuming 57. and 10 /. errors in observations. Results are summarized in Table 1.2. The table also includes the unconstrained least squares estimates of x, i.e., the values minimizing (1.86) with n = 3 and m = 20. This latter result was obtained by inserting the appropriate data into the main program of Section 3.2. ... 

A. V. Fiacco and G. R McCormick. Nonlinear Programming Sequential Unconstrained Minimization Techniques. Society for Industrial and Applied Mathematics, 1968. 

Another useful program (E04HAA) provides constrained optimization with bounds for each parameter using a sequential penalty function technique, which effectively operates around unconstrained minimization cycles. 

Programming Problems by Unconstrained Optimization", Math.Programming, 1973, 5 (3), 354-373. 

R. B. Schnabel, in Mathematical Programming, M. Iri and K. Tanabe, Eds., pp. 227-261, Kluwer Academic, Dordrecht, 1989. Sequential and Parallel Methods for Unconstrained Optimization. 

A.V. Fiacco and G.P. McCormik, Non-Linear Programming Sequential Unconstrained Optimization Technique, Wiley, New York. 

---