Newton-Type Optimization Algorithms

Newton-type algorithms have been applied to process optimization over the past decade and have been well studied. In this section we provide a concise development of these methods, as well as their extension to handle large-scale problems. First, however, we consider the following, rather general, optimization problem  

Necessary conditions for a local solution to (2) are given by the following Kuhn-Tucker conditions  

Applying Newton s method at iteration k to this set of equations leads to 

This form is convenient in that the active inequality constraints can now be replaced in the QP by all of the inequalities, with the result that Sa is determined directly from the QP solution. Finally, since second derivatives may often be hard to calculate and a unique solution is desired for the QP problem, the Hessian matrix, is approximated by a positive definite matrix, B, which is constructed by a quasi-Newton formula and requires only first-derivative information. Thus, the Newton-type derivation for (2) leads to a nonlinear programming algorithm based on the successive solution of the following QP subproblem  

Finally, a great advantage to SQP is that it does not require convergence of the equality constraints, h(x) = 0, at intermediate points. Consequently, the process model (or at least the part directly incorporated into the optimization problem) can be solved simultaneously with the optimization problem. In the next section we discuss the application of the SQP algorithm to flowsheet optimization. Here, if the number of variables in the optimization problem is small, application is straightforward. On the other hand, when the number of variables, n, becomes large (n 100, say), special-purpose extensions to SQP are required. These are discussed in the remainder of this section. 

---

In the restrained optimization scheme, the SCF algorithm is employed with a fixed multiplier until convergence is achieved and subsequently the multiplier is updated, which is in our implementation (134) realized by a Newton-type optimization algorithm when employing the second derivatives of C with respect to Xa as... 

Unlike parameter optimization, the optimal control problem has degrees of freedom that increase linearly with the number of finite elements. Here, for problems with many finite elements, the decomposition strategy for SQP becomes less efficient. As an alternative, we discussed the application of Newton-type algorithms for unconstrained optimal control problems. Through the application of Riccati-like transformations, as well as parallel solvers for banded matrices, these problems can be solved very efficiently. However, the efficient solution of large optimal control problems with... 

Newton-type. Finally, we come to those algorithms which depend on a knowledge of A and A l (the Newton-type algorithms). If we are dealing with quadratic functions, then once we know A l it follows immediately from equation (22) that we can reach the minimum in just one step, so that we need not trouble about directions of descent. However, if the function is not quadratic, then the problem of optimal directions again becomes... 

HyperChem supplies three types of optimizers or algorithms steepest descent, conjugate gradient (Hetcher-Reeves and Polak-Ribiere), and block diagonal (Newton-Raphson). 

---