Jacobians and Hessians

The definitions of the Jacobian and Hessian functional determinants were given in 29.1. Some examples of their use may be given here. 

Determinants, Jacobians and Hessians are continually appearing in different branches of applied mathematics. The following results will serve as a simple exercise on the mathematical methods of some of the earlier sections of this work. The reader should find no difficulty in assigning a meaning to most of the coefficients considered. See J. E. Trevor, Journ. Phys. Chem.,-3, 523, 573, 1899 10, 99, 1906 also R. E. Baynes Thermodynamics, Oxford, 95, 1878. 

Linear constraints must be managed separately from nonlinear constraints so as to skip the calculation of their Jacobian and Hessian. 

Finally, the information relating to which variables are nonlinear within the nonlinear equality and inequality constraints must be provided. It further reduces the number of variable groups to be simultaneously modified to calculate the Jacobians and Hessians ofthe two systems ofconstraints, also limiting the number of elements to be updated. 

A novel gradient-based optimisation framework for large-scale steady-state input/output simulators is presented. The method uses only low-dimensional Jacobian and reduced Hessian matrices calculated through on-line model-reduction techniques. The typically low-dimensional dominant system subspaces are adaptively computed using efficient subspace iterations. The corresponding low-dimensional Jacobians are constructed through a few numerical perturbations. Reduced Hessian matrices are computed numerically from a 2-step projection, firstly onto the dominant system subspace and secondly onto the subspace of the (few) degrees of freedom. The tubular reactor which is known to exhibit a rich parametric behaviour is used as an illustrative example. 

In an object from the BzzConstrainedMinimization class with sparse systems, two BzzVectorint objects, nlH and nlP, are used to indicate which variables are really nonlinear in the nonlinear equality or inequality constraints. They are necessary to efficiently update the Jacobians and the Hessians of the functions of constraints. 

Let us be more specific and consider the special case of a closed-shell system. The approximate Jacobian and the exact Hessian are then given by the expressions (10.9.21) and (10.9.22) ... 

Now, since the Hessian is the second derivative matrix, it is real and symmetric, and therefore hermitian. Thus, all its eigenvalues are real, and it is positive definite if all its eigenvalues are positive. We find that minimization amounts to finding a solution to g(x)=0 in a region where the Hessian is positive definite. Convergence properties of iterative methods to solve this equation have earlier been studied in terms of the Jacobian. We now find that for this type of problems the Jacobian is in fact a Hessian matrix. 

The iteration counter k and the argument x(l) refers to the macroiterations made in the Newton-Raphson procedure, and they are obviously constant within the context of this section. Let us drop them for convenience. Also, let us explicitly assume that the Jacobian is in fact a positive definite Hessian, and that f(x< >) is a gradient The equation to be solved is thus rewritten in the form... 

To solve the equations of problem P ), the optimization algorithms used are Levenberg-Marquardt and Trust-Region procedures. These methods enable computation of the solution by using the Jacobian matrix and the Hessian matrix (or its approximation) related to the objective function E(Y) [57]. 

A novel model reduction-based optimization framework for input/output steady state simulators has been presented. It can be considered as an extension of Reduced Hessian methods. Reduced Hessians are efficiently computed through a double-step reduction procedure first onto the dominant system subspace, adaptively computed through subspace iterations, and second onto the subspace of the decision variables. Only low-order Jacobians need to be computed through a few numerical perturbations, using only... 

The Jacobian of the system (7.10) is the Hessian of the function to minimize in the case of optimization it is a symmetric matrix and it is positive definite when the minimum is strong. This is no longer true in the case of generic systems (7.1). 

Finally, the Hessian matrix of the function (7.64) is symmetric (and positive definite, if the Jacobian) is nonsingular) and equal to the product j7j . 

If the Jacobian is well conditioned then the correction d is achieved by solving the system (7.38) instead of the system (7.59). Conversely, the use of the function (7.65) becomes interesting when the Jacobian matrix is quite ill-conditioned or even singular. In fact, the system matrix jJj is the Hessian of the function (7.65) and it is symmetric. 

The BzzMatrixSparseLocked class manages the structure of the linear and nonlinear constraints. The objects in this class allow the calculation of the Jacobian of nonlinear constraints and, when necessary, their Hessians, useful to building the Hessian of the Lagrange function they are indispensable in efficiently solving the appropriate KKT system. 

Nonlinear constraints may include linear and nonlinear terms. Only the nonlinear ones enter the calculations of the Hessians or in a new evaluation of the Jacobians. 

The structure of the functions that appears in the nonlinear equality and inequality constraints system must be provided. This allows the calculation or updating of the Jacobian of the equations and the Hessians of each function of constraints by grouping the variables in sets that can be simultaneously modified. 

CC2 linear response theory as well as SOPPA, in Section 10.3, are in principle both second-order response function methods, although there are significant differences. Nevertheless, one can compare the blocks of elements in the CC2 Jacobian with the corresponding matrices in the SOPPA Hessian. The block in CC2 and the... 

In nonlinear programming (NLP) problems, either the objective function, the constraints, or both the objective and the constraints are nonlinear. Unlike LP, NLP solution does not always lie at the vertex of the feasible region. NLP optimum lies where the Jacobean of the function obtained by combining constraints with the objective function (using Lagrange multiphers as follows) is zero. The solution is local minimum if the Jacobian J is zero and the Hessian H is positive definite, and it is a local maximum if J is zero and H is negative definite. 

At K = 0, the zero-order GBT vector is ielectronic gradient E< > (10.1.29), whereas the Jacobian (i.e. the first-derivative matrix) W< > diftes from the electronic Hessian E<2> (10.1.30) in that the nested commutator is not symmetrized. From Section 10.2.1, we conclude that and E are identical at stationary points at nonstationary points, they differ in terms that are proportional to the GBT vector. 

---