Newton-Raphson approach

The Newton-Raphson approach, being essentially a point-slope method, converges most rapidly for near linear objective functions. Thus it is helpful to note that tends to vary as 1/P and as exp(l/T). For bubble-point-temperature calculation, we can define an objective function... 

We can treat the problem of locating the limit cycle as that of finding a zero of this new function, i.e. of finding x0 such that Ax(x0) = 0. Of the various algorithms which can be used for locating zeros of functions, a Newton-Raphson approach is quite suitable. Thus, having taken an estimate of the appropriate initial value xj>, which gives rise to a non-zero difference Ax, we may obtain an improved value x 0+1 from the formula... 

The most prominent of these methods is probably the second order Newton-Raphson approach, where the energy is expanded as a Taylor series in the variational parameters. The expansion is truncated at second order, and updated values of the parameters are obtained by solving the Newton-Raphson linear equation system. This is the standard optimization method and most other methods can be treated as modifications of it. We shall therefore discuss the Newton-Raphson approach in more detail than the alternative methods. 

Several attempts have been made to devise simpler optimization methods than the lull second order Newton-Raphson approach. Some are approximations of the full method, like the unfolded two-step procedure, mentioned in the preceding section. Others avoid the construction of the Hessian in every iteration by means of update procedures. An entirely different strategy is used in the so called Super - Cl method. Here the approach is to reach the optimal MCSCF wave function by annihilating the singly excited configurations (the Brillouin states) in an iterative procedure. This method will be described below and its relation to the Newton-Raphson method will be illuminated. The method will first be described in the unfolded two-step form. The extension to a folded one-step procedure will be indicated, but not carried out in detail. We therefore assume that every MCSCF iteration starts by solving the secular problem (4 39) with the consequence that the MC reference state does not... 

The super-CI method can alternatively be given in a folded form, which includes the coupling between the Cl and orbital rotations. This is done by adding the complementary Cl space, IK>, to the super-CI secular problem. As in the Newton-Raphson approach, it is more efficient to transform the equations back to the original CSF space, and thus work with a super-CI consisting of the Cl basis states plus the SX states. It is left to the reader as an exercise to construct the corresponding secular equation and compare it with the folded one-step Newton-Raphson equations (4 22). 

Computationally the super-CI method is more complicated to work with than the Newton-Raphson approach. The major reason is that the matrix d is more complicated than the Hessian matrix c. Some of the matrix elements of d will contain up to fourth order density matrix elements for a general MCSCF wave function. In the CASSCF case only third order term remain, since rotations between the active orbitals can be excluded. Besides, if an unfolded procedure is used, where the Cl problem is solved to convergence in each iteration, the highest order terms cancel out. In this case up to third order density matrix elements will be present in the matrix elements of d in the general case. Thus super-CI does not represent any simplification compared to the Newton-Raphson method. 

Class II Methods. The methods of Class II are those that use the simultaneous Newton-Raphson approach, in which all the equations are linearized by a first order Taylor series expansion about some estimate of the primitive variables. In its most general form, this expansion includes terms arising from the dependence of the thermo-physical property models on the primitive variables. The resulting system of linear equations is solved for a set of iteration variable corrections, which are then applied to obtain a new estimate. This procedure is repeated until the magnitudes of the corrections are sufficiently small. 

The Newton-Raphson approach is another minimization method.f It is assumed that the energy surface near the minimum can be described by a quadratic function. In the Newton-Raphson procedure the second derivative or F matrix needs to be inverted and is then usedto determine the new atomic coordinates. F matrix inversion makes the Newton-Raphson method computationally demanding. Simplifying approximations for the F matrix inversion have been helpful. In the MM2 program, a modified block diagonal Newton-Raphson procedure is incorporated, whereas a full Newton-Raphson method is available in MM3 and MM4. The use of the full Newton-Raphson method is necessary for the calculation of vibrational spectra. Many commercially available packages offer a variety of methods for geometry optimization. 

In a straightforward application of the Newton-Raphson approach, Eq. (32) is solved iteratively for S and T until the convergence criteria (26) are fulfilled to the desired accuracy. This process converges nicely if the initial choice of the orbitals and the Cl coefficients are close to the final result. The energy is then in the local region where the second-order approximation is valid. Obviously such situations will not be very common in actual applications. In practice the starting orbitals are often obtained from a preceding SCF calculation, or even estimated from atomic densities , while the Cl... 

The Newton-Raphson method is often used to solve problems involving a single variable, and is implemented in Excel as Tools => Goal Seek. The method requires that a function Fix) can be formulated as an explicit mathematical expression in terms of a variable x. We now want to know for what value of x the function F(x) has a particular value, A. The Newton-Raphson approach then searches for a value of x for which Fix) is equal to A. Often one selects A = 0, in which case the corresponding value of x is called a root of the function F x). 

Newton-Raphson approach, as opposed to other methods, such as an expectation-maximization approach, is that the matrix of second derivatives of the objective function, evaluated at the optima, is immediately available. By denoting this matrix, H, 2H, is an asymptotic variance-covariance matrix of the estimated parameters G and R. Another method to estimate G and R is the noniterative MIVQUEO method. Using Monte Carlo simulation, Swallow and Monahan (1984) have shown that REML and ML are better estimators than MIVQUEO, although MIVQUEO is better when REML and ML methods fail to converge. 

The Newton-Raphson approach to solving Eqs. (2.16) and (2.17) begins by defining... 

This is solved using a Newton-Raphson approach obtained by assuming that at Q + 6Q, Eq. (13) is satisfied, that is... 

Although the calculation of the Hessian is time consuming, the effort is quickly compensated by the excellent convergence properties of the Newton-Raphson approach [293-295]. This optimization technique solved the convergence problems of first-order MCSCF methods, which optimized orbitals and Cl coefficients in an alternating manner (recall chapter 9). Even perturbative improvements of the four-component CASSCF wave function are feasible and have been implemented and investigated [527]. 

The use of the Newton-Raphson algorithm permits a linear multi-grid approach to the solution of the Reynolds equation since this technique Involves linearisation as part of the iterative procedure. The application of a Newton-Raphson approach using multi-grids Is covered briefly in [5]. 

One of the most cited representatives of the Newton methods is the Newton-Raphson approach, for which the PES function V(/J) is expanded at a point Ro clo.se to the stationary point / sp by a Taylor series expansion ... 

---