Raphson algorithm, Newton

This method, first introduced by Isaac Newton and better formulated in the actual form by Joseph Raphson, is the simplest second-order algorithm. The basic idea is to use a quadratic approximation to the objective function around the initial parameter estimate and, then, to adjust the parameters in order to minimize the quadratic approximation until their values converge. 

The function U = U(ft) can be approximated by the second-order Taylor series expansion at the point i.e., 

By setting the derivative of the function (3.36) equal to zero yields 

For nonquadratic objective functions, (3.39) can be used to obtain an iterative estimation law with fixed step size, k 0, 

The direction given by —H(0s) lVU 0s) is a descent direction only when the Hessian matrix is positive definite. For this reason, the Newton-Raphson algorithm is less robust than the steepest descent method hence, it does not guarantee the convergence toward a local minimum. On the other hand, when the Hessian matrix is positive definite, and in particular in a neighborhood of the minimum, the algorithm converges much faster than the first-order methods. 

---

The Newton-Raphson algorithm is further developed into a fairly generally applicable tool for the solving of sets of non-linear equations. 

Figure 3-11. A flow diagram for the Newton-Raphson algorithm.

The Newton-Raphson algorithm we have developed can deal with any equilibrium situation. There is no limit to the number of components or species. The disadvantage is that the computations are iterative and this is clearly unsuitable for Excel applications. While it is possible to resolve complicated equilibria, it is inconvenient for complete titrations, as only one cell at a time can be evaluated. However, there are important special cases that can be solved explicitly. We deal with one here. 

For the 1-dimensional case it is possible to represent the basic ideas graphically. This allows natural understanding and good insight into the potential shortfalls of the Newton-Raphson algorithm. We have chosen a truly irrational function... 

Figure 3-23. The Newton-Raphson algorithm with an unfortunate initial guess.

As done previously, in The Newton-Raphson Algorithm (p.48), we neglect all but the first two terms in the expansion. This leaves us with an approximation that is not very accurate but, since it is a linear equation, is easy to deal with. Algorithms that include additional higher terms in the Taylor expansion, often result in fewer iterations but require longer computation times due to the calculation of higher order derivatives. 

The central function Rcalc EqAH2. m computes the residuals and is again very similar to the ones we developed earlier. First, the total concentrations are recalculated this needs to be part of the calculation of the residuals, as we want to be able to fit initial concentrations (s. c 0) as well. Subsequently these total concentrations are passed to the Newton-Raphson function NewtonRaphson, m in order to calculate all species concentrations, see The Newton-Raphson Algorithm (p.48). The differences between measured and calculated pH define the residuals. Note that any variable used in this function to calculate the residuals can theoretically be a parameter to be fitted to the data. 

Using familiar methods of the calculus of variations, one can set the first variation of the energy with respect to the orbitals and linear coefficients to zero. This leads to a set of Fock-like operators, one for each orbital. Gerratt, et al use a second-order stabilized Newton-Raphson algorithm for the optimization. This gives a set of occupied and virtual orbitals from each Fock operator as well as optimum <7,-s. 

Improved Newton-Raphson algorithm with approximate... 

---