Second Derivative Methods The Newton-Raphson Method

5 Second Derivative Methods The Newton-Raphson Method 

Second-order methods use not only the first derivatives (i.e. the gradients) but also the second derivatives to locate a minimum. Second derivatives provide information about the curvature of the function. The Newton-Raphson method is the simplest second-order method. Recall our Taylor series expansion about the point r. . Equation (5.2)  

If the function is purely quadratic, the second derivative is the same everywhere, and so 

In practice, of course, the surface is only quadratic to a first approximation and so a number of steps will be required, at each of which the Hessian matrix must be calculated and inverted. The Hessian matrix of second derivatives must be positive definite in a Newton-Raphson minimisation. A positive definite matrix is one for which all the eigenvalues are positive. When the Hessian matrix is not positive definite then the Newton-Raphson method moves to points (e.g. saddle points) where the energy increases. In addition, far from a mimmum the harmonic approximation is not appropriate and the minimisation can become unstable. One solution to this problem is to use a more robust method to get near to the minimum (i.e. where the Hessian is positive definite) before applying the Newton-Raphson method. 

---