The Newton-Raphson Method

Therefore, with any starting point in that neighborhood, the iteration process will converge to the solution a. 

There is no need to find the interval [a, b] as in the bisection method. 

There is no guarantee of convergence (the contracting mapping theorem must be applied, but it is conservative). 

A rather slow convergence rate (linear convergence) persists. 

The Newton-Raphson method is one of the most effective methods to solve nonlinear algebraic equations. We first illustrate the method with a single equation, and then generalize it to coupled algebraic equations. 

Similar information to that obtained dining a conjugate gradient refinement is obtainable from second derivatives (curvature)1 R 51 93. For a harmonic function the gradient (linear matrix of first derivatives, [A]) multiplied by the curvature (Hessian matrix of second derivatives, [C]) should lead directly to the shifts (AX) to be applied in order to move toward the minimum (Eq. 3.3). 

In the block-diagonal Newton-Raphson minimization, the generally small size of the off-diagonal terms is exploited and the matrix describing the curvature is reduced to N 3 3 matrices, i.e., to 9N elements (Fig. 3.8). Due to the approximations 

In the block-diagonal Newton-Raphson minimization the generally small size of the off-diagonal terms is exploited and the matrix describing the curvature is re- 

The best known, and possibly the most widely used, technique for locating roots of nonlinear equations is the Newton-Raphson method. This method is based on a Taylor series expansion of the nonlinear function/(x) around an initial estimate (x,) of the root  

Because what is being sought is the value of that forces the function/(x) to assume zero value, the left side of Eq. (1.40) is set to zero, and the resulting equation is solved for x. 

However, the right-hand side is an infinite series. Therefore, a finite number of terms must be retained and the remaining terms must be truncated. Retaining only the first two terms on the right-hand side of the Taylor series is equivalent to linearizing the function f x). This operation results in 

Because the Taylor series was truncated, retaining only two terms, the new value x will not yet satisfy Eq. (1,10). We will designate this value as Xj and reapply the Taylor series lineari zati on at (shown in Fig. 1.5 to obtain Xj. Repetitive application ofthis step converts Eq. (1.41) to an iterative formula  

In contrast to the method of linear interpolation discussed in Sec. 1.5, the Newton-Raphson method uses the newly found position as the starting point for each subsequent iteration. 

Since Ax can be made very small, the slope calculated this way will be an excellent approximation to the true slope. The Newton-Raphson method usually converges rapidly. 

If a function has more than one real root, the particular root to which the Newton-Raphson method converges will depend on the initial estimate chosen. Thus, to obtain a particular root, some guidance must be provided by the user. 

Similar information to that obtained during a conjugate-gradient refinement is obtainable from second derivatives (curvature) [13, 94, 232-235]. For a harmonic 

---

An alternative, and closely related, approach is the augmented Hessian method [25]. The basic idea is to interpolate between the steepest descent method far from the minimum, and the Newton-Raphson method close to the minimum. This is done by adding to the Hessian a constant shift matrix which depends on the magnitude of the gradient. Far from the solution the gradient is large and, consequently, so is the shift d. One... 

Second Derivative Methods The Newton-Raphson Method... 

Xk) is the inverse Hessian matrix of second derivatives, which, in the Newton-Raphson method, must therefore be inverted. This cem be computationally demanding for systems u ith many atoms and can also require a significant amount of storage. The Newton-Uaphson method is thus more suited to small molecules (usually less than 100 atoms or so). For a purely quadratic function the Newton-Raphson method finds the rniriimum in one step from any point on the surface, as we will now show for our function f x,y) =x + 2/. 

The unknowns in this equation are the local coordinates of the foot (i.e. and 7]). After insertion of the global coordinates of the foot found at step 6 in the left-hand side, and the global coordinates of the nodal points in a given element in the right-hand side of this equation, it is solved using the Newton-Raphson method. If the foot is actually inside the selected element then for a quadrilateral element its local coordinates must be between -1 and +1 (a suitable criteria should be used in other types of elements). If the search is not successful then another element is selected and the procedure is repeated. 

Generalizing the Newton-Raphson method of optimization (Chapter 1) to a surface in many dimensions, the function to be optimized is expanded about the many-dimensional position vector of a point xq... 

Numerical Derivatives The results given above can be used to obtain numerical derivatives when solving problems on the computer, in particular for the Newton-Raphson method and homotopy methods. Suppose one has a program, subroutine, or other function evaluation device that will calculate/given x. One can estimate the value of the first derivative at Xq using... 

This equation must be solved for y The Newton-Raphson method can be used, and if convergence is not achieved within a few iterations, the time step can be reduced and the step repeated. In actuality, the higher-order backward-difference Gear methods are used in DASSL(Ref. 224). 

The sets of equations can he solved using the Newton-Raphson method. The first form of the derivative gives a tridiagonal system of equations, and the standard routines for solving tridiagonal equations suffice. For the other two options, some manipulation is necessary to put them into a tridiagonal form (see Ref. 105). 

This represents a set of nonlinear algebraic equations that can he solved with the Newton-Raphson method. However, in this case, a viable iterative strategy is to evaluate the transport coefficients at the last value and then solve... 

Given values fori7(/c), solve equations h(x, 17) = 0 for x(/c). These will be m equations in m unknowns. If the equations are nonlinear, solving can be done using a variant of the Newton-Raphson method. 

Simultaneous solution by the Newton-Raphson method yields x = 0.9121, y = 0.6328. Accordingly, the fractional compositions are ... 

There are several reasons that Newton-Raphson minimization is rarely used in mac-romolecular studies. First, the highly nonquadratic macromolecular energy surface, which is characterized by a multitude of local minima, is unsuitable for the Newton-Raphson method. In such cases it is inefficient, at times even pathological, in behavior. It is, however, sometimes used to complete the minimization of a structure that was already minimized by another method. In such cases it is assumed that the starting point is close enough to the real minimum to justify the quadratic approximation. Second, the need to recalculate the Hessian matrix at every iteration makes this algorithm computationally expensive. Third, it is necessary to invert the second derivative matrix at every step, a difficult task for large systems. 

Equation 5-197 is a polynomial of the third degree, and by employing either a numerieal method or a spreadsheet paekage sueh as Mierosoft Exeel, the roots (C ) of the equation ean be determined. A developed eomputer program PROGS 1 using the Newton-Raphson method to determine was used. The Newton-Raphson method for the roots of Equation 5-197 is... 

Using the Newton-Raphson method for solving the nonlinear system of Equations 5-253 gives... 

Equation 13-39 is a cubic equation in terms of the larger aspect ratio R2. It can be solved by a numerical method, using the Newton-Raphson method (Appendix D) with a suitable guess value for R2. Alternatively, a trigonometric solution may be used. The algorithm for computing R2 with the trigonometric solution is as follows ... 

By iteration, the general expression for the Newton Raphson method may be written (if f can be evaluated and is continuous near the root) ... 

This is the Newton-Raphson method. It may be advantageous to take, however,... 

It is instructive to note that both the steepest-descent and the Newton-Raphson methods lead in the direction of —VU however, the steepest-descent method is unable to tell us how far to go in each step and therefore we have to search for the minimum in a very ineffective way (see Fig. 4.3). 

FIGURE 4.3. Illustrating the effectiveness of different minimization schemes. The steepest-deicent method requires many steps to reach the minimum, while the Newton-Raphson method locates the minimum in a few steps (at the expense, however, of evaluating the second derivative matrix). 

Exercise 4.3. Use the Newton-Raphson method to minimize the system in Problem 4.1. 

The Newton-Raphson method has been applied to pipeline network problems since 1954 (Wl). Its performance has been generally very good, although convergence difficulties have been reported (S2), when starting from inappropriate initial guesses. In some cases large oscillations around... 

Unlike the other alternative methods, analytical expressions of partial derivatives are required and the Jacobian must be evaluated in the Newton-Raphson method. These requirements sometimes prove to be the undoing when the method is applied to complicated equations. Brown (B12) has developed a modification to the Newton-Raphson method, which requires only some of the partial derivatives to be calculated. We have tested Brown s method on our sample problems but have found that it actually required more computing time than the unmodified Newton-Raphson method. 

For the special case for which n = 2, it can be shown that the linearization method defined above becomes identical to the Newton-Raphson method. The result may be generalized to apply to any homogeneous function of degree n. 

While it is technically erroneous to claim that the linearization method does not require any initialization (J2), it is true that the initialization procedure used appear to be quite effective. A more comprehensive discussion of initialization procedure will be given in Section III,A,5. With this initialization procedure, the linearization method appears to converge very rapidly, usually in less than 10 iterations for formulations A and B. Since the evaluation of f(x) and its partial derivatives is not required, the method is also simpler and easier to implement than the Newton-Raphson method. 

Finally, for formulation D the flows in the tree branches can be computed sequentially assuming zero chord flows. This initialization procedure was used by Epp and Fowler (E2) who claimed that it led to fast convergence using the Newton-Raphson Method. 

If the Newton-Raphson method is used to solve Eq. (1), the Jacobian matrix (df/3x)u is already available. The computation of the sensitivity matrix amounts to solving the same Eq. (59) with m different right-hand side vectors which form the columns — (3f/<5u)x. Notice that only the partial derivatives with respect to those external variables subject to actual changes in values need be included in the m right-hand sides. 

The computational procedure can now be explained with reference to Fig. 19. Starting from points Pt and P2, Eqs. (134) and (135) hold true along the c+ characteristic curve and Eqs. (136) and (137) hold true along the c characteristic curve. At the intersection P3 both sets of equations apply and hence they may be solved simultaneously to yield p and W for the new point. To determine the conditions at the boundary, Eq. (135) is applied with the downstream boundary condition, and Eq. (137) is applied with the upstream boundary condition. It goes without saying that in the numerical procedure Eqs. (135) and (137) will be replaced by finite difference equations. The Newton-Raphson method is recommended by Streeter and Wylie (S6) for solving the nonlinear simultaneous equations. In the specified-time-... 

This equation must be solved for yn +l. The Newton-Raphson method can be used, and if convergence is not achieved within a few iterations, the time step can be reduced and the step repeated. In actuality, the higher-order backward-difference Gear methods are used in DASSL [Ascher, U. M., and L. R. Petzold, Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations, SIAM, Philadelphia (1998) and Brenan, K. E., S. L. Campbell, and L. R. Petzold, Numerical Solution of Initial-Value Problems in Differential-Algebraic Equations, North Holland Elsevier (1989)]. 

---