The regularized Newton method

The regularized Newton method can be obtained if we consider again only one iteration, 

Setting = 0 in (5.85), we find at once the regularized normal equation for the optimum step, 

The last equation allows us to construct the algorithm of the regularized Newton method  

3 Approximate regularized solution of the nonlinear inverse problem We can find an approximate solution of the regularized normal equation (5.86) for the optimum step, using the same idea which we applied for the approximate. solution of the linear inverse problem in Chapter 3. Let us assume that the regularization parameter tv is big enough to neglect the term with respect to the term 

Applying the inverse weighting operatois (IT 14 ) to both side of the last equation, we find  

---

Algorithmic Details for NLP Methods All the above NLP methods incorporate concepts from the Newton-Raphson method for equation solving. Essential features of these methods are that they rovide (1) accurate derivative information to solve for the KKT con-itions, (2) stabilization strategies to promote convergence of the Newton-like method from poor starting points, and (3) regularization of the Jacobian matrix in Newton s method (the so-called KKT matrix) if it becomes singular or ill-conditioned. 

We can apply different types of minimization methods (steepest descent, Newton method, etc.) to the straightforward minimization of the corresponding misfit functional /(m). However, all these solutions have many limitations and are very sensitive to small variations of the observed data due to the principal instability of the inverse problem. To overcome this difficulty, we have to apply a regularizing method. 

Numerical schemes of the Newton method for nonlinear regularized least-squaret... 

Instead of simply changing the value of x in regular increments until the function y approaches zero, the Newton-Rayhson method uses the slope of the function at an initial estimate of the root, x, to obtain an improved estimate of the root, X2- Figure 10-5 illustrates a chart of the function y = 3x + 2.5x -5x-11, between x = 0 and x = 6. The value x = 5 has been chosen as an initial estimate for finding a root of the function. Here s how an improved estimate for the root is calculated. The slope of the curve at x is the first derivative of the function, dyfdx. The improved estimate of the root is given by X2 = xi - (yi/slope). The process is repeated until no appreciable change in x occurs. 

---