Estimating the Jacobian and quasi-Newton methods

With Newton s method, to update our estimate of the solution we need to evaluate at the 

In these methods, we use not the exact value of the Jacobian matrix, but rather some approximation 

We then compute the update by solving the Unear system 

The use of such an approximation does not change the value of the solution that we obtain, as at a solution jCs, f xs) = 0 and Ax = (51 1) /(x ) = 0 no matter the value of 5l l (provided that it is nonsingular). This allows some freedom in the choice of. 61 1 to balance accuracy in the approximation of the true Jacobian against computational efficiency. 

The most straight forward approach to approximate the Jacobian is to use a finite difference approximation 

---