Convergence rate superlinear

In the above equation, the norm is usually the Euclidean norm. We have a linear convergence rate when 0 is equal to 1. Superlinear convergence rate refers to the case where 0=1 and the limit is equal to zero. When 0=2 the convergence rate is called quadratic. In general, the value of 0 depends on the algorithm while the value of the limit depends upon the function that is being minimized. 

At least superlinear convergence rates as the solution is approached. 

As you can see from the third and fourth columns in the table the rate of convergence of Newton s method is superlinear (and in fact quadratic) for this function. 

Also, when the constraints are linear, the method always remains in the subspace of the constraints (for a feasible starting point) and dy remains zero. Consequently, for this case the Z BY matrix is unimportant and only the n-m)x n- m) Z BZ matrix needs to be supplied or calculated. While this is not true for nonlinear constraints, it is still convenient to deal only with Z BZ and set Z BY to zero. Moreover, as long as dy remains relatively small, a reasonable superlinear rate of convergence can still be maintained for this decomposition (Nocedal and Overton, 1985). 

Alternatively, so-called secant methods can be used to approximate the Jacobian matrix with far less effort (Westerberg et al., 1979). These provide a superlinear rate of convergence that is, they reduce the errors less rapidly than the Newton-Raphson method, but more rapidly than the method of successive substitutions, which has a linear rate of convergence (i.e., the length of the error vector is reduced from 0.1, 0.01, 10 , 10 , 10 , ...). These methods are also referred to as quasi-Newton methods, with Broyden s method being the most popular. 

---