The minimal residual method

The general iterative solution of equation (4.6) can be expressed by the formula 

The func tioii T (A ) can be expre.ssed using the inner ])roduct operation in the llill)ert space M  

Difi eretitiating tlic fuiictiori (Ay) with resi)OCt to the iteration coefficient Ay, we can write an cciuation, realizing thc minimum condition (1.12)  

From thc last formula wo obtain immediately an ex])ression for the value of k that minimizes [r, 11  

According to (4.14) the residual on the (n + l)-th iteration is always less than or equal to the residual on the previous iteration  

---

We confine ourselves here to the minimal residual method and the method of steepest descent relating to two-layer schemes. As usual, the explicit scheme is considered first ... 

Tj, 2/(7i -b 72), thereby justifying estimate (17) and the convergence of the minimal residual method with the same rate as occurred before for the simple iteration method with the exact values 71 and... 

No doubt, several conclusions can be drawn from such reasoning. First, the method being employed above converges in the space Ha with the same rate as the simple iteration method although it occurs in one of the subordinate norms. Second, the minimal residual method converges in the space Ha, that is, in a more stronger norm. 

The minimal residual method can be employed for the equation Au = / with a non-self-adjoint operator A, the convergence rate of which coincides with that of scheme (30) for r = f. 

This is certainly true for the minimal residual method (13)-(14) under conditions (34). Here is a solution of problem (13) and p is specified by... 

Based on formulae (4.22) and (4.23), we sec that the minimal residual method converges if L is a positively determined (PD) liii( ar continuous operator, acting in a real Hilbert s[)ac( A/. Actually, t.he following imi)ortant theorem holds. 

Thus, we conclude that the sequence of elements m , generated by the minimal residual method, is a Cauchy sequence, because the distance between any two elements goes to zero, m — m —> 0, as f,n oo (see Appendix A, section A.2). Since the Hilbert space M is a complete linear space, the Cauchy sequence m converges to the element m G M m — in, if n —+ oo. 

Theorem 19 Let L be an absolutely positively determined (APD) linear continuous operator, acting in a complex Hilbert space M. Then the solution of the linear operator equation (4-6) exists and is unique in M, and the minimal residual method, based on the recursive formulae (4-V and (4-13), converges to this solution for any initial approximation mo... 

Note, in conclusion, that one can estimate also the convergence rate of the minimal residual method based on formula (4.22) for the residuals ... 

Thus, the Euler equation has a unique solution, ma, which can be obtained by the minimal residual method, MRM, or by the generalized MRM. We noted in the beginning of this section that the solution of the minimization problem (4.99) is also unique. Thus, we can conclude that it is equal to mo. In other words, we have proved that minimization of the Tikhonov parametric functional (4.99) is equivalent to the solution of the corresponding Euler equation (4.100). 

This result comes from the simple fact that, if we minimize a functional along some direction, described by a parametric line, the direction of the steepest ascent must be perpendicular to this line at the minimum point on the line (see Figure 5-3) otherwise we would still not be reaching the minimum along this line. A formal proof of this result was presented in Chapter 4 for the linear operator A, when we discussed the minimal residual method for the linear inverse problem solution (formula (4.51)). 

---