Orthogonal and Orthonormal Matrices

Matrices with exclusively orthogonal column (or row) vectors are called orthogonal matrices. For any two columns x and x-j of a matrix X to be orthogonal the necessary condition states that their scalar product is zero. 

In our three dimensional world we can perceive three vectors to be orthogonal. However, in a higher dimensional space the set of equations defined by (2.15) must suffice. 

If columns (or rows) of X are normalised to the square root of the sum of their squared elements (i.e. to unity length), the matrix is called orthonormal. Recall that earlier this kind of normalisation was solved most elegantly by right (left) multiplication with a diagonal matrix comprising the appropriate normalisation coefficients. See the section introducing diagonal matrices for more details. 

Alternatively, Matlab s built-in function norm can be used to determine normalisation coefficients and perform the same task. An example for column-wise normalisation of a matrix X with orthogonal columns is given below. It is worthwhile to compare X with equation (2.15) the subspace command can be used to determine the angle between the vectors (in rad) and reconfirm orthogonality.  

Note that this kind of normalisation, via the norm function, can only be performed column- (or row-) wise via a loop as seen in the Matlab box above. Calling norm with one matrix argument determines a different kind of normalisation coefficients. We refer to the Matlab help and function references for more detail. 

---

Now the expression Normal Equations starts to make sense. The residual vector r is normal to the grey plane and thus normal to both vectors f ,i and f , 2 As outlined earlier, in Chapter Orthogonal and Orthonormal Matrices (p.25), for orthogonal (normal) vectors the scalar product is zero. Thus, the scalar product between each column of F and vector r is zero. The system of equations corresponding to this statement is ... 

SVD is completely automatic. It is one of the most stable algorithms available and thus can be used blindly. It is one command in Matlab [U, S, Vt]=svd (Y, 0). The matrices U and V1 contain as columns so-called eigenvectors. They are orthonormal (see Orthogonal and Orthonormal Matrices, p.25) which means that the products... 

---