Vector algebra orthonormality

In the previous section we have developed principal components analysis (PCA) from the fundamental theorem of singular value decomposition (SVD). In particular we have shown by means of eq. (31.1) how an nxp rectangular data matrix X can be decomposed into an nxr orthonormal matrix of row-latent vectors U, a pxr orthonormal matrix of column-latent vectors V and an rxr diagonal matrix of latent values A. Now we focus on the geometrical interpretation of this algebraic decomposition. 

In the previous section we saw that, in spite of appearances, we do not need to know the angle between two vectors in order to evaluate the scalar product according to equation (5.8) we simply exploit the properties of the orthonormal base vectors to evaluate the result algebraically. However, we can approach from a different perspective, and use the right side of equation (5.8) to find the angle between two vectors, having evaluated the scalar product using the approach detailed above. The next Worked Problem details how this is accomplished. 

Our development thus far is quite similar to the one we used in Subsection 1.1.4, where we discussed iV-dimensional vector spaces. Indeed, the theory of complete orthonormal functions can be regarded as a generalization of ordinary linear algebra. To make the analogy explicit, it is convenient to introduce the shorthand notation... 

---