A some PCA results

In PCA the weights equal the loadings. This can be shown as follows. Let USV be the SVD of X. Then wi = vi and ti = s ui. The loadings pi can be found by solving the regression problem X - tipi 2. This results in 

If the additional constraint is added that Q = I then the columns of T are orthogonal to each other. The first column in T has the highest possible variance the second column has the highest variance under the additional constraint of being orthogonal to the first column, and so on. This gives the principal components of the variance of approach. 

Instead of using the constraint W W = I, it is also possible to apply the constraint that T T is diagonal in Equation (3.43). This results in both W W and P P being diagonal. If 

For Q = I, the solutions of Equation (3.43) with increasing R are nested the rank R solution contains the rank — 1 solution with an extra component (the 7 th component) added. This is easily seen from Equation (3.44) K = kfi], where is the rth eigenvector of XX. 

The scores T and loadings P can always be transformed to T = TZ and P = P(Z 1) without changing the fit of X in Equation (3.43). If T and P resulted from Equation (3.44) then column orthogonality of P still holds after orthogonal transformation (Z Z = ZZ = I and P P = I), whereas the column orthogonality of T is destroyed (see later). This also shows that the constraint W W = I is not active with respect to the fit. 

---