Variance-covariance ellipsoid

The value of Eq. (4.32) is that it shows exactly which features of the structure are well determined and which are poorly determined in the fitting procedure. For the two-dimensional example of Fig. 4.1, the eigenparameters correspond to the principal axes of the variance-covariance ellipsoid in the figure. In general, they define the principal axes of the hyper-ellipsoid in w-dimensional parameter space which represents the variance-covariance matrix. 

The standard way to answer the above question would be to compute the probability distribution of the parameter and, from it, to compute, for example, the 95% confidence region on the parameter estimate obtained. We would, in other words, find a set of values h such that the probability that we are correct in asserting that the true value 0 of the parameter lies in 7e is 95%. If we assumed that the parameter estimates are at least approximately normally distributed around the true parameter value (which is asymptotically true in the case of least squares under some mild regularity assumptions), then it would be sufficient to know the parameter dispersion (variance-covariance matrix) in order to be able to compute approximate ellipsoidal confidence regions. 

As stated earlier, LDA requires that the variance-covariance matrices of the classes being considered can be pooled. This is only so when these matrices can be considered to be equal, in the same way that variances can only be pooled, when they are considered equal (see Section 2.1.4.4). Equal variance-covariance means that the 95% confidence ellipsoids have an equal volume (variance) and orientation in space (covariance). Figure 33.10 illustrates situations of unequal variance or covariance. Clearly, Fig. 33.1 displays unequal variance-covariance, so that one must expect that QDA gives better classification, as is indeed the case (Fig. 33.2). When the number of objects is smaller than the number of variables m, the variance-covariance matrix is singular. Clearly, this problem is more severe for QDA (which requires m < n ) than for LDA, where the variance-covariance matrix is pooled and therefore the number of objects N is the sum of all objects... 

In the rare case where the parameters are uncorrelated, the matrix (Af. Y) is diagonal, the axes of the confidence ellipsoid would be parallel to the coordinates of the parameter space, and the individual parameter confidence intervals would hold for each parameter independently. However, since the parameters are usually correlated, the extent of the correlation can be measured from the correlation coefficient matrix, R. This is obtained by applying Eq. (7.34) to the variance-covariance matrix (7.135) ... 

---