Multivariate normal

The conceptually simplest model, which for reasons explained later is called UNEQ, is based on the multivariate normal distribution. Suppose we have carried... 

We also make a distinction between parametric and non-parametric techniques. In the parametric techniques such as linear discriminant analysis, UNEQ and SIMCA, statistical parameters of the distribution of the objects are used in the derivation of the decision function (almost always a multivariate normal distribution... 

The Mahalanobis distance representation will help us to have a more general look at discriminant analysis. The multivariate normal distribution for w variables and class K can be described by... 

Thus, we see that CCA forms a canonical analysis, namely a decomposition of each data set into a set of mutually orthogonal components. A similar type of decomposition is at the heart of many types of multivariate analysis, e.g. PCA and PLS. Under the assumption of multivariate normality for both populations the canonical correlations can be tested for significance [6]. Retaining only the significant canonical correlations may allow for a considerable dimension reduction. 

We have assumed that the prior information can be described by the multivariate normal distribution, i.e., k is normally distributed with mean kB and co-variance matrix VB. 

We will begin with the concept of the multivariate normal distribution. 

Figure 1-1 Development of the concept of the Multivariate Normal Distribution (this one shown having three dimensions) - see text for details. The density of points along a cross-section of the distribution in any direction is also an MND, of lower dimension.

Baxter, M.J. and Gale, N.H. (1998). Testing for multivariate normality via univariate tests a case study using lead isotope ratio data. Journal of Applied Statistics 25 671-683. 

Vector e has a multivariate normal distribution. Mah and Tamhane (1982) proposed the use of the test on the estimates,... 

As was indicated in Section 7.2, the vector of measurement adjustments, e, has a multivariate normal distribution with zero mean and covariance matrix V. Thus, the objective function value of the least square estimation problem (7.21), ofv = eT l> 1 e, has a central chi-square distribution with a number of degrees of freedom equal to the rank of A. 

A multivariate normal distribution data set with the variance and mean given by this i and x was generated by the Monte Carlo method to simulate the process sampling data. The data size was 1000 and it was used to investigate the performance of the indirect method. 

The confidence intervals defined for a single random variable become confidence regions for jointly distributed random variables. In the case of a multivariate normal distribution, the equation of the surface limiting the confidence region of the mean vector will now be shown to be an n-dimensional ellipsoid. Let us assume that X is a vector of n normally distributed variables with mean n-column vector p and covariance matrix Ex. A sample of m observations has a mean vector x and an n x n covariance matrix S. 

We assume that a random variable vector Y of (here upper-case is used to indicate not a matrix but an ordered set of m random variables) distributed as a multivariate normal distribution has been measured through an adequate analytical protocol (e.g., CaO concentration, the 87Sr/86Sr ratio,...). The outcome of this measurement is the data vector jm. Here ym is the mean of a large number of measurements with expected... 

We now proceed to m observations. The ith observation provides the estimates xi of the independent variables Xj and the estimate y, of the dependent variable Y. The n estimates xtj of the variables Xj provided by this ith observation are lumped together into the vector xt. We assume that the set of the (n+1) data (i/,y,) associated with the ith observation represent unbiased estimates of the mean ( yf) of a random (n + 1)-vector distributed as a multivariate normal distribution. The unbiased character of the estimates is equivalent to... 

In Sections 1.6.3 and 1.6.4, different possibilities were mentioned for estimating the central value and the spread, respectively, of the underlying data distribution. Also in the context of covariance and correlation, we assume an underlying distribution, but now this distribution is no longer univariate but multivariate, for instance a multivariate normal distribution. The covariance matrix X mentioned above expresses the covariance structure of the underlying—unknown—distribution. Now, we can measure n observations (objects) on all m variables, and we assume that these are random samples from the underlying population. The observations are represented as rows in the data matrix X(n x m) with n objects and m variables. The task is then to estimate the covariance matrix from the observed data X. Naturally, there exist several possibilities for estimating X (Table 2.2). The choice should depend on the distribution and quality of the data at hand. If the data follow a multivariate normal distribution, the classical covariance measure (which is the basis for the Pearson correlation) is the best choice. If the data distribution is skewed, one could either transform them to more symmetry and apply the classical methods, or alternatively... 

If it can be assumed that the multivariate data follow a multivariate normal distribution with a certain mean and covariance matrix, then it can be shown that the squared Mahalanobis distance approximately follows a chi-square distribution... 

If the data majority is multivariate normally distributed, the squared score distances can be approximated by a chi-square distribution, y2, with a degrees of freedom. 

Note that the approximation by the chi-square distribution is only possible for multivariate normally distributed data which somehow is in conflict if outliers are present that should be identified with this measure. We recommend that robust PCA is used whenever diagnostics is done because robust methods tolerate deviations from multivariate normal distribution. 

Outliers may heavily influence the result of PCA. Diagnostic plots help to find outliers (leverage points and orthogonal outliers) falling outside the hyper-ellipsoid which defines the PCA model. Essential is the use of robust methods that are tolerant against deviations from multivariate normal distributions. 

The canonical correlation coefficients can also be used for hypothesis testing. The most important test is a test for uncorrelatedness of the x- and y-variables. This corresponds to testing the null hypothesis that the theoretical covariance matrix between the x- and y-variables is a zero matrix (of dimension mx x mY). Under the assumption of multivariate normal distribution, the test statistic... 

For the Bayesian discriminant rule, an underlying data distribution fj for each group j =l,..., k is required, which is usually assumed to be a multivariate normal... 

Maximizing the posterior probabilities in case of multivariate normal densities will result in quadratic or linear discriminant rules. However, the mles are linear if we use the additional assumption that the covariance matrices of all groups are equal, i.e., X = = Xk=X- In this case, the classification rule is based on linear discriminant scores dj for groups j... 

The approach of Fisher (1938) was originally proposed for discriminating two populations (binary classification), and later on extended to the case of more than two groups (Rao 1948). Here we will first describe the case of two groups, and then extend to the more general case. Although this method also leads to linear functions for classification, it does not explicitly require multivariate normal distributions of the groups with equal covariance matrices. However, if these assumptions are not... 

If the assumptions (multivariate normal distributions with equal group covariance matrices) are fulfilled, the Fisher rule gives the same result as the Bayesian rule. However, there is an interesting aspect for the Fisher rule in the context of visualization, because this formulation allows for dimension reduction. By projecting the data... 

Although model-based clustering seems to be restrictive to elliptical cluster forms resulting from models of multivariate normal distributions, this method has several advantages. Model-based clustering does not require the choice of a distance measure, nor the choice of a cluster validity measure because the BIC measure can be... 

Model-based clustering assumes that each cluster can be modeled by a multivariate normal distribution (with varying parameters). If the clusters can be well modeled in this way, the method is powerful, and can estimate an optimum number of clusters. Especially for higher-dimensional data it is computer time demanding. 

Such a measure of the separation between classes will work best when It can be assumed that the classes approximate multivariate normal distributions. That Is a reasonable assumption for the classes modeled by the output of the FCV algorithms. 

Parametric techniques based on the multivariate normal (MVN) distribution are particularly well developed. Parameters of the MVN distribution include a covariance or correlation for each pair of variables, as well as a mean and variance for each variable. 

LDA is the first classification technique introduced into multivariate analysis by Fisher (1936). It is a probabilistic parametric technique, that is, it is based on the estimation of multivariate probability density fimc-tions, which are entirely described by a minimum number of parameters means, variances, and covariances, like in the case of the well-knovm univariate normal distribution. LDA is based on the hypotheses that the probability density distributions are multivariate normal and that the dispersion is the same for all the categories. This means that the variance-covariance matrix is the same for all of the categories, while the centroids are different (different location). In the case of two variables, the probability density fimction is bell-shaped and its elliptic section lines correspond to equal probability density values and to the same Mahala-nobis distance from the centroid (see Fig. 2.15A). 

---