Explained sum of squares

The regression equation passes through the point ( x), (y)), where (x) and (y) are the means of the dependent and independent variables, respectively. The quality of a simple linear regression equation is often reported as the squared correlation coefficient, or value. This indicates the fraction of the total variation in the dependent variables that is explained by the regression equation. To determine r, the total sum of squares (TBS) of the deviations of the observed y values from the mean (y) is calculated together with the explained sum of squares (ESS), which is the sum of squares of the deviations of the y values calculated from the model, from the mean ... 

A nonlinear regression routine is applied to iteratively adjust the polynomial coefficients of Ca(f)g ess and minimize the difference between 7j,(f) and /flt (f).The explained sum of squares (ESS) should be calculated as... 

This sum of squares is made up from two components the variance in y that is explained by the regression equation (known as the explained sum of squares, ESS), and the residual or unexplained sum of squares, RSS. The ESS is given by a comparison of the predicted y values (y) with the mean... 

These sums of squares are shown in the analysis of variance (ANOVA) table (Table 6.2). The mean squares are obtained by division of the sums of squares by the appropriate degrees of freedom. One degree of freedom is lost with each parameter calculated from a set of data so the total sum of squares has n — 1 degrees of freedom (where n is the number of data points) due to calculation of the mean. The residual sum of squares has —2 degrees of freedom due to calculation of the mean and the slope of the line. The explained sum of squares has one degree of freedom corresponding to the slope of the regression line. 

Knowledge of the mean squares and degree of freedom allows assessment of the significance of a regression equation as described in the next section, but how can we assess how well the line fits the data Perhaps the best known and most misused regression statistic is the correlation coefficient. The squared correlation coefficient (r ) is given by division of the explained sum of squares by the total sum of squares... 

FIGURE 9.6 Loading weight wj and loading pj, corresponding tali j = 0.34 and an explained sum of squares... 

After preprocessing of a raw data matrix, one proceeds to extract the structural features from the corresponding patterns of points in the two dual spaces as is explained in Chapters 31 and 32. These features are contained in the matrices of sums of squares and cross-products, or cross-product matrices for short, which result from multiplying a matrix X (or X ) with its transpose ... 

The least squares criterion states that the norm of the error between observed and predicted (dependent) measurements 11 y - yl I must be minimal. Note that the latter condition involves the minimization of a sum of squares, from which the unknown elements of the vector b can be determined, as is explained in Chapter 10. 

Hence is the fraction of the total sum of squares (or inertia) c of the data X that is accounted for by v,. The sum of squares (or inertia) of the projections upon a certain axis is also proportional to the variance of these projections, when the mean value (or sum) of these projections is zero. In data analysis we can assign different masses (or weights) to individual points. This is the case in correspondence factor analysis which is explained in Chapter 32, but for the moment we assume that all masses are identical and equal to one. 

Finally, a measure of lack of fit using a PCs can be defined using the sum of the squared errors (SSE) from the test set, flSSETEST = Latest 2 (prediction sum of squares). Here, 2 stands for the sum of squared matrix elements. This measure can be related to the overall sum of squares of the data from the test set, SStest = -Xtest 2- The quotient of both measures is between 0 and 1. Subtraction from 1 gives a measure of the quality of fit or explained variance for a fixed number of a PCs ... 

This measure can be related to the sum of squared elements of the columns of X to obtain a proportion of unexplained variance for each variable. Subtraction from 1 results in a measure Qj of explained variance for each variable using a PCs... 

The data were modeled by a principal components model with three components. The statistical results method (25. 31) are presented in Table IV and V. In addition, the measured total PCB concentration is included in Table IV. One of the three sets of two-dimension plots (Theta 1 vs Theta 2) is presented in Figure 10. Individual samples of a given Aroclor were distributed regularly in these plots and samples were ordered according to concentration. The sums of squares decreased from 4,360 to 52.4 (Table V.) and approximately 88 percent of the standard deviation was explained by the three term component model. 

It is important to realize that an / or r value (instead of an or value) might give a false sense of how well the factors explain the data. For example, the R value of 0.956 arises because the factors explain 91.4% of the sum of squares corrected for the mean. An R value of 0.60 indicates that only 36% of 55 has been explained by the factors. Although most regression analysis programs will supply both R (or r) and R (or r ) values, researchers seem to prefer to report the coefficients of correlation R and r) simply because they are numerically larger and make the fit of the model look better. 

Bias corrections are sometimes applied to MLEs (which often have some bias) or other estimates (as explained in the following section, [mean] bias occurs when the mean of the sampling distribution does not equal the parameter to be estimated). A simple bootstrap approach can be used to correct the bias of any estimate (Efron and Tibshirani 1993). A particularly important situation where it is not conventional to use the true MLE is in estimating the variance of a normal distribution. The conventional formula for the sample variance can be written as = SSR/(n - 1) where SSR denotes the sum of squared residuals (observed values, minus mean value) is an unbiased estimator of the variance, whether the data are from a normal distribution... 

It is important to realize that R might give a false sense of how well the factors explain the data. For example, the R value of 0.956 arises because the factors explain 91.4% of the sum of squares corrected for the mean. An R value of 0.60 indicates that only 36% of SScori has been explained by the factors. 

The total variance in a data matrix A is the sum of the diagonal elements in ATA or AAT (also called the trace of ATA or the trace of Z). This total sum of squares represents the total amount of variability in the original data. The magnitude of the eigenvalues is directly proportional to the amount of variation explained by a corresponding principal component. In fact, the sum of all of the eigenvalues is equal to the trace of Z. 

All of the statistical figures of merit used for judging the quality of least-squares fits are based upon the fundamental relationship shown in Equation 5.15, which describes how the total sum of squares is partitioned into two parts (1) the sum of squares explained by the regression and (2) the residual sum of squares, where y is the mean concentration value for the calibration samples. 

Box and Meyer also derived a useful result (which is applied in some of the subsequent methods in this chapter) that relates dispersion effects to location effects in regular 2k p designs. We present the result first for 2k designs and then explain how to extend it to fractional factorial designs. First, fit a fully saturated regression model, which includes all main effects and all possible interactions. Let /3, denote the estimated regression coefficient associated with contrast i in the saturated model. Based on the results, determine a location model for the data that is, decide which of the are needed to describe real location effects. We now compute the Box-Meyer statistic associated with contrast j from the coefficients 0, that are not in the location model. Let i o u denote the contrast obtained by elementwise multiplication of the columns of +1 s and—1 s for contrasts i and u. The n regression coefficients from the saturated model can be decomposed into n/2 pairs such that for each pair, the associated contrasts satisfy i o u = j that is, contrast i o u is identical to contrast j . Then Box and Meyer proved that equivalent expressions for the sums of squares SS(j+) and SS(j-) in their dispersion statistic are... 

The resulting equation, correlation index R2, and error about the regression line are shown in Figure 3. The correlation index, R2 = 0.77, indicates that 77% of the sum of squares is explained by the model in Equation 1. Two other possible variables that might have contributed... 

We then calculate the Sums of Squares of y explained by the two independent variables independently of each other. These are... 

Then the sum of squares explained by the addition of the second independent variable Xa is the difference between the sum of squares explained by Xw and Xa together and the sum of squares explained by Xw alone, i.e. 

The degrees of freedom are such that each regression coefficient takes 1 and the residual takes what is left. Similarly the Residual sum of squares is what is left out of the Total after the total explained by the two regression coefficients has been subtracted. 

This is a part of the total variance explained by the regression model as opposed to the residual sum of squares RSS. Moreover, a reference quantity is the Total Sum of Squares, TSS, defined as the sum of the squared differences between the experimental responses and the average response ... 

Coefficient of determination, Bf. The squared multiple correlation coefficient that is the percent of total variance of the response explained by a regression model. It can be calculated from the model sum of squares MSS or from the residual sum of squares RSS ... 

When the average centred variable matrix (X — X) is used, the matrix (X - X) (X - X) contains the sums of squares and the crossproducts of the variables. Since the means of each variable have been subtracted, the elements in (X - X) (X - X) are related to the variances and the covariances of the variables over the set of N compounds. The total sum of squares is equal to the sum of the eigenvalues. The variation described by a component is proportional to the sum of squares associated with this component, and this sum of squares is equal to the corresponding eigenvalue. It is usually expressed as a percentage of the total sum of squares and is often called "explained variance", although this entity is not corrected for the number of degrees of freedom. Percent "explained variance" by component "j" is therefore obtained as follows ... 

The next step, is to leave out part of the data from the residual matrix and compute the next component from the truncated data matrix. The left-out data can then be predicted from the expanded model and the error of prediction fir - ir ir s determined. This is repeated over and over again, until all elements in E have been left out once and only once. The prediction enor sum of squares, PRESS = 22yij, is computed from the estimated errors of prediction. If it should be found that PRESS exceeds the residual sum of squares RSS calculated firom the smaller model, the new component does not improve the prediction and is considered to be insignificant. A ratio PRESS/RSS > 1 implies that the new component predicts more noise than it explains the variation. 

---