Variance inflation factors

The first simple diagnostic is to examine the correlation matrix of the covariates. High correlations, either positive or negative, are indicative of collinearity. However, the correlation matrix is sometimes unable to detect the situation where three or more covariates are collinear but no two correlations are high (Belsley, Kuh, and Welsch, 1980). Related to the inverse of the correlation matrix are variance inflation factors (VIF), calculated as... 

Table 3.22 showed that the only factorial designs allowing estimation of all first-order interactions are 2 , 2 2, 2, 2, 2 , and 2 . Rechtschaffner designs may be used instead, for 4, 6 or 7 factors, to give less expensive estimates of the effects. For A = 8 a D-optimal design is probably better. See also the discussion of these designs in terms of variance inflation factors (VIF) in chapter 4, section IV. 

Variance inflation factors (3, 4, 5, 6) provide a simpler, overall measure of the non-independence of the estimators, that is, the co-linearity of the variables of the model. 

Although the designs have low variance inflation factors, they are not rotatable, often showing considerable deviations from the property. Also as a general rule the saturated or almost saturated designs show considerable differences in the variance of prediction over the domain. The maximum prediction variance function in the (hyper-)cube can be high (d > 1.8). 

If the experimenter requires a more certain or more precise identification of the optimum, or to know how the response or responses vary about the final point, he should not try to obtain a mathematical model at the end of the optimization by multi-linear regression over the points (not even limiting the points to those about the optimum). The variance inflation factors are often very high, a measure of the lack of reliability of the regression. Rather, he should carry out a response surface study around the supposed optimum. 

The two designs for N = 12 have the exactly the same characteristics in I X X I, I M I, rr(X X) , and but they are not equivalent. It is not possible to obtain one from the other by permutating lines and columns. TTiey also differ in their variance inflation factors, which are slightly more constant in the D-optimal than the % design. However, they are very close and there is no reason to choose one rather than another on the basis of these properties. 

A more formal and often used approach to measuring correlation between predictor variables is the variance inflation factor (VIF) value. It is computed as... 

Several techniques have been proposed for detecting intercorrelation. These include examination of the off-diagonal elements of X X, the examination of eigenvalues of X X, the use of principal components, and the use of variance inflation factors (VIFs). Where VIFs are the diagonal elements of the (X X) matrix. Most current regression software will display these VIFs. 

The coefficient of determination of each predictor variable j regressed on all the other predictor variables exceeds 0.90, or the variance inflation factor VIFj = (1 — R j) is less than 10 (variable j is colinear with one or more of the other predictors). VIFj is the j,j)th diagonal element of the matrix V V where V — [zij]. R j can be computed from the relationship between Rj and VIFj. 

Adj adjusted LL lower limit (95% Cl), S square root of the mean square error (MSE) of the fit, UL upper limit (95% Cl), VIF variance inflation factor. 

---