Variance of prediction

As is the case for PRESS, the variance of prediction can be calculated for predictions made on independent validation sets as well as predictions made on the data set which was used to generate the calibration. 

The Standard Error of Prediction (SEP) is supposed to refer uniquely to those situations when a calibration is generated with one data set and evaluated for its predictive performance with an independent data set. Unfortunately, there are times when the term SEP is wrongly applied to the errors in predicting y variables of the same data set which was used to generate the calibration. Thus, when we encounter the term SEP, it is important to examine the context in order to verify that the term is being used correctly. SEP is simply the square root of the Variance of Prediction, s2. The RMSEP (see below) is sometimes wrongly called the SEP. Fortunately, the difference between the two is usually negligible. 

Although the derivation is beyond the scope of this presentation, it can be shown that the estimated variance of predicting a single new value of response at a given point in factor space, is equal to the purely experimental uncertainty variance, plus the variance of estimating the mean response at that point, 5, 0 that is. 

For a given experimental design (such as that of Equation 11.15), the variance of predicting the mean response at a point in factor space is... 

The standard uncertainty in the single new value of response (the square root of the estimated variance of predicting the response, analogous to the standard uncertainty of a parameter estimate defined in Section 6.2) is... 

The most important among the known criteria of design optimality is the requirement of D- and G-optimality. A design is said to be D-optimal when it minimizes the volume of the scatter ellipsoid for estimates of regression equation coefficients. The property of G-optimality provides the least maximum variance of predicted response values in a region under investigation. 

For updating the values of the inverse matrix, the determinant, and the variances of prediction, they used the following formulas ... 

In the fourth column of this table, we find the estimates of the maximum variance of prediction, calculated for each of the designs. For instance, the value d = 1.4 is the maximum variance of prediction for the design consisting of points 1 to 7, whereas the value d = 0.4446 corresponds to the design consisting of points 1 to 14. The confidence interval for the predicted value of the response is given by... 

When a model is fitted to experimental data, the experimental error is transmitted to the coefficients of the modeL These parameters will therefore have a probability distribution. When the model is then used to predict the response for a given combination of the experimental variables, Xj — ( ii> 2i —prediction is also afflicted by the probability distribution of the model parameteis. We will have a certain degree of uncertainty in the predictions. There will be a variance of the predicted response, KO/ ). The variance of prediction is described by the following relation K(y ) = Zi (X X)-izj < = d.. ... 

For the cases where the inverse of X X does not exist or if X X is ill-conditioned (that is, X X is nearly singular), there is always a numerical solution to Equations (3.27) and (3.29). However, this does not mean that this solution is always desirable from a statistical or practical point of view. Specifically, the estimated regression vector b tends to be uncertain because the solution is mostly governed by the noise part of the data. This can lead to high variances of predicted y values for new samples or objects. [Belsley et at. 1980],... 

For the composite design there is a value of a for which the variance of prediction of the response depends only on the distance from the centre and not on the direction in which one moves from the centre. The isovariance lines or surfaces are spherical. This property is known as rotatability. 

A completely constant variance of prediction is not possible. Mathematically it can be shown that the closest to uniform precision can be obtained for = 5. However, if we compare the isovariance curves (figure 5.9) for Aq = 5 with those for = 3, we see that the difference may not be great enough to justify carrying out the extra experiments if these are very costly. (Note that the variance function may be transformed to a relative standard deviation by taking the square root.) On the other hand a single experiment at the centre appears to be insufficient. 

Although the designs have low variance inflation factors, they are not rotatable, often showing considerable deviations from the property. Also as a general rule the saturated or almost saturated designs show considerable differences in the variance of prediction over the domain. The maximum prediction variance function in the (hyper-)cube can be high (d > 1.8). 

Fig. 5.25 Normalised estimates of first-order contributions to the overall variance of predicted butane mole fraction at 750 K calculated using first-order local sensitivities (grey) and the global HDMR method (black). Both are derived from a model describing the oxidation of n-butane in a jet stirred reactor (residence time of 6 s, atmospheric pressure, stoichiometric mixtures containing 4 % (mol) -butane diluted in helium). EXGAS notation is used. Adapted with permission from Cord et al. (2012). Copyright (2012) American Chemical Society...

The first-order second moment method (FOSM) is the method adopted within the framework to propagate input parameter uncertainty through numerical models (26, 27). FOSM provides two moments, mean and variance of predicted variables. This method is based on Taylor series expansion, of which second-order and higher terms are truncated. The expected value of concentration, E[u] and its covariance, COV[u] are (25, 27),... 

---