Prediction intervals

To perform the maximization over (X,t), we need an algorithm such as the Nelder-Mead simplex search (14). An alternative that is adequate in many cases is a simple search over a (X,t) grid. The critical value XX has an interpretation of its own. It is the upper bound on a simultaneous prediction interval for ng as yet unobserved observations from the background population. 

The value has an assodated uncertainty that generally spedfies a range within which the true value is expected to lie at a level of confidence of approximately 95 % if the sample is homogeneous or the uncertainty represents a prediction interval within which the true values of 95 % of all samples are expected to lie at a stated level of confidence if significant sample heterogeneity is included. 

A form of this approach has long been followed by RT Corporation in the USA. In their certification of soils, sediments and waste materials they give a certified value, a normal confidence interval and a prediction interval . A rigorous statistical process is employed, based on that first described by Kadafar (1982,), to produce the two intervals the prediction interval (PI) and the confidence interval (Cl). The prediction interval is a wider range than the confidence interval. The analyst should expect results to fall 19 times out of 20 into the prediction interval. In real-world QC procedures, the PI value is of value where Shewhart (1931) charts are used and batch, daily, or weekly QC values are recorded see Section 4.1. Provided the recorded value falls inside the PI 95 % of the time, the method can be considered to be in control. So occasional abnormal results, where the accumulated uncertainty of the analytical procedure cause an outher value, need no longer cause concern. 

Mention has already been made of the EPA recommended use of both confidence interval and prediction interval. However, many users and their customers may be satisfied by some simplistic comparisons. Two methods for comparing experimental results with certified values are presented here. The users are referred to statistical handbooks for comparing results of sets of determinations such as using/test, etc. 

Fig. 6.3. Three-dimensional model of calibration, analytical evaluation and recovery spatial model (A) the three relevant planes are given separately in (B) as the calibration function with confidence interval, in (C) as the recovery function with confidence interval, and in (C) as the evaluation function with prediction interval (D)... 

Fundamentally, the uncertainties of measured values y estimated by calibration, e.g. according to Eq. (6.6), on the one hand and of analytical results x (analyte contents, concentrations) estimated by means of a calibration function, e.g. according to Eq. (6.17), on the other hand differ from one another as can be seen from Fig. 6.3B,C, and Fig. 6.7. Whereas the uncertainty of y values in calibration is characterized by the confidence interval cnf(y), the uncertainty of estimated x values is characterized by the prediction interval prd(x). 

Prediction interval of a single value yprj at position x,... 

Prediction interval of a mean xprd from N repetitions for a measured value y... 

Precision. The precision of the calibration is characterized by the confidence interval cnffyf of the estimated y values at position x, according to Eq. (6.30). In contrast, the precision of analysis is expressed by the prediction intervals prd(y ) and prd(x,), respectively, according to Eqs. (6.32) and (6.33). The precision of analytical results on the basis of experimental calibration is closely related to the adequacy of the calibration model. 

The reliability of multispecies analysis has to be validated according to the usual criteria selectivity, accuracy (trueness) and precision, confidence and prediction intervals and, calculated from these, multivariate critical values and limits of detection. In multivariate calibration collinearities of variables caused by correlated concentrations in calibration samples should be avoided. Therefore, the composition of the calibration mixtures should not be varied randomly but by principles of experimental design (Deming and Morgan [1993] Morgan [1991]). 

Precision of an analytical result (x) is expressed by the prediction interval which include not only the dispersion of the measured results but additionally the uncertainty of the calibration on which the estimation of x is based see Sect. 6.2.2. 

Results of ultra trace analyses are sometimes characterized by relatively high uncertainties up to more than 100%. In such cases it is not allowed that the lower uncertainty limit falls below zero. Results like, e.g., (0.07 0.10) must be replaced by such as (0.07 + 0.10/ — 0.07) or (0.07/q o°), respectively. That means, the total uncertainty interval (confidence interval, prediction interval is 0...0.17). In general, when the confidence interval includes a negative content (concentration), the result has to be given in the form... 

Both assumptions are mainly needed for constructing confidence intervals and tests for the regression parameters, as well as for prediction intervals for new observations in x. The assumption of normal distribution additionally helps avoid skewness and outliers, mean 0 guarantees a linear relationship. The constant variance, also called homoscedasticity, is also needed for inference (confidence intervals and tests). This assumption would be violated if the variance of y (which is equal to the residual variance a2, see below) is dependent on the value of x, a situation called heteroscedasticity, see Figure 4.8. 

In Hitchell s work unequal variance of the response data was compensated for by weighting the data by the variance at each level. The regression parameters and the confidence band around the regression line were estimated by least squares ( ) The overall level of uncertainty, OL, was divided between the variation in response values and the variance in the regression estimation. His overall a was 0.05. The prediction interval was estimated around a single response determination. 

Fig. 3. Plot of logio normalized ion-exchange rate at amorphous silica saturation vs. the amount of excess alkalis (Na, K), denoted by the molar ratio XAlk/(Al + IVB + FeT). All boron is treated as four-fold coordinated (IVB) and total iron (FeT) is regarded as ferric. The ion-exchange rate subtracts out the contribution of alkalis to solution from matrix dissolution. As the amount of excess alkali increases, the ion-exchange rate increases. This increase in rate reflects the increasing amount of alkalis in non-bridging oxygen (NBO) configurations. Error bars represent 2-

As in univariate calibration, prediction intervals (Pis) can be constructed from the above estimated standard error of prediction, by means of a Student s /-statistic, as ... 

N. M. Faber, X.-H. Song and P. K. Hopke, Prediction intervals for partial least squares regression, Trends Anal. Chem., 22, 2003, 330-334. 

The adiabatic model predicts 4snf-4sng intervals a factor of 2 too high and the nonadiabatic model predicts intervals 10% lower than the measured frequencies. 

A number of prediction intervals can also be generated. A two-sided prediction interval for a single future observation may be of interest. This is an interval that will contain a future observation from a population with a specified degree of confidence, 100(1 - a)%. The form of this equation is... 

Note for any stated confidence level, the confidence interval about the mean is the narrowest interval, the prediction interval for a single future observation is wider, and the tolerance interval (to contain 95% of the population) is the widest.]... 

Standard Deviation Prediction Interval (SDPI). Since uniformity is of primary interest in powder blend validation and because of a concern that a constant sampling error can occur, one approach is to base the criteria only on variability. The SDPI allows one to predict, from a sample of size n and with a specified level of assurance, an upper bound on the standard deviation of a future sample of size m from the same population. This approach is recommended in the PDA paper on blend uniformity [1],... 

Meeting the foregoing criterion should not be interpreted to mean that an individual composite potency assay will meet the in-house limits with high assurance. If this is desired, a prediction interval for a single future observation, or better yet, a tolerance interval, should be used. The validation specialist should be cautioned that additional composite assays might need to be tested to meet either one of these criteria with high confidence. 

Fig. 1 Regression line and prediction interval Pity) for the prediction of a single value y red at position x (based on Eq. 2) of the OLS-regression for the calibration of Example 1...

In this step, one or more independent experts should evaluate the quality of the training set data along with any other available data for the endpoint predicted by the QSAR. This should enable an evaluation to be made of the maximal predictive capacity that could be expected for the QSAR. For QSARs, the inevitable variability in descriptor and response variable data should be taken into consideration when defining criteria for predictive capacity. For example, in the case of a regression-based QSAR, it might be decided that its predictions should fall within a specified prediction interval, and that the R2 value for predictions of independent data should exceed a specified value. Issues relating to the quality of data for use in QSARs are discussed in Cronin and Schultz (2003) and Schultz and Cronin (2003). 

The QSAR is acceptable for the prediction of the acute toxicity to Pimephales promelas of organic chemicals, considered to act by the non-polar narcosis mechanism of action. The range of acceptable log Kow values for which it can be applied is from -1.24 to 5.13. The coefficient of determination (r2 value) of the model is 0.9, and its expected accuracy (95% prediction interval) is 0.64 log unit... 

Some would argue that if there is sufficient information upon which to quantify ranges, then there is also likely information upon which to base a judgement regarding the type of distribution that could be used to describe uncertainty in the input. Interval methods, however, may be useful for problems in which there may be complex dependencies between inputs but for which the dependencies are not known. A disadvantage of interval methods is that the predicted intervals for a model output can be quite wide, since they are informed only by the end-points of the ranges for each input. Interval methods can be useful as a quality... 

To assess the certainty with which the extent of difference in amino acid sequence can be inferred from immunological distance, we have estimated prediction intervals for five kinds of monomeric proteins. These estimates are made possible by the extensive calibration work done in our laboratory on proteins of known amino acid sequence. This chapter then uses prediction intervals to test the robustness of hypotheses about phene-tic relationships among monomeric proteins of mammals, frogs, and birds. The probabilities of arriving at various conclusions by chance range from 1 in 20 to 1 in 10 billion. 

Figure 1 shows for five proteins the 90% prediction intervals, along with regression lines and their 95% confidence limits. These parameters are... 

---