Measurement error, statistical validation

Rectification accounts for systematic measurement error. During rectification, measurements that are systematically in error are identified and discarded. Rectification can be done either cyclically or simultaneously with reconciliation, and either intuitively or algorithmically. Simple methods such as data validation and complicated methods using various statistical tests can be used to identify the presence of large systematic (gross) errors in the measurements. Coupled with successive elimination and addition, the measurements with the errors can be identified and discarded. No method is completely reliable. Plant-performance analysts must recognize that rectification is approximate, at best. Frequently, systematic errors go unnoticed, and some bias is likely in the adjusted measurements. 

In equation 3.4-18, the right side is linear with respect to both the parameters and the variables, j/the variables are interpreted as 1/T, In cA, In cB,.. . . However, the transformation of the function from a nonlinear to a linear form may result in a poorer fit. For example, in the Arrhenius equation, it is usually better to estimate A and EA by nonlinear regression applied to k = A exp( —EJRT), equation 3.1-8, than by linear regression applied to Ini = In A — EJRT, equation 3.1-7. This is because the linearization is statistically valid only if the experimental data are subject to constant relative errors (i.e., measurements are subject to fixed percentage errors) if, as is more often the case, constant absolute errors are observed, linearization misrepresents the error distribution, and leads to incorrect parameter estimates. 

Calculating statistically valid error bars after a small number N of measurements (for example, six measurement of the concentration) is an important application of statistics. Sometimes we can estimate the error in each measurement based on our knowledge of the equipment used, but more often the sources of error are so numerous that the best estimate is based on the spread of the values we measure. 

Whereas the results in this section could probably be obtained fairly easily by inspecting the original data, numerical values of class membership have been obtained which can be converted into probabilities, assuming that the measurement error is normally distributed. In most real situations, there will be a much larger number of measurements, and discrimination (e.g. by spectroscopy) is not easy to visualise without further data analysis. Statistics such as %CC can readily be obtained from the data, and it is also possible to classify unknowns or validation samples as discussed in Section 4.5.1 by this means. Many chemometricians use the Mahalanobis distance as defined above, but the normal Euclidean distance or a wide range of other measures can also be employed, if justified by the data, just as in cluster analysis. 

All of this recent activity and some that has yet to be published are expected to result in further recommendations from the CIE, including a new color tolerance equation that may be termed CIEDE200062. Whatever the actual content of the recommendation, it can be expected that the predictions of perceptible color differences and acceptable color differences will be statistically valid with an error measure that will be generally no greater than the errors made by experienced, professional color matchers. 

The difference in approach is self-evident. In the mechanistically based model, key internal and external variables are identified. Their variabilities are readily incorporated into the model to assess the overall variabihty in response. The contribution of each of the random variables on the variability in response may be readily assessed. Given the explicit functional dependence, when duly validated, it can be used to predict response beyond the range of the experimental data. The experi-entially based statistical model, on the other hand, represents a statistical fit to the data in which the key internal variables could not be identified. As such, it is incapable of capturing the functional dependence on these variables, and its usefulness is limited to the range of the experimental data. Because experimental (including measurement) errors are lumped into estimates of the fitting parameters and their variability, the quality of the subsequent reliabihty analyses may be overly conservative, or uncertain. A more detailed discussion of these approaches may be found in [7]. 

Psychophysiological measures derived during a simple choice reaction task and a mental arithmetic task, cut for the reaction phase only and classified according to the occurrence of errors (methodological study witii 4 subjects, no statistical validation). 

In geodetic networks misidentihcations of points are the biggest source of blunders because the measurement values themselves are collected on tape. Only the identification of the points is given by the observer and it is poorly controlled. A small amount of the blunders are due to measurement errors. Those blunders may be in the order of up to 100 times the accuracy of the measurements and do not cause problems of convergence. They are to be detected by statistical tests with appropriate mathematical models. But these tests also require initial values for the parameters which have to be in the range of validity of the linearization. 

Statistical and algebraic methods, too, can be classed as either rugged or not they are rugged when algorithms are chosen that on repetition of the experiment do not get derailed by the random analytical error inherent in every measurement,i° 433 is, when similar coefficients are found for the mathematical model, and equivalent conclusions are drawn. Obviously, the choice of the fitted model plays a pivotal role. If a model is to be fitted by means of an iterative algorithm, the initial guess for the coefficients should not be too critical. In a simple calculation a combination of numbers and truncation errors might lead to a division by zero and crash the computer. If the data evaluation scheme is such that errors of this type could occur, the validation plan must make provisions to test this aspect. 

The sums of squares of the individual items discussed above divided by its degrees of freedom are termed mean squares. Regardless of the validity of the model, a pure-error mean square is a measure of the experimental error variance. A test of whether a model is grossly adequate, then, can be made by acertaining the ratio of the lack-of-fit mean square to the pure-error mean square if this ratio is very large, it suggests that the model inadequately fits the data. Since an F statistic is defined as the ratio of sum of squares of independent normal deviates, the test of inadequacy can frequently be stated... 

As noted in the last section, the correct answer to an analysis is usually not known in advance. So the key question becomes How can a laboratory be absolutely sure that the result it is reporting is accurate First, the bias, if any, of a method must be determined and the method must be validated as mentioned in the last section (see also Section 5.6). Besides periodically checking to be sure that all instruments and measuring devices are calibrated and functioning properly, and besides assuring that the sample on which the work was performed truly represents the entire bulk system (in other words, besides making certain the work performed is free of avoidable error), the analyst relies on the precision of a series of measurements or analysis results to be the indicator of accuracy. If a series of tests all provide the same or nearly the same result, and that result is free of bias or compensated for bias, it is taken to be an accurate answer. Obviously, what degree of precision is required and how to deal with the data in order to have the confidence that is needed or wanted are important questions. The answer lies in the use of statistics. Statistical methods take a look at the series of measurements that are the data, provide some mathematical indication of the precision, and reject or retain outliers, or suspect data values, based on predetermined limits. 

In Equation 5.28, s is a function of the concentration residuals observed during calibration, r is tlie measurement vector for the prediction sample, and R contains the calilxation measurements for the variables used in the model. Because the assumptions of linear regression are often not rigorously obeyed, the statistical pret ion error should be used empirically rather than absolutely. It is useful for validating the prediction samples by comparing the values for... 

Measurement Residual Plot The statistical prediction errors indicate which samples have large spectral residuals. It can be instructive to then plot the residuals to diagnose the problem. In practice, only samples with large statistical prediction errors are examined, but all four will be plotted here. Tlte residuals for unknowns 1-4 are shown in Figures 5.20-5.23, respectively. Also shown are the measured and predicted responses. The residual for unknown 1 in Figure 5.20 resembles the model validation residuals shown in Figure 5.18. 

Statistical Prediction Errors The statistical prediction errors are plotted in Figure 5.60. The maximum from the model validation is indicated by a horizontal line. There are a few samples above tliis maximum and one sample (54) that has an error considerably larger than tlie rest. The measurement residuals for these samples will be investigated further. 

---