Error distribution

Clearly, the HF method, independent of basis, systematically underestimates the bond lengdis over a broad percentage range. The CISD method is neither systematic nor narrowly distributed in its errors, but the MP2 and MP4 (but not MP3) methods are reasonably accurate and have narrow error distributions if valence TZ or QZ bases are used. The CCSD(T), but not the CCSD, method can be quite reliable if valence TZ or QZ bases are used. 

Experience gained in the ZAF analysis of major and minor constituents in multielement standards analyzed against pure element standards has produced detailed error distribution histograms for quantitative EPMA. The error distribution is a normal distribution centered about 0%, with a standard deviation of approximately 2% relative. Errors as high as 10% relative are rarely encountered. There are several important caveats that must be observed to achieve errors that can be expected to lie within this distribution ... 

Adaptive optics requires a reference source to measure the phase error distribution over the whole telescope pupil, in order to properly control DMs. The sampling of phase measurements depends on the coherence length tq of the wavefront and of its coherence time tq. Both vary with the wavelength A as A / (see Ch. 1). Of course the residual error in the correction of the incoming wavefront depends on the signal to noise ratio of the phase measurements, and in particular of the photon noise, i.e. of the flux from the reference. This residual error in the phase results in the Strehl ratio following S = exp —a ). 

MCT allows one to choose any conceivable error distribution for the variables, and to transform these into a result by any set of equations or algorithms, such as recursive (e.g., root-finding according to Newton) or matrix inversion (e.g., solving a set of simultaneous equations) procedures. Characteristic error distributions are obtained from experience or the literature, e.g.. Ref. 95. 

The flowsheet shown in the introduction and that used in connection with a simulation (Section 1.4) provide insights into the pervasiveness of errors at the source, random errors are experienced as an inherent feature of every measurement process. The standard deviation is commonly substituted for a more detailed description of the error distribution (see also Section 1.2), as this suffices in most cases. Systematic errors due to interference or faulty interpretation cannot be detected by statistical methods alone control experiments are necessary. One or more such primary results must usually be inserted into a more or less complex system of equations to obtain the final result (for examples, see Refs. 23, 91-94, 104, 105, 142. The question that imposes itself at this point is how reliable is the final result Two different mechanisms of action must be discussed ... 

The estimate p (x) retained need not be at the center of the confidence interval ]qw(2L) (x) Since the confidence intervals are obtained directly, there is no need to calculate the estimation variance, nor to hypothesize any model for the error distribution. 

For a normal (Gaussian) error distribution, the RMSE is by a factor of Jl larger than the mean absolute error, also denoted as mean unsigned error. The error distribution of log Sw prediction methods appears to be somewhat less inhomogeneous than a Gaussian distribution and typically leads to a ratio of RMSE/mean absolute error... 

One must note that probability alone can only detect alikeness in special cases, thus cause-effect cannot be directly determined - only estimated. If linear regression is to be used for comparison of X and Y, one must assess whether the five assumptions for use of regression apply. As a refresher, recall that the assumptions required for the application of linear regression for comparisons of X and Y include the following (1) the errors (variations) are independent of the magnitudes of X or Y, (2) the error distributions for both X and Y are known to be normally distributed (Gaussian), (3) the mean and variance of Y depend solely upon the absolute value of X, (4) the mean of each Y distribution is a straight-line function of X, and (5) the variance of X is zero, while the variance of Y is exactly the same for all values of X. 

In equation 3.4-18, the right side is linear with respect to both the parameters and the variables, j/the variables are interpreted as 1/T, In cA, In cB,.. . . However, the transformation of the function from a nonlinear to a linear form may result in a poorer fit. For example, in the Arrhenius equation, it is usually better to estimate A and EA by nonlinear regression applied to k = A exp( —EJRT), equation 3.1-8, than by linear regression applied to Ini = In A — EJRT, equation 3.1-7. This is because the linearization is statistically valid only if the experimental data are subject to constant relative errors (i.e., measurements are subject to fixed percentage errors) if, as is more often the case, constant absolute errors are observed, linearization misrepresents the error distribution, and leads to incorrect parameter estimates. 

An analysis of the influence of errors shows clearly that the double-reciprocal plot according to Lineweaver-Burk [32] is the least suitable. Although it is by far the most widely used plot in enzyme kinetics, it cannot be recommended, because it gives a grossly misleading impression of the experimental error for small values of v small errors in v lead to enormous errors in 1/y but for large values of v the same small errors in v lead to barely noticeable errors in 1/17 [23]. Due to the error distribution, that is much more uniform, the plot according to Hanes (Eq. (7)), is the most favored. 

Albuquerque and Biegler (1996) followed a different approach to incorporating bias into the dynamic data reconciliation, by taking into account the presence of a bias from the very beginning through the use of contaminated error distributions. This approach is fully discussed in Chapter 11. 

The general problem is then to estimate 0 and u knowing the values of the measurements, y, and the probability distribution function of e (measurement error). If P(e) is the error distribution, then y will be distributed according to P y - x 9, u). Thus, according to Bayes theorem, (Alburquerque and Biegler, 1996), the posterior... 

Now, if (m2 > g), the solution of Eq. (10.24), under the assumption of an independent and normal error distribution with constant variance can be obtained as the maximum likelihood estimator of d and is given by... 

The performances of the indirect conventional methods described previously are very sensitive to outliers, so they are not robust. The main reason for this is that they use a direct method to calculate the covariance matrix of the residuals (). If outliers are present in the sampling data, the assumption about the error distribution will be... 

Normal distribution is commonly used for random and gross errors, but gross error distributions have a higher variance than the random error distributions. This leads to the n-contaminated normal distribution with normal contamination, that is,... 

Figures 1 to 4 illustrate the results of the reconciliation for the four variables involved. As can be seen, this approach does not completely eliminate the influence of the outliers. For some of the variables, the prediction after reconciliation is actually deteriorated because of the presence of outliers in some of the other measurements. This is in agreement with the findings of Albuquerque and Biegler (1996), in the sense that the results of this approach can be very misleading if the gross error distribution is not well characterized.

Moberg et al. (1980) use the residuals from the preliminary fit to classify the error distribution according to the tail-weight and skewness characteristics. They then use the classification to select a it function. The general form of the function for a skewed light-tailed distribution according to Moberg is... 

An alternative approach to these methods is to obtain the influence function directly from the error distribution. In this case, for the maximum likelihood estimation of the parameters, the i/ function can be chosen as follows ... 

This Monte Carlo study shows that, for this family of error distributions, the WARME method has the best performance. Also, as expected, the OLS performs the best for the normal distribution, but performs very poorly for the case when outliers are present in the data set. The Huber minimax estimator can decrease the influence of the outliers, but it only works well for the symmetric distribution. Biegler s fair function is also designed for symmetric distributions. 

Frequently, the measurement error distributions arising in a practical data set deviate from the assumed Gaussian model, and they are often characterized by heavier tails (due to the presence of outliers). A typical heavy-tailed noise record is given in Fig. 7, while Fig. 8 shows the QQ-plots of this record, based on the hypothesized standard normal distribution. 

The same problem discussed in Examples 11.1 and 11.3 is taken to illustrate the ideas described in this section (Chen and Romagnoli, 1997). To evaluate the performance of the proposed approach under different error distributions, Monte Carlo simulations have again been performed on the four previous distributions. 

Theory for the transformation of the dependent variable has been presented (Bll) and applied to reaction rate models (K4, K10, M8). In transforming the dependent variable of a model, we wish to obtain more perfectly (a) linearity of the model (b) constancy of error variance, (c) normality of error distribution and (d) independence of the observations to the extent that all are simultaneously possible. This transformation will also allow a simpler and more precise data analysis than would otherwise be possible. 

If this procedure is followed, then a reaction order will be obtained which is not masked by the effects of the error distribution of the dependent variables If the transformation achieves the four qualities (a-d) listed at the first of this section, an unweighted linear least-squares analysis may be used rigorously. The reaction order, a = X + 1, and the transformed forward rate constant, B, possess all of the desirable properties of maximum likelihood estimates. Finally, the equivalent of the likelihood function can be represented b the plot of the transformed sum of squares versus the reaction order. This provides not only a reliable confidence interval on the reaction order, but also the entire sum-of-squares curve as a function of the reaction order. Then, for example, one could readily determine whether any previously postulated reaction order can be reconciled with the available data. 

An informative description of the prediction errors is a visual representation, for instance by a histogram or a probability density curve these plots, however, require a reasonable large number of predictions. For practical reasons, the error distribution can be characterized by a single number, for instance the standard deviation. 

The basis of all performance criteria are prediction errors (residuals), yt - yh obtained from an independent test set, or by CV or bootstrap, or sometimes by less reliable methods. It is crucial to document from which data set and by which strategy the prediction errors have been obtained furthermore, a large number of prediction errors is desirable. Various measures can be derived from the residuals to characterize the prediction performance of a single model or a model type. If enough values are available, visualization of the error distribution gives a comprehensive picture. In many cases, the distribution is similar to a normal distribution and has a mean of approximately zero. Such distribution can well be described by a single parameter that measures the spread. Other distributions of the errors, for instance a bimodal distribution or a skewed distribution, may occur and can for instance be characterized by a tolerance interval. 

The classical standard deviation of prediction errors is widely used as a measure of the spread of the error distribution, and is called in this application standard error of prediction (SEP) defined by... 

A single CV as described gives n predictions. For many data sets in chemistry n is too small for a visualization of the error distribution. Furthermore, the obtained performance measure may heavily depend on the split of the objects into segments. It is therefore recommended to repeat the CV with different random splits into segments (repeated CV), and to summarize the results. Knowing the variability of MSEcv at different levels of model complexities also allows a better estimation of the optimum model complexity, see one standard error rule in Section 4.2.2 (Hastie et al. 2001). 

---