Experimental uncertainty, estimation

For the general, multiparameter case, the product of the purely experimental uncertainty estimate, and the matrix gives the estimated... 

Although the computed and experimentally estimated [84] (extrapolated to zero frequency) hyperpolarizability for Ne are in perfect agreement, the results for Ar do not agree within their respective uncertainties. Actually, this is due to an experimental uncertainty estimate that is almost certainly too optimistic. The uncertainty quoted for the experimental value is derived from the experimental statistics, that is, it is a measure of random error in the measurement. It contains essentially no contribution from any possible source of systematic error. In fact, Shelton believes that a more realistic uncertainty would be 20 or perhaps 30 [85]. If that were the case there would be no disagreement between theory and experiment. This is an excellent illustration of the dangers of relying on a given experimental estimate or uncertainty. It is always necessary to ascertain exactly what the experimentalist means by his/her error bars. 

The temperature dependence of A predicted by Eq. (5-11) makes a very weak contribution to the temperature dependence of the rate constant, which is dominated by the exponential term. It is, therefore, not feasible to establish, on the basis of temperature studies of the rate constant, whether the predicted dependence of A is observed experimentally. Uncertainties in estimates of A tend to be quite large because this parameter is, in effect, determined by a long extrapolation of the Arrhenius plot to 1/T = 0. 

There are to be found lists of chemical substances in handbooks for each of which log P = f (T), and whose coefficients are to be inserted, are given. These lists are limited but nevertheless provide solutions for the most common chemical substances. When there are several experimental estimates of vapour pressures it is possible to estimate the importance of the experimental uncertainty from the standard deviation of the measurements. The relevance of the values can be verified from a series of different sources (to be rigorous, checking that it is a Gaussian sequence would be required). 

A brief study of the available data related to limits of inflammability in Part Two shows that these parameters are subject to high experimental uncertainty. For a large number of substances, the experimental values are widely dispersed. When they are submitted to quality estimation using statistical tools, in many cases they reveal that it is impossible to use them with confidence. The examples of difficulties raised by the statistical analysis of the LEL data can be multiplied. 

Both in cases (i) and (ii) the experimental uncertainty has to be estimated realistically as is done for the reference value, too. All the sources of variations and deviations have to be included in the calculation of the uncertainty. Efforts of analysts to shine with excellent analytical performance characteristic can have a detrimental effect as the examples in Table 8.1 demonstrate. 

Flash photolysis techniques were unsuitable for measuring the slow off reactions for the iron(II) model complexes such as Fen(TPPS)(NO), since the experimental uncertainties in the extrapolated intercepts of kohs vs. [NO] plots were larger than the values of the intercepts themselves. When trapping methods were used to evaluate NO labilization from FeII(TPPS)(NO), k(,n values were found to be quite small but were sensitive to the nature of the trapping agents used. Lewis bases that could coordinate the metal center appeared to accelerate NO loss. More reliable estimates for the uncatalyzed off reaction were obtained by using Ru(edta)- as an NO scavenger, and the koS values listed in Table I were obtained in this manner (21c). The small kQ values found for Fe(II) models are consistent with the trend observed for the ferro-heme proteins discussed above. 

The values of AHf (g) carry both the experimental uncertainty in the standard enthalpy of formation of the crystalline (or liquid) metal compound and the uncertainty (experimental or estimated) in the enthalpy of sublimation (or vaporization). 

Equation 2.67 indicates that the standard enthalpy and entropy of reaction 2.64 derived from Kc data may be close to the values obtained with molality equilibrium constants. Because Ar// is calculated from the slope of In AT versus l/T, it will be similar to the value derived with Km data provided that the density of the solution remains approximately constant in the experimental temperature range. On the other hand, the error in ArSj calculated with Kc data can be roughly estimated as R In p (from equations 2.57 and 2.67). In the case of water, this is about zero for most solvents, which have p in the range of 0.7-2 kg dm-3, the corrections are smaller (from —3 to 6 J K-1 mol-1) than the usual experimental uncertainties associated with the statistical analysis of the data. 

Is it justifiable to conclude that the factor x, has an influence on the output y, The answer requires a knowledge of the purely experimental uncertainty of the response, the variability that arises from causes other than intentional changes in the factor levels. This variance due to purely experimental uncertainty is given the symbol a, and its estimate is denoted sL. 

Unfortunately, two experiments at two different levels of a single factor cannot provide an estimate of the purely experimental uncertainty. The difference in the two observed responses might be due to experimental uncertainty, or it might be caused by a sloping response surface, or it might be caused by both. For this particular experimental design, the effects are confused (or confounded) and there is no way to separate the relative importance of these two sources of variation. 

The estimation of purely experimental uncertainty is essential for testing the adequacy of a model. The material in Chapter 3 and especially in Figure 3.1 suggests one of the important principles of experimental design the purely experimental uncertainty can be obtained only by setting all of the controlled factors at fixed levels and replicating the experiment. 

Replication is the independent performance of two or more experiments at the same set of levels of all controlled factors. Replication allows both the calculation of a mean response, y and the estimation of the purely experimental uncertainty, 5, at that set of factor levels. 

In Section 5.5 a question was raised concerning the adequacy of models when fit to experimental data (see also Section 2.4). It was suggested that any test of the adequacy of a given model must involve an estimate of the purely experimental uncertainty. In Section 5.6 it was indicated that replication provides the information necessary for calculating the estimate of (. We now consider in more detail how this information can be used to test the adequacy of linear models [Davies (1956)]. 

Although it is beyond the scope of this presentation, it can be shown that if the model yj. = 0 + r, is a true representation of the behavior of the system, then the three sui.. s of squares SS and divided by the associated degrees of freedom (2, 1, and 1 respectively for this example) will all provide unbiased estimates of and there will not be significant differences among these estimates. If y, = 0 + r, is not the true model, the parameter estimate will still be a good estimate of the purely experimental uncertainty, (the estimate of purely experimental uncertainty is independent of any model - see Sections 5.5 and 5.6). The parameter estimate however, will be inflated because it now includes a non-random contribution from a nonzero difference between the mean of the observed replicate responses, y, and the responses predicted by the model, y, (see Equation 6.13). The less likely it is that y, - 0 + r, is the true model, the more biased and therefore larger should be the term Si f compared to 5. ... 

In Section 6.2, the standard uncertainty of the parameter estimate was obtained by taking the square root of the product of the purely experimental uncertainty variance estimate, and the matrix (see Equation 6.3). A single number was... 

If we assume the model y, = 0 + r, , the data and uncertainties are as shown in Figure 8.1. We can test this model for lack of fit because there is replication in the experimental design which allows an estimate of the purely experimental uncertainty (with two degrees of freedom). 

Figure 8.5 shows that can be estimated most precisely when the third experiment is located at x,3 = 0. This is reasonable, for the contribution of the third experiment at x, = 0 to the variance associated with involves no interpolation or extrapolation of a model if the third experiment is carried out at X = 0, then any discrepancy between y,3 and the true intercept must be due to purely experimental uncertainty only. As the third experiment is moved away from x, = 0, does increase, but not drastically the two stationary experiments remain positioned near X, = 0 and provide reasonably good estimates of hg by themselves. 

In Section 6.4, it was shown for replicate experiments at one factor level that the sum of squares of residuals, SS can be partitioned into a sum of squares due to purely experimental uncertainty, SS, and a sum of squares due to lack of fit, SSi f. Each sum of squares divided by its associated degrees of freedom gives an estimated variance. Two of these variances, and were used to calculate a Fisher F-ratio from which the significance of the lack of fit could be estimated. 

In a sense, calculating the mean replicate response removes the effect of purely experimental uncertainty from the data. It is not unreasonable, then, to expect that the deviation of these mean replicate responses from the estimated responses is due to a lack of fit of the model to the data. The matrix of lack-of-fit deviations, L, is obtained by subtracting f from J... 

We emphasize that if the lack of fit of a model is to be tested, f-p (the degrees of freedom associated with 55, f) and n - p (the degrees of freedom associated with 55pj) must each be greater than zero that is, the number of factor combinations must be greater than the number of parameters in the model, and there should be replication to provide an estimate of the variance due to purely experimental uncertainty. 

A set of n measured responses has a total of n degrees of freedom. Of these, n -f degrees of freedom are given to the estimation of variance due to purely experimental uncertainty (5p, f - p degrees of freedom are used to estimate the variance due to lack of fit (5 of)> P degrees of freedom are used to estimate the parameters of the model (see Table 9.2). 

Although the derivation is beyond the scope of this presentation, it can be shown that the estimated variance of predicting a single new value of response at a given point in factor space, is equal to the purely experimental uncertainty variance, plus the variance of estimating the mean response at that point, 5, 0 that is. 

Before leaving this example, we would point out that there is an excessive amount of purely experimental uncertainty in the system under study. The range of values obtained for replicate determinations is rather large, and suggests the existence of important factors that are not being controlled (see Figure 11.10). If steps are taken to bring the system under better experimental control, the parameters of the model can be estimated with better precision (see Equation 7.1). 

Because four parameters were estimated from data obtained at four factor combinations, there are no degrees of freedom for lack of fit further, there was no replication in this example, so there are no degrees of freedom for purely experimental uncertainty. Thus, there can be no degrees of freedom for residuals, and the estimated model will appear to fit the data perfectly . This is verified by estimating the responses using the fitted model parameters. 

The book has been written around a framework of linear models and matrix least squares. Because we authors are so often involved in the measurement aspects of investigations, we have a special fondness for the estimation of purely experimental uncertainty. The text reflects this prejudice. We also prefer the term purely experimental uncertainty rather than the traditional pure error , for reasons we as analytical chemists believe should be obvious. 

---