Uncertainty purely experimental

Is it justifiable to conclude that the factor x, has an influence on the output y, The answer requires a knowledge of the purely experimental uncertainty of the response, the variability that arises from causes other than intentional changes in the factor levels. This variance due to purely experimental uncertainty is given the symbol a, and its estimate is denoted sL. 

If the purely experimental uncertainty is small compared with the difference in... 

Unfortunately, two experiments at two different levels of a single factor cannot provide an estimate of the purely experimental uncertainty. The difference in the two observed responses might be due to experimental uncertainty, or it might be caused by a sloping response surface, or it might be caused by both. For this particular experimental design, the effects are confused (or confounded) and there is no way to separate the relative importance of these two sources of variation. 

If the purely experimental uncertainty were known, it would then be possible to judge the adequacy of the model y, = Po + r, if s were very much greater than (see Figure 5.6), then it would be unlikely the large residuals for that model occurred by chance, and we would conclude that the model does not adequately describe the true behavior of the system. However, if 5 were approximately the same as 5 (see Figure 5.7), then we would conclude that the model was adequate. (The actual decision compares s to a variance slightly different from s], but the reasoning is similar.)... 

The estimation of purely experimental uncertainty is essential for testing the adequacy of a model. The material in Chapter 3 and especially in Figure 3.1 suggests one of the important principles of experimental design the purely experimental uncertainty can be obtained only by setting all of the controlled factors at fixed levels and replicating the experiment. 

Replication is the independent performance of two or more experiments at the same set of levels of all controlled factors. Replication allows both the calculation of a mean response, y and the estimation of the purely experimental uncertainty, 5, at that set of factor levels. 

Figure 5.7 Graphs of three models O , = Po + PiJC, = Po + fi,. and yu = Pi ti. + fw) A two data points having large purely experimental uncertainty.

Most textbooks refer to as the variance due to pure error , or the pure error variance . In this textbook, is called the variance due to purely experimental uncertainty , or the purely experimental uncertainty variance . What assumptions might underlie each of these systems of naming See Problem 6.14. [See, also, pages 123-127 in Mandel (1964).]... 

Refer to Figure 3.1. How can purely experimental uncertainty be decreased What are the advantages of making the purely experimental uncertainty small What are the disadvantages of making the purely experimental uncertainty small ... 

Consider the following four experiments Xj, = 3, yj, = 2, x,2 = 3, y,2 = 4, x,3 = 6, y,3 = 6, x,4 = 6, y,4 = 4. At how many levels of x, have experiments been carried out What is the mean value of yj at each level of x, How many degrees of freedom are removed by calculation of these two means How many degrees of freedom remain for the calculation of j If n is the number of experiments in this set and / is the number of levels of x then what is the relationship between n and /that expresses the number of degrees of freedom available for calculating the purely experimental uncertainty Why is = 2 for this set of data ... 

In Section 5.5 a question was raised concerning the adequacy of models when fit to experimental data (see also Section 2.4). It was suggested that any test of the adequacy of a given model must involve an estimate of the purely experimental uncertainty. In Section 5.6 it was indicated that replication provides the information necessary for calculating the estimate of (. We now consider in more detail how this information can be used to test the adequacy of linear models [Davies (1956)]. 

If the model = 0 + r, does describe the true behavior of the system, we would expect replicate experiments to have a mean value of zero (y,- = 0) the sum of squares due to purely experimental uncertainty would be expected to be... 

If (and only if) replicate experiments have been carried out on a system, it is possible to partition the sum of squares of residuals, SS, into two components (see Figure 6.10) one component is the already familiar sum of squares due to purely experimental uncertainty, 55. the other component is associated with variation attributed to the lack of fit of the model to the data and is called the sum of squares due to lack of fit, SS. ... 

Although it is beyond the scope of this presentation, it can be shown that if the model yj. = 0 + r, is a true representation of the behavior of the system, then the three sui.. s of squares SS and divided by the associated degrees of freedom (2, 1, and 1 respectively for this example) will all provide unbiased estimates of and there will not be significant differences among these estimates. If y, = 0 + r, is not the true model, the parameter estimate will still be a good estimate of the purely experimental uncertainty, (the estimate of purely experimental uncertainty is independent of any model - see Sections 5.5 and 5.6). The parameter estimate however, will be inflated because it now includes a non-random contribution from a nonzero difference between the mean of the observed replicate responses, y, and the responses predicted by the model, y, (see Equation 6.13). The less likely it is that y, - 0 + r, is the true model, the more biased and therefore larger should be the term Si f compared to 5. ... 

Assume the model = 0 + r, is used to describe the nine data points in Section 3.1. Calculate directly the sum of squares of residuals, the sum of squares due to purely experimental uncertainty, and the sum of squares due to lack of fit. How many degrees of freedom are associated with each sum of squares Do and SS add up to give SS l Calculate and What is the value of the Fisher F-ratio for lack of fit (Equation 6.27)7 Is the lack of fit significant at or above the 95% level of confidence ... 

In Section 6.2, the standard uncertainty of the parameter estimate was obtained by taking the square root of the product of the purely experimental uncertainty variance estimate, and the matrix (see Equation 6.3). A single number was... 

For the general, multiparameter case, the product of the purely experimental uncertainty estimate, and the matrix gives the estimated... 

If we assume the model y, = 0 + r, , the data and uncertainties are as shown in Figure 8.1. We can test this model for lack of fit because there is replication in the experimental design which allows an estimate of the purely experimental uncertainty (with two degrees of freedom). 

The variance due to purely experimental uncertainty (with two degrees of freedom) is... 

Figure 8.5 shows that can be estimated most precisely when the third experiment is located at x,3 = 0. This is reasonable, for the contribution of the third experiment at x, = 0 to the variance associated with involves no interpolation or extrapolation of a model if the third experiment is carried out at X = 0, then any discrepancy between y,3 and the true intercept must be due to purely experimental uncertainty only. As the third experiment is moved away from x, = 0, does increase, but not drastically the two stationary experiments remain positioned near X, = 0 and provide reasonably good estimates of hg by themselves. 

In Section 6.4, it was shown for replicate experiments at one factor level that the sum of squares of residuals, SS can be partitioned into a sum of squares due to purely experimental uncertainty, SS, and a sum of squares due to lack of fit, SSi f. Each sum of squares divided by its associated degrees of freedom gives an estimated variance. Two of these variances, and were used to calculate a Fisher F-ratio from which the significance of the lack of fit could be estimated. 

Finally, from the mean of replicate responses (y,) to the response itself (y,). This distance is a measure of the purely experimental uncertainty. If the measurement of response is precise, this distance should be small. 

Before discussing the sum of squares due to lack of fit and, later, the sum of squares due to purely experimental uncertainty, it is computationally useful to define a matrix of mean replicate responses, J, which is structured the same as the Y matrix, but contains mean values of response from replicates. For those experiments that were not replicated, the mean response is simply the single value of response. The J matrix is of the form... 

In a sense, calculating the mean replicate response removes the effect of purely experimental uncertainty from the data. It is not unreasonable, then, to expect that the deviation of these mean replicate responses from the estimated responses is due to a lack of fit of the model to the data. The matrix of lack-of-fit deviations, L, is obtained by subtracting f from J... 

Zeros will appear in the P matrix for those experiments that were not replicated (yn = y,( for these experiments). The sum of squares due to purely experimental uncertainty is easily calculated. 

Although the coefficients of determination and the correlation coefficients are conceptually simple and attractive, and are frequently used as a measure of how well a model fits a set of data, they are not, by themselves, a good measure of the effectiveness of the factors as they appear in the model, primarily because they do not take into account the degrees of freedom. Thus, the value of R can usually be increased by adding another parameter to the model (until p =J), but this increased R value does not necessarily mean that the expanded model offers a significantly better fit. It should also be noted that the coefficient of determination gives no indication of whether the lack of perfect prediction is caused by an inadequate model or by purely experimental uncertainty. 

We emphasize that if the lack of fit of a model is to be tested, f-p (the degrees of freedom associated with 55, f) and n - p (the degrees of freedom associated with 55pj) must each be greater than zero that is, the number of factor combinations must be greater than the number of parameters in the model, and there should be replication to provide an estimate of the variance due to purely experimental uncertainty. 

Note that the Fisher F-ratio for the significance of lack of fit cannot be tested because there are no degrees of freedom for purely experimental uncertainty. This lack of degrees of freedom for replication is a usual feature of observational data. Any information about lack of fit must be obtained from patterns in the residuals. 

A set of n measured responses has a total of n degrees of freedom. Of these, n -f degrees of freedom are given to the estimation of variance due to purely experimental uncertainty (5p, f - p degrees of freedom are used to estimate the variance due to lack of fit (5 of)> P degrees of freedom are used to estimate the parameters of the model (see Table 9.2). 

---