Lack of fit error

The lack-of-fit error is slightly smaller titan the replicate error, in all cases except when the intercept is removed from die model for the dataset B, where it is large, 10.693. This suggests that adding the intercept term to the second dataset makes a big difference to the quality of the model and so die intercept is significant. 

Determine the sum of square replicate error, and so, from question 4, die sum of square lack-of-fit error. Divide die sum of square residual, lack-of-fit and replicate errors by dieir appropriate degrees of freedom and so construct a simple ANOVA table widi diese three errors, and compute die F-ratio. 

Calculate die sum of square lack-of-fit error by subtracting die value in question 3 from that in question 2. 

Determine the sum of squares lack-of-fit error as follows (i) replace the six replicate responses by the average response for tire replicates (ii) using the 20 responses (with the replicates averaged) and the corresponding predicted responses, calculate tire differences, square them and sum them. 

How many degrees of freedom are available for assessment of the replicate and lack-of-fit errors Using this information, comment on whether the lack-of-fit is significant, and hence whether tire model is adequate. 

For each experiment, the true values of the measured variables are related by one or more constraints. Because the number of data points exceeds the number of parameters to be estimated, all constraint equations are not exactly satisfied for all experimental measurements. Exact agreement between theory and experiment is not achieved due to random and systematic errors in the data and to "lack of fit" of the model to the data. Optimum parameters and true values corresponding to the experimental measurements must be found by satisfaction of an appropriate statistical criterion. 

In the maximum-likelihood method used here, the "true" value of each measured variable is also found in the course of parameter estimation. The differences between these "true" values and the corresponding experimentally measured values are the residuals (also called deviations). When there are many data points, the residuals can be analyzed by standard statistical methods (Draper and Smith, 1966). If, however, there are only a few data points, examination of the residuals for trends, when plotted versus other system variables, may provide valuable information. Often these plots can indicate at a glance excessive experimental error, systematic error, or "lack of fit." Data points which are obviously bad can also be readily detected. If the model is suitable and if there are no systematic errors, such a plot shows the residuals randomly distributed with zero means. This behavior is shown in Figure 3 for the ethyl-acetate-n-propanol data of Murti and Van Winkle (1958), fitted with the van Laar equation. 

A number of replications under at least one set of operating conditions must be carried out to test the model adequacy (or lack of fit of the model). An estimate of the pure error variance is then calculated from ... 

An F-test for lack of fit is based on the ratio of the lack of fit sum to the pure error sum of squares divided by their corresponding degrees of freedom ... 

Here, y is the average of all of the replicated data points. If the residual sum of squares is the amount of variation in the data as seen by the model, and the pure-error of squares is the true measure of error in the data, then the inability of the model to fit the data is given by the difference between these two quantities. That is, the lack-of-fit sum of squares is given by... 

If there are n replications at q different settings of the independent variables, then the pure-error sum of squares is said to possess (n — 1) degrees of freedom (1 degree of freedom being used to estimate y) while the lack-of-fit sum of squares is said to possess N — p — q(n — 1) degrees of freedom, i.e., the difference between the degrees of freedom of the residual sum of squares and the pure-error sum of squares. 

The sums of squares of the individual items discussed above divided by its degrees of freedom are termed mean squares. Regardless of the validity of the model, a pure-error mean square is a measure of the experimental error variance. A test of whether a model is grossly adequate, then, can be made by acertaining the ratio of the lack-of-fit mean square to the pure-error mean square if this ratio is very large, it suggests that the model inadequately fits the data. Since an F statistic is defined as the ratio of sum of squares of independent normal deviates, the test of inadequacy can frequently be stated... 

In some cases when estimates of the pure-error mean square are unavailable owing to lack of replicated data, more approximate methods of testing lack of fit may be used. Here, quadratic terms would be added to the models of Eqs. (32) and (33), the complete model would be fitted to the data, and a residual mean square calculated. Assuming this quadratic model will adequately fit the data (lack of fit unimportant), this quadratic residual mean square may be used in Eq. (68) in place of the pure-error mean square. The lack-of-fit mean square in this equation would be the difference between the linear residual mean square [i.e., using Eqs. (32) and (33)] and the quadratic residual mean square. A model should be rejected only if the ratio is very much greater than the F statistic, however, since these two mean squares are no longer independent. 

This is, then, the regression sum of squares due to the first-order terms of Eq. (69). Then, we calculate the regression sum of squares using the complete second-order model of Eq. (69). The difference between these two sums of squares is the extra regression sum of squares due to the second-order terms. The residual sum of squares is calculated as before using the second-order model of Eq. (69) the lack-of-fit and pure-error sums of squares are thus the same as in Table IV. The ratio contained in Eq. (68) still tests the adequacy of Eq. (69). Since the ratio of lack-of-fit to pure-error mean squares in Table VII is smaller than the F statistic, there is no evidence of lack of fit hence, the residual mean square can be considered to be an estimate of the experimental error variance. The ratio... 

Finally, a measure of lack of fit using a PCs can be defined using the sum of the squared errors (SSE) from the test set, flSSETEST = Latest 2 (prediction sum of squares). Here, 2 stands for the sum of squared matrix elements. This measure can be related to the overall sum of squares of the data from the test set, SStest = -Xtest 2- The quotient of both measures is between 0 and 1. Subtraction from 1 gives a measure of the quality of fit or explained variance for a fixed number of a PCs ... 

For example, a) in (radioactivity) counting experiments a non-Poisson random error component, equal in magnitude (variance) to the Poisson component, will not be detected until there are 46 degrees of freedom ( ), and b) it was necessary for a minor component in a mixed Y-ray spectrum to exceed its detection limit by -50 , before its absence was detected by lack-of-fit (x, model error) (7). 

This is perhaps the "best solution for the given data set, and it is certainly the most interesting. It is not offered as a rigorous solution, however, for the lack of fit (x /df -[9.64]2) implies additional sources of error, which may be due to additional scatter about the calibration curve (oy -"between" component), residual error in the analytic model for the calibration function, or errors in the "standard" x-values. (We believe the last source of error to be the most likely for this data set.) For these reasons, and because we wish to avoid complications introduced by non-linear least squares fitting, we take the model y=B+Axl 12 and the relation Oy = 0.028 + 0.49x to be exact and then apply linear WLS for the estimation of B and A and their standard errors. 

The amount data corresponding to the response values in 1 above were transformed by the same general family of power transformations until linearity was obtained. The F-test statistic that relates lack of fit and pure error was used as the criterion for linearity. 

F =MSLF/MSPE, based on the ratio mean square for lack of fit (MSLF) over the mean square for pure error (MSPE) ( 31 ). F follows the F distribujfion with (r-2) and (N-r) degrees of freedom. A value of F regression equation. Since the data were manipulated by transforming the amount values jfo obtain linearity, i. e., to achieve the smallest lack of fit F statistic, the significance level of this test is not reliable. 

The relationship takes into account that the day-to-day samples determined are subject to error from several sources random error, instrument error, observer error, preparation error, etc. This view is the basis of the process of fitting data to a model, which results in confidence intervals based on the intrinsic lack of fit and the random variation in the data. 

First, amount error estimations in Wegscheider s work were the result of only the response uncertainty with no regression (confidence band) uncertainty about the spline. His spline function knots were found from the means of the individual values at each level. Hence the spline exactly followed the points and there was no lack of fit in this method. Confidence intervals around spline functions have not been calculated in the past but are currently being explored ( 5 ). 

Significant lack-of-fit can be detected by various sensible methods. However, the sample correlation coefficient r does not belong to the pool of these methods to assess linearity. The sample correlation coefficient may be misleading and is, despite its widespread use, to be discouraged for two reasons First, r depends on the slope. That is, for lines with the same scatter of the points about the line, r increases with the slope. Second, the numerical value of the correlation coefficient cannot be interpreted in terms of degree of deviation from linearity. Put differently, a correlation coefficient of 0.99 may be due to random error of a strictly linear relationship or due to systematic deviations from the regression line. 

Formal tests are also available. The ANOVA lack-of-fit test ° capitalizes on the decomposition of the residual sum of squares (RSS) into the sum of squares due to pure error SSs and the sum of squares due to lack of fit SSiof. Replicate measurements at the design points must be available to calculate the statistic. First, the means of the replicates (4=1,. .., m = number of different design points) at all design points are calculated. Next, the squared deviations of all replicates U — j number of replicates] from their respective mean... 

When an unreplicated experiment is run, the error or residual sum of squares is composed of both experimental error and lack-of-fit of the model. Thus, formal statistical significance testing of the factor effects can lead to erroneous conclusions if there is lack-of-fit of the model. Therefore, it is recommended that the experiment be replicated so that an independent estimate of the experimental error can be calculated and both lack-of-fit and the statistical significance of the factor effects can be formally tested. 

As I have shown, the response given by the model equation (3.5) has an error term that includes the lack of fit of the model and dispersion due to the measurement (repeatability). For the three-factor example discussed above, there are four estimates of each effect, and in general the number of estimates are equal to half the number of runs. The variance of these estimated effects gives some indication of how well the model and the measurement bear up when experiments are actually done, if this value can be compared with an expected variance due to measurement alone. There are two ways to estimate measurement repeatability. First, if there are repeated measurements, then the standard deviation of these replicates (s) is an estimate of the repeatability. For N/2 estimates of the factor effect, the standard deviation of the effect is... 

Spreadsheet 8.2. Calculations for test of linear range of data shown in spreadsheet 8.1. Range tested, 3.1-50.0 nM N=24 data points number of concentrations, k, = 6. ME = measurement error, LOF = lack of fit, SS = sum of squares, MS = mean square. 

---