Tests of Model Adequacy

The several modeling methods discussed in the accompanying sections are quite useful in testing the ability of a model to fit a particular set of data. These methods do not, however, supplant the more conventional tests of model adequacy of classical statistical theory, i.e., the analysis of variance and tests of residuals. 

The analysis of variance is used to compare the amount of variability of the differences of predicted and experimental rates with the amount of variability in the data itself. By such comparisons, the experimenter is able to determine (a) whether the overall model is adequate and (b) whether every portion of the model under consideration is necessary. 

For every set of reaction-rate data, a total amount of variability in the data may be expressed as 

by a least-squares analysis or some other suitable means, estimates of the parameters within a proposed model may be obtained. This allows the calculation of predicted reaction rates at each experimental point and thus an assessment of the total amount of variability which can be accounted for by the proposed model 

The difference in the predicted and observed rates (yf — r ), is termed a residual and is a measure of the inability of the model to describe exactly the experimental data. If the model is entirely correct, in fact, the residual will be a measure of experimental error. A measure of the total amount of variation unaccounted for by the proposed model, then, is 

---

Test of Model Adequacy. The final step is to test the adequacy of the model. Figure 4 is a plot of the residual errors from the model vs. the observed values. The residuals are the differences between the observed and predicted values. Random scatter about a zero mean is desireable. 

Statistical testing of model adequacy and significance of parameter estimates is a very important part of kinetic modelling. Only those models with a positive evaluation in statistical analysis should be applied in reactor scale-up. The statistical analysis presented below is restricted to linear regression and normal or Gaussian distribution of experimental errors. If the experimental error has a zero mean, constant variance and is independently distributed, its variance can be evaluated by dividing SSres by the number of degrees of freedom, i.e. 

Figure 30 portrays the grid of values of the independent variables over which values of D were calculated to choose experimental points after the initial nine. The additional five points chosen are also shown in Fig. 30. Note that points at high hydrogen and low propylene partial pressures are required. Figure 31 shows the posterior probabilities associated with each model. The acceptability of model 2 declines rapidly as data are taken according to the model-discrimination design. If, in addition, model 2 cannot pass standard lack-of-fit tests, residual plots, and other tests of model adequacy, then it should be rejected. Similarly, model 1 should be shown to remain adequate after these tests. Many more data points than these 14 have shown less conclusive results, when this procedure is not used for this experimental system.

Nonlinear Models in Parameters, Single Reaction In practice, the parameters appear often in nonlinear form in the rate expressions, requiring nonlinear regression. Nonlinear regression does not guarantee optimal parameter estimates even if the kinetic model adequately represents the true kinetics and the data width is adequate. Further, the statistical tests of model adequacy apply rigorously only to models linear in parameters, and can only be considered approximate for nonlinear models. 

A number of replications under at least one set of operating conditions must be carried out to test the model adequacy (or lack of fit of the model). An estimate of the pure error variance is then calculated from ... 

There is a plethora of model adequacy tests that the user can employ to decide whether the assumed mathematical model is indeed adequate. Generally speaking these tests are based on the comparison of the experimental error variance estimated by the model to that obtained experimentally or through other means. 

In Section 5.5 a question was raised concerning the adequacy of models when fit to experimental data (see also Section 2.4). It was suggested that any test of the adequacy of a given model must involve an estimate of the purely experimental uncertainty. In Section 5.6 it was indicated that replication provides the information necessary for calculating the estimate of (. We now consider in more detail how this information can be used to test the adequacy of linear models [Davies (1956)]. 

It is only on the basis of a correct knowledge of the most probable instantaneous structure that an accurate calculation of the static electric permittiwty can be made. It is indeed likdy that the comparison of experimental and calculated temperature variation of the static electric pmnitti-vity can serve as a test of the adequacy of the structural model proposed. 

Classical statistical tests can be applied mainly to validate regression models that are linear with respect to the model parameters. The most common empirical models used in EXDE are linear models (main effect models), linear plus interactions models, and quadratic models. They all are linear with respect to the p>arameters. The most useful of these (in DOE context) are 1) t-tests for testing the significance of the individual terms of the model, 2) the lack-of-fit test for testing the model adequacy, and 3) outlier tests based on so-called externally studentized residuals, see e.g. (Neter et. al., 1996). 

Box and Hill [1967] and Box and Henson [1969] tested the model adequacy on the basis of Bayesian probabilities. 

Although all the underlying assumptions (local linearity, statistical independence, etc.) are rarely satisfied, Bartlett s jf-test procedure has been found adequate in both simulated and experimental applications (Dumez et al., 1977 Froment, 1975). However, it should be emphasized that only the x2-test and the F-test are true model adequacy tests. Consequently, they may eliminate all rival models if none of them is truly adequate. On the other hand, Bartlett s x2-test does not guarantee that the retained model is truly adequate. It simply suggests that it is the best one among a set of inadequate models ... 

Step 6. Perform the appropriate model adequacy test (x2-test, F-test or Bartlett s -/2-test) for all rival models (just in case one of the models... 

Table 12.7 Chemostat Kinetics Results from Model Adequacy Tests Assuming af. is Known (yftest) Performed at a=0.0I Level of Significance...

With this book the reader can expect to learn how to formulate and solve parameter estimation problems, compute the statistical properties of the parameters, perform model adequacy tests, and design experiments for parameter estimation or model discrimination. 

In addition to the maximum point, inflection points of the rate data can be used for testing model adequacy (M6). 

Membrane-Screening Test Membrane-screening tests were performed to evaluate (1) the adequacy of flushing procedures, (2) the rejection of model compounds by the RO membrane, and (3) losses of model compounds to system components via adsorption, volatilization, solubility artifacts, or other phenomena. 

Here, we want to emphasize that one is able to calculate the fraction of the experimental error only if replicate measurements (at least at one point x ) have been taken. It is then possible to compare model and experimental errors and to test the sources of residual errors. Then, in addition to the GOF test one can perform the test of lack of fit, LOF, and the test of adequacy, ADE, (commonly used in experimental design). In the lack of fit test the model error is tested against the experimental error and in the adequacy test the residual error is compared with the experimental error. 

If interpolation of the values of the target variable y is necessary, for example to find better estimates of the model error (for tests of adequacy), the variables, z, in Eq. 3-9 must be retransformed using Eqs. 3-6 to 3-8. 

---