Model discrimination criterion

Inferences on the parameter estimates are made the same as for a linear model, but the inferences are approximate. Therefore, using a T-test in nonlinear regression to test for some parameter being equal to zero or some other value is risky and should be discouraged (Myers, 1986). However, that is not to say that the standard errors of the parameter estimates cannot be used as a model discrimination criterion. Indeed, a model with small standard errors is a better model than a model with large standard errors, all other factors... 

The positive values obtained in practically all cases indicate that all these models may be plausible representations of the data and indeed, the correlation coefBcients, R, are greater than 0.9. Thus, statistical compliance is not a sufficient basis for model discrimination. Specifically, the thermodynamic consistency of the estimates, as proposed by Boudart et al. [3], is appropriate further scrutinizing criterion during kinetic modelling and has been gainfully employed in other reactions [4-6]. 

Once model discrimination has been accomplished (that is, from a large group of rival models one best model has been selected as being adequate), further experimentation can be conducted to improve parameter estimates. For such parameter-estimation experiments, one design criterion suggested is to choose experiments providing the most desirable posterior distribution of the parameters. Under certain assumptions (B15), the procedure reduces to the maximization of the determinant... 

Several criteria for design selection have been proposed in the literature. The choice of criterion depends on the objective of the experiment. Here, criteria for model estimation capacity and model discrimination capability are described. 

Bayesian methods have often proved useful for design of experiments, especially in situations in which the optimal design depends on unknown quantities. Certainly, to identify a design for optimal estimation of /3, the correct subset of active effects must be identified. Bayesian approaches that express uncertainty about the correct subset enable construction of optimality criteria that account for this uncertainty. Such approaches typically find a design that optimizes a criterion which is averaged over many possible subsets. DuMouchel and Jones (1994) exploited this idea with a formulation in which some effects have uncertainty associated with whether they are active. Meyer et al. (1996) extended the prior distributions of Box and Meyer (1993) and constructed a model discrimination design criterion. The criterion is based on a Kullback-Leibler measure of dissimilarity between... 

Early strategies for model discrimination called for experiments at locations where the predictions of the leading candidate models differed most. Such a criterion was given by Hunter and Reiner (1965) for discrimination between... 

Hill, W. J., W. G. Hunter, and D. W. Wichern. A joint design criterion for the dual problem of model discrimination and parameter estimation. Technometrics,... 

The chlorination of tetrachloroethane successively to the penta and hexa chlorides on a deactivating catalyst is used as a good complex model system for illustrating the application of rigorous statistical methods for model discrimination using a powerful discriminatory criterion. 

For systems that are more complex, the second technique, true model discrimination is applied. This involves the use of a statistical criterion to select settings of the independent variables, e.g. experimental conditions, such that differences between models are emphasized. 

The discrimination was based on the divergence criterion of Eq. 2.3.d.l-2 in which y is replaced by r and model adequacy criterion Eq. 2.3.d.l-4 utilized. Since the experiments were performed in a differential reactor the independent variables were the partial pressures of butene, pg, butadiene pj, and hydrogen, TTie operability region for the experiments at 525°C is shown in Fig. 1. The equilibrium surface is also represented in this figure by means of hyperbola paralle] to the PoPHj-plane and straight lines parallel to the p pni- and p po-plane respectively. Possible experiments are marked with a white dot. Experimental settings too close to the equilibrium were avoided, for obvious reasons. The maximum number of parameters in the possible models is six, so that at least seven preliminary... 

It should be noted that this criterion does not carry fully over to the error-in-variables case. It is used here based on the heuristic argument that the errors in the measured mole fractions are expected to be much smaller than the values in the estimated diffusion coefficient. The corrected mole fractions are therefore assumed to be error-free in the model discrimination step. 

In this paper I have attempted to demonstrate a method for the development of hyperbolic rate models that are adequate for the design of chemical reactors. The method is rapid and overcomes most of the problems that historically have hampered the development of such models for complex reactions. I have shown that the quality of fit of a model to error-containing data is a poor criterion for model discrimination, and that several models may predict almost equally well. This, of course, has been known for a long time, but it has not been widely recognized that the model that fits the data least well may be the best model, and that the converse also may be true. In the final analysis... 

For homonuclear molecules s = / — j takes only even values whereas j is even for para modification and odd for ortho modification of the molecules. With a proper choice of fitting parameters any fitting law reproduces experimental line width rather well. Hence the good fit to their -dependence may not be considered as a criterion of quality of a fitting law. To discriminate between models it is necessary to gain agreement with experimental data on te or xE, which are much more... 

Buzzi-Ferraris et al., (1983, 1984) proposed the use of a more powerful statistic for the discrimination among rival models however, computationally it is more intensive as it requires the calculation of the sensitivity coefficients at each grid point of the operability region. Thus, the simple divergence criterion of Hunter and Reimer (1965) appears to be the most attractive. 

Figure 12.8 displays an organization chart of various quantitative methods, in an effort to better understand their similarities and differences. Note that the first discriminator between these methods is the direct versus inverse property. Inverse methods, such as MLR and partial least squares (PLS), have had a great deal of success in PAT over the past few decades. However, direct methods, such as classical least squares (CLS) and extensions thereof, have seen a recent resurgence [46-51]. The criterion used to distinguish between a direct and an inverse method is the general form of the model, as shown below ... 

The basic idea in discriminating among multiple global solutions of the MILP transshipment model of Papoulias and Grossmann (1983) is to use as criterion the verticality of heat transfer,... 

Remark 3 Model P4 will provide good results only when the heat transfer coefficients are equal or close in values, since in this case the vertical heat transfer results in minimum heat transfer area. If however, the heat transfer coefficients are different, then nonvertical heat transfer can result in less heat transfer area. Therefore, for such cases the vertical MILP model P4 is not applicable since it will discriminate among multiple global solutions of P2 with the wrong criterion. 

The difficulty that arises in the selection of matches task is due to having multiple feasible combinations of matches which satisfy the minimum number of matches target. Discrimination among them can be achieved only via the vertical MILP model which however assumes the vertical heat transfer criterion. 

The space arrangement of the important feature is also to be considered. Indeed, many SAR studies include length modifications in scaffold side-chains to estimate the influence of flexibility and steric tolerance of the binding site (see, for example, [29, 30]). Chirality of the model is another spatial criterion that is necessary if two enantiomers of different activity level are to be discriminated. At least four points are necessary, but not sufficient to assure enantio-selectivity. The use of directional features (H-bond acceptor/donor) or shape criteria can help in obtaining chiral models. 

In addition to this pharmacophore hypothesis, although it met only three of the four criteria, model 1 from run 6 was retained. Surprisingly, despite criterion number 2 not being satisfied (RMS= 1.62, r=0.79), this model exhibits a remarkable ability to discriminate between active and inactive compounds as assessed by the ROC curve, AUC=0.95. In contrast, model 1 from run 8 has good statistics (RMS=0.76, r= 0.96) but a lower AUC of 0.87. This illustrates that a good model for activity prediction may not be the best for virtual screening applications. Let us analyze these two pharmacophore hypotheses further. 

Note, however, that, in the case of fundamental models, there is not always a need to discriminate among rival models since, often, only a single model has been built up. Furthermore, the best criterion of the quality of a model is the consistency of fundamental parameter estimates with other values obtained by means of several methods under a large range of experimental conditions. Let us not be misled about the principle enemy the systematic errors both in experiments and in reaction and reactor models. 

The various approaches to discriminate between models and to test a given model s adequacies have been treated above, and focus is given here on the design criterion A simple approach can be followed to determine the maximum divergence between model predictions Based on statistical considerations for two models, a simple expression was derived to measure the divergence D(x ) at the experimental settings x of the experimental grid... 

The double summation is used to take each of the models as a reference in order to avoid mislocations of optimal discrimination conditions Since the model adequacy criteria and the design criterion are independent of each other, any type of design criterion can be used An alternative one is [9] given by eq 59... 

The sequential procedures discussed above are appropriate when the investigator wants either to estimate parameters or to discriminate among rival models. However, often an investigator wants to do both. A design procedure for this purpose should emphasize discrimination until one model is gaining favor and should emphasize parameter estimation after that point. The criterion should guard against premature choice of a model, with consequent waste of effort on parameter estimation for an inferior model. 

Buzzi-Ferraris, G., P. Forzatti, and P. Canu, An improved version of a sequential design criterion of discriminating among rival response models, Chem. Eng. Sci., 48, 477-481 (1990). 

Other tests have been used in the past, particularly for aviation gasoline, where it was more important to discriminate accurately between fuels with ON > 100. These had the advantage of being more related to real physical phenomena. For example, the performance number [11] was based on a single standard fuel, iso-octane, and the relative indicated mean effective power (imep) (defined in terms of the cylinder pressure [3]), and so is directly related to combustion. The performance number was 100 times the ratio of the knock limited imeps of the fuel and iso-octane. Much of the API 45 project on octane number of mixtures of pure hydrocarbons (see Section 7.2.5) was reported in terms of performance number. This project of the American Petroleum Institute ran from 1938 to 1957, and has provided an invaluable source of basic data. The articles by Lovell [10] and Scott [12] review and interpret these data. Whilst this criterion and these values of CCRs [10,13] are no longer in widespread use for automotive fuels, the data available in the older literature could still be useful in testing chemical models. Because the octane number scale is based on two reference fuels, modelling the octane number of any hydro-... 

---