Evaluation of the Model

Frazao C, C Topham, V Dhanaraj and T L Blundell 1994. Comparative Modelling of Human Rer Retrospective Evaluation of the Model with Respect to the X-ray Crystal Structure, Pure and A Chemistry 66 43-50. 

This phase is intended as a final check of the model as a whole. Testing of individual model elements should be conducted during earlier phases. Evaluation of the model is carried out according to the evaluation criteria and test plan established in the problem definition phase. Next, carry out sensitivity testing of the model inputs... 

An important point is the evaluation of the models. While most methods select the best model at the basis of a criterion like adjusted R2, AIC, BIC, or Mallow s Cp (see Section 4.2.4), the resulting optimal model must not necessarily be optimal for prediction. These criteria take into consideration the residual sum of squared errors (RSS), and they penalize for a larger number of variables in the model. However, selection of the final best model has to be based on an appropriate evaluation scheme and on an appropriate performance measure for the prediction of new cases. A final model selection based on fit-criteria (as mostly used in variable selection) is not acceptable. 

In PAT, one is often faced with the task of building, optimizing, evaluating, and deploying a model based on a limited set of calibration data. In such a situation, one can use model validation and cross-validation techniques to perform two of these functions namely to optimize the model by determining the optimal model complexity and to perform preliminary evaluation of the model s performance before it is deployed. There are several validation methods that are commonly used in PAT applications, and some of these are discussed below. 

To date, the use of computational methods to investigate iminium ion catalysis has been limited. The focus has been on rationalising the diastereo- and enantiose-lectivities observed in the laboratory, but this has largely been retrospective and the clear potential of these models as predictive tools for the design of improved catalysts or even entirely new scaffolds has yet to be realised. There are few examples of solid kinetic data in the literature making evaluation of the models difficult. 

The PARROT programme uses the Poly-3 subroutine in Thermo-Calc to calculate Gibbs energies of the various phases and find the equilibrium state. In such equilibrium calculations the temperature, pressiue and chemical potentials are treated as independent variables, and preselected state variables are used to define the conditions for an equilibrium calculation. The dependent state variables, i.e., the responses to the system, can then be given as a function of the independent state variable and the model parameters. It is thus possible to use almost any type of experimental information in the evaluation of the model parameters. 

In these relations, Ki denotes the equilibrium constant of reaction step i. For the numerical evaluation of the model, it is assumed that the backward reaction of step lb has the same transition state as the transition state for the re-desorption of A2 in Model 1, and that the entropy of the molecular precursor on the surface is negligible. The results are shown in Figure 4.37. It is observed that the model predicts that catalysts of much larger reactivity (more negative AEt) will be optimal for reactions where the diatomic molecule is strongly bound to the surface before the dissociation. 

This shift of the use forced a different evaluation of the model, toward a more statistical evaluation. The first QSAR models were evaluated in their capability to fit the property data with one or more chemical descriptors, but no proof was given about the predictivity of the model. Today, a number of criteria are requested to check if a model is predictive or not [13-15],... 

Correction of the sequence alignment based on die evaluation of the model Iteration of the last three steps... 

When additional data are available, a QS AR model should be validated by predicting the activity of other chemicals not used in the training set, but whose activities are known (i.e., the test set). This is called external validation. The major difference between the cross-validation and external validation is that the chemicals selected in the latter case are in a sense random. This provides a more robust evaluation of the model s predictive capability for untested chemicals than cross-validation. We feel strongly that the confidence in a model s predictive capability can be tested and validated when robust prediction has been demonstrated with an external test set. Further details regarding a formal framework for the validation of QSARs are provided in Chapter 20. 

Phenanthrene Solubilization. A model characterizing the distribution of HOC in systems of soil and micellar nonionic surfactant solution was described previously (7). In this model HOC is assumed to partition among three distinct compartments the soil, the micellar pseudophase, and the aqueous pseudophase. The solubilization model accounts for the partitioning of HOC between the micellar pseudophase and the aqueous pseudophase, the increase in apparent HOC solubility associated with nonionic surfactant monomers in the aqueous pseudophase, the sorption of surfactant onto soil, and the increase in fractional organic carbon content of a soil as a result of surfactant sorption. Evaluation of the model with experimental data was described by Edwards et al. (12). 

Zangenberg, N.H., Mullertz, A., Gjelstrup Kristensen, H. and Hovgaard, L. (2001) A dynamic in vitro lipolysis model. II. Evaluation of the model. European Journal of Pharmaceutical Sciences, 14 (3), 237. 

K nothing is known and/or if there are a large number of experimental variables to consider, it is advisable to start with a linear model. It is always possible to run a second, complementary set of experiments to estimate also the interaction coefficients. The advantage of starting with a linear model is, that it may be possible to remove certain variables from further consideration after a first evaluation of the model. If it is still necessary to determine interaction effects, the complementary set of experiment will be smaller after the elimination of insignificant variables. 

Zangenberg NH, MullertzA, Kristensen HG, and Hovgaard L. A Dynamic in Vitro Lipolysis Model 11 Evaluation of the Model. EurJPhann Sci2001h 14 237-244. 

Data analysis method Stating the software, model building procedures, model diagnostics, structural model, covariate model, stochastic model, and sensitivity analysis to be used and how the evaluation of the model is to be conducted... 

Calculation of the Number of Chromophores. One important parameter in the model is the number of chromophores. In previous studies [132, 175] this number was always used as an adjustable parameter in the model. In our opinion it seemed to be more correct to calculate the chromophore number independently. This would result in a better use and evaluation of the model. The most important property of the triazene polymer is the absorption maximum around 330 nm which was assigned to the triazene chromophore in structurally similar compounds [119, 140]. 

Frazao, C., Topham, C., Dhanaraj, V. and Blundell, T.L. (1994) Comparative modelling of human renin a retrospective evaluation of the model with respect to the X-ray crystal structure. Pure Appl. Chem. 66 43-50. 

---