Model goodness of fit

Principle 4 points to performing statistical validation to establish the performance of the model. Goodness-of-fit and robustness refer to the internal model performance while predictivity refers to the external model validation. 

Weight is treated as an a priori covariate, regardless of whether its inclusion improves the model goodness of fit, for two reasons. First, almost without exception, studies in pediatric populations have shown weight to affect clearance and volume of distribution terms. Second, extrapolations outside the weight range used to develop the model tends to lead to unrealistic predictions. 

The most common measure of model goodness-of-fit is chi-square/degrees-of-freedom (CMIN/DF). The value of (CMIN/DF) ratio should not exceed 5. AMOS provides some additional goodness-of-fit indices researchers can use it to resolve the problem of chi-square biasness to large sample size and normality issue (Abdel Hamid 2013). 

In many process-design calculations it is not necessary to fit the data to within the experimental uncertainty. Here, economics dictates that a minimum number of adjustable parameters be fitted to scarce data with the best accuracy possible. This compromise between "goodness of fit" and number of parameters requires some method of discriminating between models. One way is to compare the uncertainties in the calculated parameters. An alternative method consists of examination of the residuals for trends and excessive errors when plotted versus other system variables (Draper and Smith, 1966). A more useful quantity for comparison is obtained from the sum of the weighted squared residuals given by Equation (1). 

Both func tions are tabulated in mathematical handbooks (Ref. I). The function P gives the goodness of fit. Call %q the value of at the minimum. Then P > O.I represents a believable fit if ( > 0.001, it might be an acceptable fit smaller values of Q indicate the model may be in error (or the <7 are really larger.) A typical value of for a moderately good fit is X" - V. Asymptotic Iy for large V, the statistic X becomes normally distributed with a mean V ana a standard deviation V( (Ref. 231). 

Model discrimination is a procedure for developing a suitable description of the unit performance. The techniques are drawn from the mathematics hterature where the goodness-of-fit of various proposed models are compared. Unfortunately, the various proposed models will usually describe a unit s performance equally well. Model discrimination is better accomplished when raw or adjusted measurements from many, unique operating conditions provide the foundation for the comparisons. 

Usually goodness of fit is provided by adding new parameters in the model, but it decreases the prediction capability of the retention model and influences on the optimization results of mobile phase composition. 

Different tests for estimation the accuracy of fit and prediction capability of the retention models were investigated in this work. Distribution of the residuals with taking into account their statistical weights chai acterizes the goodness of fit. For the application of statistical weights the scedastic functions of retention factor were constmcted. Was established that random errors of the retention factor k ai e distributed normally that permits to use the statistical criteria for prediction capability and goodness of fit correctly. 

The process of curve fitting utilizes the sum of least squares (denoted SSq) as the means of assessing goodness of fit of data points to the model. Specifically, SSq is the sum of the differences between the real data values (yd) and the value calculated by the model (yc) squared to cancel the effects of arithmetic sign ... 

A good model is consistent with physical phenomena (i.e., 01 has a physically plausible form) and reduces crresidual to experimental error using as few adjustable parameters as possible. There is a philosophical principle known as Occam s razor that is particularly appropriate to statistical data analysis when two theories can explain the data, the simpler theory is preferred. In complex reactions, particularly heterogeneous reactions, several models may fit the data equally well. As seen in Section 5.1 on the various forms of Arrhenius temperature dependence, it is usually impossible to distinguish between mechanisms based on goodness of fit. The choice of the simplest form of Arrhenius behavior (m = 0) is based on Occam s razor. 

The fit with H= 1.53 is quite good. The results for the fits with n = 1 andn = 2 show systematic deviations between the data and the fitted model. The reaction order is approximately 1.5, and this value could be used instead of n= 1.53 with nearly the same goodness of fit, a = 0.00654 versus 0.00646. This result should motivate a search for a mechanism that predicts an order of 1.5. Absent such a mechanism, the best-fit value of 1.53 may as well be retained. 

Table 1.26. Results of a x -Test for Testing the Goodness-of-fit of a Model...

As noted earlier, the x -test for goodness-of-fit gives a more balanced view of the concept of fit than does the pure least-squares model however, there is no direct comparison between x and the reproducibility of an analytical method. 

The weighting model with which the goodness-of-fit or figure-of-merit (GOF = E(m,)) is arrived at can take any of a number of forms. These continuous functions can be further modified to restrict the individual contributions M, to a certain range, for instance r, is minimally equal to the expected experimental error, and all residuals larger than a given number r ax are set equal to rmax- The transformed residuals are then weighted and summed over all points to obtain the GOF. (See Table 3.5.)... 

The quantitation of the goodness of fit between a model and a data set by calculation of the residual standard deviation. 

Interpretation The model can only be improved upon if the residual standard deviation remains significantly larger (F-test ) than the experimental repeatability (standard deviation over many repeat measurements under constant conditions, which usually implies within a short period of time ). Goodness of fit can also be judged by glancing along the horizontal (residual = 0) and looking for systematic curvature. 

An example of the goodness of fit between measured residual monomer levels the optimized model predictions is shown in Figure 3. Model predicted values corresponding with measured residual monomer data for all five experimental runs are given in Table I. 

Models should be assessed not only in terms of their goodness of fit (i.e., statistical quality) but also in terms of their predictive power. The predictive power of a model can be assessed only by estimating the activity of a set of compounds not included in the original model. 

In QSAR equations, n is the number of data points, r is the correlation coefficient between observed values of the dependent and the values predicted from the equation, is the square of the correlation coefficient and represents the goodness of fit, is the cross-validated (a measure of the quality of the QSAR model), and s is the standard deviation. The cross-validated (q ) is obtained by using leave-one-out (LOO) procedure [33]. Q is the quality factor (quality ratio), where Q = r/s. Chance correlation, due to the excessive number of parameters (which increases the r and s values also), can. 

There are two statistical assumptions made regarding the valid application of mathematical models used to describe data. The first assumption is that row and column effects are additive. The first assumption is met by the nature of the smdy design, since the regression is a series of X, Y pairs distributed through time. The second assumption is that residuals are independent, random variables, and that they are normally distributed about the mean. Based on the literature, the second assumption is typically ignored when researchers apply equations to describe data. Rather, the correlation coefficient (r) is typically used to determine goodness of fit. However, this approach is not valid for determining whether the function or model properly described the data. 

A new idea has recently been presented that makes use of Monte Carlo simulations [60,61], By defining a range of parameter values, the parameter space can be examined in a random fashion to obtain the best model and associated parameter set to characterize the experimental data. This method avoids difficulties in achieving convergence through an optimization algorithm, which could be a formidable problem for a complex model. Each set of simulated concentration-time data can be evaluated by a goodness-of-fit criterion to determine the models that predict most accurately. 

Two issues present themselves when the question of PB-PK model validation is raised. The first issue is the accuracy with which the model predicts actual drug concentrations. The actual concentration-time data have most likely been used to estimate certain total parameters. Quantitative assessment, via goodness-of-fit tests, should be done to assess the accuracy of the model predictions. Too often, model acceptance is based on subjective evaluation of graphical comparisons of observed and predicted concentration values. 

Goodness-of-fit tests may be a simple calculation of the sum of squared residuals for each organ in the model [26] or calculation of a log likelihood function [60], In the former case,... 

Frequency domain performance has been analyzed with goodness-of-fit tests such as the Chi-square, Kolmogorov-Smirnov, and Wilcoxon Rank Sum tests. The studies by Young and Alward (14) and Hartigan et. al. (J 3) demonstrate the use of these tests for pesticide runoff and large-scale river basin modeling efforts, respectively, in conjunction with the paired-data tests. James and Burges ( 1 6 ) discuss the use of the above statistics and some additional tests in both the calibration and verification phases of model validation. They also discuss methods of data analysis for detection of errors this last topic needs additional research in order to consider uncertainties in the data which provide both the model input and the output to which model predictions are compared. 

---