Goodness of prediction

Regression analysis includes not only the estimation of model regression parameters, but also the calculation of goodness of fit and -> goodness of prediction statistics, regression diagnostics, residual analysis, and influence analysis [Atkinson, 1985]. 

Statistical indices used to evaluate the performance of classification models [Frank and Todeschini, 1994]. They are derived from two kind of statistics, called - goodness of fit and -> goodness of prediction. 

In order to estimate the predictive capabilities of a model by validation techniques, the data set can be split into different parts the training set (or learning set), the set of objects used for modelling, a test set, the set of objects used to optimize the goodness of prediction of a model obtained from the training set, and the external evaluation set (or evaluation set), which is a new data set used to perform further external validation of the model obtained from the training set. 

The goodness of prediction statistic measures how well a model can be used to estimate future (test) data, e.g. how well a regression model (or a classification model)... 

This constraint is included in the maximization (or minimization) of some goodness of prediction statistic and prevents models with collinearity but without predictive power, i.e. chance correlation, from being taken into account. 

In the equation, n is the number of data points, r is the correlation coefficient, r2 is the goodness of fit, q2 is the leave-one-out cross-validated correlation coefficient expressing the goodness of prediction, x is the standard deviation, and F is the ratio of the variance of the calculated values to that of the observed values. The numbers in parentheses are the 95% confidence intervals. According to equation (20.1), the following structural factors affect the affinity of ligands for hERG ... 

Statistical indices used for the evaluation of the performance of regression models. They are derived from two kinds of statistics, called goodness of fit and goodness of prediction. The former pays more attention to the fitting ability of models, while the latter to the prediction power of models, it being based on validation techniques. 

Besides some special parameters adopted in variable selection, such as Friedman s lack-offit Junction (LOF), the most important indices of goodness of prediction are listed below. 

Model Validation. In a next step, the fit of the model to the experimental data can be evaluated. This can be done by the approaches summarized below. However, in an optimization context, such evaluation is not always performed. The reason is that the model often only needs to predict a value (the optimum) once and is then not used anymore. The goodness of prediction is then usually experimentally verified, and often method optimization stops here. 

Correlation of experimental and calculated activities assesses the quality of 3D-QSAR models. The squared correlation coefficient (r ) yielded by this statistics is a measure of the goodness of fit. The robustness of the model is tested via cross-validation techniques (leave-x%-out), indicating the goodness of prediction q ). Models with > 0.4—0.5 are considered to yield reasonable predictions for hypo-... 

Critical validation of the PLS models is essential. Explained variance (R X) and goodness of fit (R Y) are important parameters but not sufficient. Goodness of prediction (root mean square error [RMSE] or Q ) obtained after cross-validation [22] is essential to avoid overfit and correlations that are simply due to chance. A final validation should be performed by distinguishing between a training set and a test set. The training set is used to create a PLS model, which is subsequently used to predict values obtained from an independent test set. Furthermore, repeatability and reproducibility should be evaluated using repeated analyses and parallel samples. 

The PLS analysis (multivariate calibration) of the fingerprint data of the binary mixtures of crude oil with RME versus the blend matrix resulted in very good PLS models with goodness of fit = 0.98 and goodness of prediction = 0.98. This demonstrates that ESI-MS and PLS can be used to create a regression model for the prediction of FAME in petrodiesel. 

---