Mean absolute relative error

In discussing the Hartree-Fock electronic reaction enthalpies, it should be noted that the errors plotted in Figures 15.15 and 15.16 are absolute errors. The relative errors are more acceptable, as may be seen from Table 15.31, where we have decomposed the CCSD(T) enthalpies into contributions from the Hartree-Fock determinant, from the singles and doubles amplitudes, and from the triples amplitudes. Disregarding the weakly exothermic reaction Rl, the mean absolute relative error in the Hartree-Fock enthalpies is about 10%, and the largest error of 21% occurs for the ozone reaction R17. Also, except for R8 and R9, the Hartree-Fock model predicts the... 

Fig. 4 Left the mean 1961-1990 monthly temperature for the Ebro catchment. Part (a) shows the annual cycle, each line representing a different RCM simulation and the bold line representing the CRU observed series. The shading represents the 95% confidence interval for the estimate of the observed 30-year sample mean. Part (b) represents the individual monthly model means as an anomaly from the CRU mean with 95% confidence interval superimposed. Part (c) represents the mean absolute annual error for each of the RCMs. Right-, as for left column but for mean precipitation (d) for the Gallego catchment. Model anomalies in parts (e) and (f) are expressed as a percentage relative to the CRU monthly mean. Model numbers correspond to experiments shown in Table 1. Figure from [35]...

Figure 3.2 The effect of prolonged subcutaneous implantation on biosensor function. Blood glucose values shown in solid circles and glucose sensor values in the continuous lines. The early study (top panel), but not the late study (bottom), shows excellent sensor accuracy and minimal lag between blood glucose and sensed glucose values. MARD (mean absolute relative difference) refers to a sensor accuracy metric. EGA refers to the Clarke error grid analysis accuracy metric.

MAD, mean absolute difference MARD, mean absolute relative difference SD, standard deviation SE, standard error SRE standard relative error. 

The different manufacturers publish their own results in their user manuals. Mean absolute relative difference and bias results from the three manufacturers are shown in Table 5.2. The MARD measures indicates the average difference while the direction of the difference and the bias indicates if the differences are uniform or skewed to positive or negative values. The Clarke error grid analysis for the three manufacturers (Table 5.3) shows a wide difference in A zone results between the... 

Table 19.2 Mean absolute percent errors of (p ) for 78 molecules relative to coupled cluster QCISD values in the cc-pVTZ basis set...

The mean absolute percent error MAPE, which is 9.99 less than 10, indicates that the forecasted values are in a high accuracy. Besides, Thell Inequality Coefficient TIC is 0.057, and Bias Proportion BP 0.059 and Variance Proportion VP 0.005 are relatively small while Covariance Proportion CP 0.935 is relatively large. All these parameters suggest that the result of forecasting is desirable. See the Figure 2. 

Accuracy, which comprehends actual measures like the Mean Error (ME), Mean Absolute Error (MAE), Mean Squared Error (MSE), and Measures relative to a Perfect Forecast like the Percent Error (PE) and the Mean Absolute Percent Error (MAPE). 

Mean absolute percentage error (MAPE) MAPE measures the relative dispersion of forecast errors and is given by... 

Determine the density at least five times, (a) Report the mean, the standard deviation, and the 95% confidence interval for your results, (b) Eind the accepted value for the density of your metal, and determine the absolute and relative error for your experimentally determined density, (c) Use the propagation of uncertainty to determine the uncertainty for your chosen method. Are the results of this calculation consistent with your experimental results ff not, suggest some possible reasons for this disagreement. 

Note d and d denote mean and mean absolute errors, respectively, relative to the experiment, m, c, and R2 denote the gradient, intercept, and correlation parameters, respectively, of the correlation plots, relative to the experiment. 

Table 1.12 Statistical measures of errors for extrapolated CCSD(T) AEs relative to experiment (kJ/mol). A is the mean error, Astd is the standard deviation around the mean error, Aabs is the mean absolute error, and Amax is the maximum absolute error.

The operation of Eq. (3.3) is illustrated by the results given in Table 2 out of 48 molecules of the cc-pVTZ set. They are listed in order of increasing correlation energy. The first column of the table lists the molecule. The next 6 columns show how many orbitals and orbital pairs of the various types are in each molecule, i.e. the numbers Nl, Nb, Nu, Nlb etc. The seventh column lists the CCSD(T)/triple-zeta correlation energy and the eight column lists the difference between the latter and the prediction by Eq. (3.3). The mean absolute deviation over the entire set of cc-pVTZ data set is 3.14 kcal/mol. For the 18 molecules of the CBS-limit data set it is found to be 1.57 kcal/mol. The maximum absolute deviations for the two data sets are 11.29 kcal/mol and 4.64 kcal/mol, respectively. Since the errors do not increase with the size of the molecule, the errors in the estimates of the individual contributions must fluctuate randomly within any one molecule, i. e. there does not seem to exist a systematic error. The relative accuracy of the predictions increases thus with the size of the system. It should be kept in mind that CCSD(T) results can in fact deviate from full Cl results by amounts comparable to the mean absolute deviation associated with Eq. (3.3). 

In classical statistics, the most important type of criterion for judging estimators is a high probability that a parameter estimate will be close to the actual value of the parameter estimated. To implement the classical approach, it is necessary to quantify the closeness of an estimate to a parameter. One may rely on indices of absolute, relative, or squared error. Mean squared error (MSB) has often been used by statisticians, perhaps usually because of mathematical convenience. However, if estimators are evaluated using Monte Carlo simulation it is easy to use whatever criterion seems most reasonable in a given situation. 

A comparison of calculated and experimental anion geometries are provided in Table 5-16. Included are Hartree-Fock models with STO-3G, 3-21G, 6-31G and 6-311+G basis sets, local density models, BP, BLYP, EDFl and B3LYP density functional models and MP2 models, all with 6-31G and 6-311+G basis sets, and MNDO, AMI and PM3 semi-empirical models. Experimental bond lengths are given as ranges established from examination of distances in a selection of different systems, that is, different counterions, and mean absolute errors are relative to the closest experimental distance. 

Data are provided in Table 6-10, with the same calculation models previously examined for hydrogenation reactions. As might be expected from the experience with hydrogenation reactions, Hartree-Fock models with 6-3IG and 6-311+G basis sets perform relatively well. In fact, they turn in the lowest mean absolute errors of any of the models examined. The performance of density functional models (excluding local density models) and MP2 models with both 6-3IG and 6-311+G basis sets is not much worse. On the other hand, local density models yield very poor results in all cases showing reactions which are too exothermic. The reason is unclear. Semi-empirical models yield completely unacceptable results, consistent with their performance for hydrogenation reactions. 

Calculated relative energies for a small selection of structural isomers are compared with experimental values and with the results of G3 calculations in Table 6-11. These have been drawn from a much more extensive set of comparisons found in Appendix A6 (Tables A6-24 to A6-31). Mean absolute errors from the full set of comparisons are collected in Table 6-12, and a series of graphical comparisons involving Hartree-Fock, EDF1, B3LYP and MP2 models... 

Table 6-12 Mean Absolute Errors in Relative Energies of Structural Isomers... 

Better accounts of relative isomer energies are provided by density functional models and by MP2 models. With both 6-3IG and 6-311+G basis sets, BP, EDFl and MP2 models perform best and BLYP models perform worst, although the differences are not great. In terms of mean absolute errors, all models improve upon replacement of the 6-3 IG by the 6-311+G basis set. With some notable exceptions, individual errors also decrease in moving from the 6-31G to 6-311+G basis sets. (A further breakdown of basis set effects is provided in Tables A6-32 to A6-35 in Appendix A6.) The improvements are, however, not great in most cases, and it may be difficult to justify of the extra expense incurred in moving from 6-3IG to the larger basis set. 

With the exception of STO-3G and both MP2 models, all models (including semi-empirical models) provide a credible account of relative CH bond energies. In terms of mean absolute error, BP and B3LYP models with the 6-311+G basis set are best and Hartree-Fock 3-21G and 6-3IG models, local density 6-3IG models and semi-empirical models are worst. More careful scrutiny turns up sizeable individual errors which may in part be due to the experimental data. For example, the best of the models appear to converge on a CH bond dissociation for cycloheptatriene which is 35-37 kcal/mol less than that in methane (the reference compound) compared with the experimental estimate of 31 kcal/mol. It is quite possible that the latter is in error. The reason for the poor performance of MP2 models, with individual errors as large as 16 kcal/mol (for cycloheptatriene) is unclear. The reason behind the unexpected good performance of all three semi-empirical models is also unclear. 

With the exception of semi-empirical models, all models provide very good descriptions of relative nitrogen basicities. Even STO-3G performs acceptably compounds are properly ordered and individual errors rarely exceed 1 -2 kcal/mol. One unexpected result is that neither Hartree-Fock nor any of the density functional models improve on moving from the 6-3IG to the 6-311+G basis set (local density models are an exception). Some individual comparisons improve, but mean absolute errors increase significantly. The reason is unclear. The best overall description is provided by MP2 models. Unlike bond separation energy comparisons (see Table 6-11), these show little sensitivity to underlying basis set and results from the MP2/6-3IG model are as good as those from the MP2/6-311+G model. 

In terms of mean absolute error, choice of reactant and transition-state geometry has very little effect on calculated relative activation energies. Nearly perfect agreement between calculated and experimental relative activation energies is found for 6-3IG calculations, irrespective of whether or not approximate geometries are employed. Somewhat larger discrepancies are found in the case of MP2/6-31G calculations, but overall the effects are small. 

The relative error ER on a measurement (or on the mean value) corresponds to the ratio of the absolute value of the deviation e, (or e ) over the real value. ER can be expressed in % or ppm. 

---