Errors in the Fitted Parameters

As there is a difference between the measurements and the values of the calculated function, we can safely assume that the fitted parameters are not perfect. They are our best estimates for the true parameters and an obvious question is, how reliable are these fitted parameters Are they tightly or they are loosely defined As long as the assumption of random white noise applies, there are formulas that allow the computation of the standard deviation of the fitted parameters. While these answers should always be taken with a grain of salt, they do give an indication of how well defined the parameters are. 

We are not going to derive the formulas that allow the calculation of the standard deviations of the parameters. The reader is invited to refer to more specialised texts on statistics. We just give the formulas and also give ways of calculating the required information. In equation (4.32) the standard deviation of the parameter aj is given. 

Here m is the number of points in y and np the number of fitted parameters. The difference m-np is the number of degrees of freedom, df The elements djj in equation (4.32) are the diagonal elements of the inverse of the so-called curvature matrix, Curv, that contains the second derivatives of the sum of squares with respect to the parameters. The definition of the element Curvjk is 

This looks horrendous, but as will be shown in a moment it is not. In fact it is trivial to compute. We start with the first derivative 

The complete set of first and second derivatives can be written elegantly in matrix notation  

---

This gives the standard errors in the fitted parameters a. 

The errors in the fitting parameters may be obtained from the covariance matrix of the fit if it is available, but they are more commonly estimated by varying one parameter away from its optimal value while optimizing all other parameters until a defined increase in the statistical function is obtained. However, the statistical error values obtained do not represent the true accuracies of the parameters. In fact, it is difficult to determine coordination numbers to much better than 5%, and 20% is more realistic when the data are collected at room temperature taking into account the strong coupling between the coordination number and Debye Waller terms, the error in the latter may be 30%. 

The variation with temperature of n and for an undoped GaAs crystal grown by the horizontal Bridgman method is shown in Fig. 3. The parameters that give the best fit to Eq. (12) are also shown. The power of this method is illustrated by the small probable errors in the fitted parameters, i.e., less than 15% for Nd and less than 1% for ED0. Very few techniques can lay claim to such accuracy. Note that for maximum reliability it is necessary to know the Hall r factor [Eq. (A 17)], since n = r/eR. A variational calculation, with NAS as the only undetermined parameter, was used to fit the fi versus T data, and also obtain r versus T (Meyer and Bartoli, 1981 Look et al., 1982a). 

Set X = 0 and compute the estimated covariance matrix C = a . This gives the standard errors in the fitted parameters a. 

As before, the standard error (Tpi in the fitted parameters pi can be estimated from the expression... 

Note that the standard errors in the rate constants (kx = 2.996 0.005 x 10-3 s 1 and 2 = 1.501 0.002 x 10 3 s ) are delivered in addition to the standard deviation (<7y = 9.991 x 10 3) in Y. The ability to directly estimate errors in the calculated parameters is a distinct advantage of the NGL/M fitting procedure. Furthermore, even for this relatively simple example, the computation times are already faster than using a simplex by a factor of five. This difference dramatically increases with increasing complexity of the kinetic model. 

Fig 5. Temperature and area curves extracted from time-resolved spectra of statically-detonated Type A Small munitions. The periodic error bars reflect die standard error in die fit parameter. 

A critical component of comodeling multiple outputs is the appropriate weighting of individual observations. The weights must be appropriate for small and large responses within an output and the relative weights must be appropriate between outputs. Failure of the former standard can lead to regions of systematic error in the fitted function and failure in the latter standard can cause some of the outputs to inappropriately dominate the determination of fitted parameters. However, error variance model selection, as for structural model development, should be guided by parsimony stay as simple as possible. 

The error in the relaxation parameters consisted of a) experimental errors (about 2 %), b) an error ascribable to the chosen fitting function (estimated to be about 5 %) and c) uncertainties of the fitting (about 0.5 %). Repeated experiments using the same sample showed that the standard deviation of the results was smaller than 2-3 %. 

For this purpose, we reanalyze all the available static EOS data for Th, as shown in table 1, with a set of three different EOS forms, and compare the effect of the different EOS forms with the effects resulting from different data sets. As EOS forms we use the Birch equation (Birch 1978) in second order, BE2, and two recently proposed forms (Holzapfel 1990,1991) in second-order form, H02 and HI2, which are related to the Thomas-Fermi theory and are distinguished by the fact that H12 is bound to approach the Fermi-gas limit at infinite compression. A close inspection of table 1 shows very clearly that most of the data are fitted almost equally well by any of these forms, without any significant difference in the fitted parameters Kq and K g or in the minimized standard deviation of the pressure, Tp. In contrast to many other publications, table 1 presents the unrestricted standard deviations of Kq and K, which correspond to the extreme values of the error ellipsoids presented in fig. 11, and not just to the width of the error ellipsoids along and K at the center points, which are usually given as (restricted) statistical errors. Thus, it becomes obvious that... 

To overcome possible errors in the derived parameters when fitting NMR spectra of powdered samples with large quadrupolar interactions, Perras et al. developed and implemented an exact treatment of the quadrupolar and Zeeman interactions for stationary powdered samples [9]. The software, QUEST (QUadrupolar Exact SofTware), is freely available and produces spectral simulations in a fraction of a second on a modem computer [10]. The importance of this exact treatment was demonstrated in a SSNMR study of a series of organic compounds featuring covalent... 

Let us assume that there are N data points xy i/i), Xy 1/2)/- / ( n/ 3/n)- We seek a function of X that closely represents the data. Let/(x) be a guess of the function. It should be a function that has one or more adjustable parameters that constitute the set c,. For any particular data point, the error in the fit is the difference between y and/(x) for that point. If we were to sum the errors at each of the points, we would have a poor measure of the quality of the guess function. There could be one point where/(x) is much greater than the y value, and another point where f(x) is much less than the y value. The sum of these two errors might be zero, and that would not reflect the fact that f x) is not representing these two points well. For this reason, it is the square of the errors (always positively valued) at each point that should be used to measure the error. The total of the squared errors shall be designated E, and so... 

The last word about the meaning of the intensity parameters has not been said yet, and further research is necessary. A problem for a rigorous comparison is the relatively large error in the Q), parameter (more than 5% or sometimes up to 10-20%). Parameter sets for the same system, but determined by different authors, may be substantially different. This is partially due to the fact that the values of the parameters depend on the transitions chosen for the fitting procedure (as the standard least-squares fitting procedure is chosen). Comparison between fits is only possible if the chi-square method is used to determine the parameters (Goldner and Auzel 1996). 

Listing 9.11 shows flie code changes to Listing 9.6 for this example. The best fit values of the parameters as shown in the selected ou ut of the listing and have a very small standard error. This arises because the model fits very accurately the data and there is very little random error associated with the data points. In fact almost all the fitting error occurs in the two data points with the smallest time value. The RMS error in the fitting is only 0.0019 miits along the dependent variable axis. With a model fliat provides an excellent functional fit to the data and... 

For each experiment, the true values of the measured variables are related by one or more constraints. Because the number of data points exceeds the number of parameters to be estimated, all constraint equations are not exactly satisfied for all experimental measurements. Exact agreement between theory and experiment is not achieved due to random and systematic errors in the data and to "lack of fit" of the model to the data. Optimum parameters and true values corresponding to the experimental measurements must be found by satisfaction of an appropriate statistical criterion. 

If there is sufficient flexibility in the choice of model and if the number of parameters is large, it is possible to fit data to within the experimental uncertainties of the measurements. If such a fit is not obtained, there is either a shortcoming of the model, greater random measurement errors than expected, or some systematic error in the measurements. 

In many process-design calculations it is not necessary to fit the data to within the experimental uncertainty. Here, economics dictates that a minimum number of adjustable parameters be fitted to scarce data with the best accuracy possible. This compromise between "goodness of fit" and number of parameters requires some method of discriminating between models. One way is to compare the uncertainties in the calculated parameters. An alternative method consists of examination of the residuals for trends and excessive errors when plotted versus other system variables (Draper and Smith, 1966). A more useful quantity for comparison is obtained from the sum of the weighted squared residuals given by Equation (1). 

This sum, when divided by the number of data points minus the number of degrees of freedom, approximates the overall variance of errors. It is a measure of the overall fit of the equation to the data. Thus, two different models with the same number of adjustable parameters yield different values for this variance when fit to the same data with the same estimated standard errors in the measured variables. Similarly, the same model, fit to different sets of data, yields different values for the overall variance. The differences in these variances are the basis for many standard statistical tests for model and data comparison. Such statistical tests are discussed in detail by Crow et al. (1960) and Brownlee (1965). 

Using the data from Fig. 19-11, the calculated cr from the CWI concentration is 239 m from the observed peak concentration it is 232 m and from the fitted peak concentration it is 235 m. Note that errors in any ofthe parameters H, Q, or u, will cause errors in the estimated [Pg.314]

---