Weighted sum of squared residuals

We will now investigate the sampling properties of the statistic representing the weighted sum of squared residuals i1 given by equation (5.4.13). We first observe that the slightly different expression (y — l)rSy i(y —is zero since... 

This discussion of mathematical modeling is limited to methods based on the assumption of error in the y (dependent) term only. The objective or minimization function will generally be restricted to the sum of the weighted residuals between the observed and calculated data, weighted sum of squared residuals (WSS). The objective of the mathematical modeling approaches is to adjust the parameter values so that a minimum value of the WSS is achieved. 

Fitting the model to the observed data is an important task. Each mathematical model studied consists of independent and dependent variables, constants (possibly), and parameters that have to be estimated. The objective is to reduce the overall difference between the observed data and the calculated points by adjusting the values of the parameters. As mentioned earlier, the methods to be discussed assume that there is no error in the independent variable. Also, in general, the criteria of best fit will be the weighted sum of squared residuals between the observed and calculated data, the WSS. The chosen model can be validated in part by accurately describing the observed data. 

There are several techniques for minimization of the sum of squared residuals described by Eq. (7.160). We review some of these methods in this section. The methods developed in this section will enable us to fit models consisting of multiple dependent variables, such as the one described earlier, to multiresponse experimental data, in order to obtain the values of the parameters of the model that minimize the overall weighted) sum of squared residuals. In addition, a thorough statistical analysis of the regression results will enable us to... 

In the previous four sections, the sum of squared residuals that was minimized was that given by Eq. (7.160). This was the sum of squared residuals determined from fitting one equation to measurements of one variable. However, most mathematical models may involve simultaneous equations in multiple dependent variables. For such a case, when more than one equation is fitted to multiresponse data, where there are v dependent variables in the model, the weighted sum of squared residuals is given by... 

The experimental data in Table E7.1 were obtained from two penicillin fermentation runs conducted at essentially identical operating conditions. Using the Marquardt method, fit the above two equations to the experimental data and determine the values of the parameters I , b2, 3, and K which minimize the weighted sum of squared residuals. 

For multiple regression, the variances are used in Eq. (7.180) to determine the unbiased weighting factors, Wj which are in turn used in Eq. (7.176) to determine the unbiased weighted sum of squared residuals. In the case where repeated experimental data are available, the program searches for repeated experimental points and evaluates the variance of each dependent variable by dividing the sum of squared differences by the number of degrees of freedom of each dependent variable (see Table 7.2). The degrees of freedom of each variable... 

The simplest procedure is merely to assume reasonable values for A and to make plots according to Eq. (2-52). That value of A yielding the best straight line is taken as the correct value. (Notice how essential it is that the reaction be accurately first-order for this method to be reliable.) Williams and Taylor have shown that the standard deviation about the line shows a sharp minimum at the correct A . Holt and Norris describe an efficient search strategy in this procedure, using as their criterion minimization of the weighted sum of squares of residuals. (Least-squares regression is treated later in this section.)... 

In implicit estimation rather than minimizing a weighted sum of squares of the residuals in the response variables, we minimize a suitable implicit function of the measured variables dictated by the model equations. Namely, if we substitute the actual measured variables in Equation 2.8, an error term arises always even if the mathematical model is exact. 

In this case we minimize a weighted sum of squares of residuals with constant weights, i.e., the user-supplied weighting matrix is kept the same for all experiments, Q,=Q for all i=l,...,N and Equation 3.7 reduces to... 

The best regression was determined using a minimization routine incorporated into the program which minimized the sum of squared residuals between the calculated and observed EXAFS oscillation, x and % The result was visually checked comparing the k and k weighted Fourier transforms of the regression and contributions of each regressed shell to the acquired data. 

Given N measurements of the output vector, the parameters can be obtained by minimizing the Least Squares (LS) objective function which is given below as the weighted sum of squares of the residuals, namely. 

In weighted least squares, we combine the different measurements by forming the weighted sum of the residuals. Let et be the residual vector at the ith sample time... 

The weight ratio (or ratios) may be refined, but it is oft i easier to refine structural parameters for various fixed ratios and compare the resulting sum of squared residuals. The accuracy of the ratio determination may be estimated by the factor test (c/. see p. 54). In practice it is necessary to assume that some of the parameters are equal in both (or all) conformers. A typical example is the investigation of CH2=CH-S-CH . Two conformers were found, one syn form with a planar skeleton, and a second form which seems to have a non-planar skeleton. The bond lengths and bond angles were assumed to be independent of the torsional angle about the >C-S- bond, though it is rather likely that some parameters e.g. 

Sum of squared, weighted residuals °F Number of degrees of freedom... 

If, however, the standard deviations, ayij, for all elements of the matrix Y are known or can be estimated reliably, it does make sense to use this information in the data analysis. Then, instead of the sum of squares, it is the sum of all appropriately weighted and squared residuals that has to be minimised. This is known as the chi-square or x2 -fitting. If the data matrix Y has the dimensions nsxnl, %2 is defined by... 

Figure 4-64. The result of the %2-Tit (left) and sum-of-squares fit (right). Note the uniformly distributed weighted residuals for the %2 fit.

Least squares (LS) estimation minimizes the sum of squared deviations, comparing observed values to values predicted by a curve with particular parameter values. Weighted LS (WLS) can take into account differences in the variances of residuals generalized LS (GLS) can take into account covariances of residuals as well as differences in weights. Cases of LS estimation include the following ... 

Time series without systematic changes (trend or seasonal fluctuations), i.e. with a fixed level, are best approximated by the mean of the series, i.e. a = 0. The mean over the full time range gives a minimum sum of squared differences between the mean and the original series (squared residues). All cases have the same weight, because a is equal to zero. 

The system identification step in the core-box modeling framework has two major sub-steps parameter estimation and model quality analysis. The parameter estimation step is usually solved as an optimization problem that minimizes a cost function that depends on the model s parameters. One choice of cost function is the sum of squares of the residuals, Si(t p) = yi(t) — yl(t p). However, one usually needs to put different weights, up (t), on the different samples, and additional information that is not part of the time-series is often added as extra terms k(p). These extra terms are large if the extra information is violated by the model, and small otherwise. A general least-squares cost function, Vp(p), is thus of the form... 

When possible, one should also test the residual sum of squares against its predictive probability distribution. If the weighted residuals are distributed in the manner assumed in Section 6.1, then the standardized sum of squares = 5/cr has the predictive probability density... 

Our choice of model differs from that of Tschernitz et al. (1946), who preferred Model d over Model h on the basis of a better fit. The difference lies in the weightings used. Tschernitz et al. transformed each model to get a linear least-squares problem (a necessity for their desk calculations) but inappropriately used weights of 1 for the transformed observations and response functions. For comparison, we refitted the data with the same linearized models, but with weights Wu derived for each model and each event according to the variance expression in Eq. (6.8-1) for In 7. The residual sums of squares thus found were comparable to those in Table 6.5. confirming the superiority of Model h among those tested. 

---