Linear regression residual error

Repeat the input identification experiment with the model order MD = 2. Compare the linear regression residual errors for the two cases. Select the "best" model order on the basis of the Akaike Information Criterion (see Section 3.10.3 and ref. 27). ... 

Residual error in linear regression, where the filled circle shows the experimental value/, and the open circle shows the predicted value/,. 

A linear regression analysis should not be accepted without evaluating the validity of the model on which the calculations were based. Perhaps the simplest way to evaluate a regression analysis is to calculate and plot the residual error for each value of x. The residual error for a single calibration standard, r , is given as... 

An approach that is sometimes helpful, particularly for recent pesticide risk assessments, is to use the parameter values that result in best fit (in the sense of LS), comparing the fitted cdf to the cdf of the empirical distribution. In some cases, such as when fitting a log-normal distribution, formulae from linear regression can be used after transformations are applied to linearize the cdf. In other cases, the residual SS is minimized using numerical optimization, i.e., one uses nonlinear regression. This approach seems reasonable for point estimation. However, the statistical assumptions that would often be invoked to justify LS regression will not be met in this application. Therefore the use of any additional regression results (beyond the point estimates) is questionable. If there is a need to provide standard errors or confidence intervals for the estimates, bootstrap procedures are recommended. 

In Equation 5.28, s is a function of the concentration residuals observed during calibration, r is tlie measurement vector for the prediction sample, and R contains the calilxation measurements for the variables used in the model. Because the assumptions of linear regression are often not rigorously obeyed, the statistical pret ion error should be used empirically rather than absolutely. It is useful for validating the prediction samples by comparing the values for... 

Statistical Prediction Errors (Model and Sample Diag Jostic) Uncertainties in the concentrations can be estimated because the predicted concentrations are regression coefficients from a linear regression (see Equations 5.7-5.10). These are referred to as statistical prediction errors to distinguish them from simple concentration residuals (c — c). Tlie statistical prediction errors are calculated for one prediction sample as... 

Figure 2.3. Linear regression analysis with Excel. Simple linear regression analysis is performed with Excel using Tools -> Data Analysis -> Regression. The output is reorganized to show regression statistics, ANOVA residual plot and line fit plot (standard error in coefficients and a listing of the residues are not shown here).

The multiple linear regression models are validated using standard statistical techniques. These techniques include inspection of residual plots, standard deviation, and multiple correlation coefficient. Both regression and computational neural network models are validated using external prediction. The prediction set is not used for descriptor selection, descriptor reduction, or model development, and it therefore represents a true unknown data set. In order to ascertain the predictive power of a model the rms error is computed for the prediction set. 

This will be done by the method of least squares multiple linear regression. This method will estimate the model parameter so that the sum of the squared error terms (residuals), will be as small as possible. How this is done is illustrated by an example. 

FIGURE 47.9 Diagnostic plots used in the model evaluation of the linear regression model that related log(AUC) to log(dose) incorporating subject type (1 for diseased subjects (patient), and 0 for healthy subject) as covariate. Top row The left and right panels are residuals showing that the adequacy of the model fit. Bottom row The left panel plot reinforces the fact that the model adequately describes the data. The right-hand plot shows the adequacy of the error model. 

Uncertainty in the linear regression is estimated by determining the standard errors of adjustable parameters and predictions from the linear model. With a reliable estimate of the variance of the response variable, o, the known value is used for uncertainty predictions. Otherwise, the variance is estimated in terms of the sum of square residuals. The residual at each measurement is defined as... 

Buonaccorsi (1995) present equations for using Option 2 or 3 for the simple linear regression model. In summary, measurement error is not a problem if the goal of the model is prediction, but keep in mind the assumption that the predictor data set must have the same measurement error distribution as the modeling data set. The problem with using option 2 is that there are three variance terms to deal with the residual variance of the model, a2, the uncertainty in 9, and the measurement error in the sample to be predicted. For complex models, the estimation of a corrected a2 may be difficult to obtain. 

Figure 6.1 illustrates this relationship graphically for a 1-compartment open model plotted on a semi-log scale for two different subjects. Equation (6.15) has fixed effects (3 and random effects U . Note that if z = 0, then Eq. (6.15) simplifies to a general linear model. If there are no fixed effects in the model and all model parameters are allowed to vary across subjects, then Eq. (6.16) is referred to as a random coefficients model. It is assumed that U is normally distributed with mean 0 and variance G (which assesses between-subject variability), s is normally distributed with mean 0 and variance R (which assesses residual variability), and that the random effects and residuals are independent. Sometimes R is referred to as within-subject or intrasubject variability but this is not technically correct because within-subject variability is but one component of residual variability. There may be other sources of variability in R, sometimes many others, like model misspecification or measurement variability. However, in this book within-subject variability and residual variability will be used interchangeably. Notice that the model assumes that each subject follows a linear regression model where some parameters are population-specific and others are subject-specific. Also note that the residual errors are within-subject errors. 

As one can see, the ANOVA consists of the regression and residual error (SSe) term. The regression is highly significant, with an of 390.00. The residual error (SSe) is broken into lack-of-fit and pure error. Moreover, the researcher sees that the lack-of-fit component is significant. That is, the linear model is not a precise fit, even though, from a practical perspective, the linear regression model may be adequate. 

We have discussed transforming y and x values to linearize them, as well as removing effects of serial correlation. But transformations can also be valuable in eliminating nonconstant error variances. Unequal error variances are often easily determined by a residual plot. For a simple linear regression, y = ho + hi JCi e, the residual plot will appear similar to Figure 8.8, if a constant variance is present. 

For one jc, predictor variable, in cases of simple linear regression, ej equals the squared residual (j, — y), as always. Let SSr, equal the sum of squares regression on the ej vs. the x,-. That is, the value, (y, - y) — ej, is used as the y, or predictor value in this test. The e, values are squared, and a simple linear regression is performed to provide the SS/ term. The SSg term is the sum of squares error of the original equation, where ef is not used as the dependent variable. The Chi-Square test statistic tabled value, o degree of... 

From the standpoint of statistics, the transformation Eq. 2.2-19 into Eq. 2.3.b-l and the determination of the parameters from this equation may be criticized. What is minimized by linear regression are the (residuals) between experimental and calculated y-values. The theory requires the error to be normally distributed. This may be true for r, but not necessarily for the group /(Pa - PrPs/I Wa and this may, in principle, affect the values of k, K, K, Ks, — However, when the rate equation is not rearranged, the regression is no longer linear, in general, and the minimization of the sum of squares of residuals becomes iterative. Search procedures are recommended for this (see Marquardt [41]). It is even possible to consider the data at all temperatures simultaneously. The Arrhenius law for the temperature dependence then enters into the equations and increases their nonlinear character. 

Estimates of linear regression coefficients, standard errors of coefficients, significance of coefficients, blocking of variables, residual calculation and residual analysis, standard ANOVA, weighted least-squares analysis... 

---