Regression assumptions

The multiple linear response variables (j,s) are assumed statistically independent of one another. As in simple linear regression, when data are collected in a series of time intervals, the researcher must be cautious of serial or autocorrelation. The same basic procedures described in Chapter 2 must be followed, as is discussed later. 

The variance cr of y is considered constant for any fixed combination of x, predictor variables. In practice, the assumption is rarely satisfied completely, and small departures usually have no adverse influence on the performance and validity of the regression model. 

Additionally, it is assumed that, for any set of predictor values, the corresponding y,s are normally distributed about the regression plane. This is a requirement for general inference making, e.g., confidence intervals, prediction of y, etc. The predictor variables, x,s, are also considered independent of each other, or additive. Therefore, the value of Xj does not, in any way, affect or depend on X2, if they are independent. This is often not the case, so the researcher must check and account for the presence of interaction between the predictor x, variables. 

The general multiple linear regression model for a first-order model, that is, when all the predictor variable x,s are linear, is 

As additional x, predictor variables are added to a model, interaction among them is possible. That is, the x, variables are not independent, so as one builds a regression model, one wants to measure and account for possible interactions. 

---

Many of the regression assumptions are not obeyed when analyaing chemical data (e.g., spectroscopic data). In these cases the statistical output can be misleading and should only be used for qualitative assessment. 

Autocorrelated variables and autocorrelated errors, which occur frequently in time series analysis, violate the general regression assumption of uncorrelated errors. 

While the nature of the error structure of the measurements is often ignored or understated in electrochemical impedance spectroscopy, recent developments have made possible experimental identification of error structure. Quantitative assessment of stochastic and experimental bias errors has been used to filter data, to design experiments, and to assess the validity of regression assumptions. 

In contrast, the data in the top plot of Fig. 4.2 using a constant residual variance model led to the following parameter estimates after fitting the same model volume of distribution =10.2 0.10L, clearance = 1.49 0.008 L/h, and absorption rate constant = 0.71 0.02 per h. Note that this model is the data generating model with no regression assumption violations. The residual plots from this analysis are shown in Fig. 4.4. None of the residual plots show any trend or increasing variance with increasing predicted value. Notice that the parameter estimates are less biased and have smaller standard errors than the estimates obtained from the constant variance plus proportional error model. 

If the standard regression assumptions are met, and the same number of replicates is taken at each x, value, the standardized, the Studentized, and jackknife residuals look the same. Outliers are often best identified by the jackknife residual, for it makes suspect data more obvious. For example, if the /th residual observation is extreme (hes outside the data pool), the, ) value will tend to be much smaller than which will make the r(, ) value larger in comparison to Sr the Studentized residual. Hence, the r(, ) value will stand out for detection. 

Classical Nonlinear Regression Assumptions with Application in a Population Pharmacokinetic Setting... 

The updatfng formulas. Under the normal linear regression assumptions, the least squares estimates maximize the likelihood function. This makes them the maximum likelihood estimates and their covariance matrix the eovariance matrix of the maximum likelihood estimates. Thus the posterior has the multivariate normal where the constants are found by the updating formulas "the posterior precision matrix equals the sum of the prior precision matrix plus the precision matrix of the "maximum likelihood estimates"... 

Provided that each class is tight and occupies a small and separate volume in X space, one can find a plane (a discriminant plane) in which the projected observations are well separated according to class. If the X variables are few and independent (i.e., the regression assumptions are fulfilled), one can derive this discriminant plane by means of multiple regression with X and a dummy matrix Y that expres.ses the class-belonging of the training set observations. This dummy... 

If we want to calculate the full spectrum of scattered light then we find that the inclusion of these off-shell events prevents us from using the quantum fluctuation regression theory. To calculate spectra the dipole autocorrelation must be calculated directly without the use of any regression assumptions. This is an interesting example of where a microscopic calculation of a correlation function can be calculated for a system driven from thermal equilibrium. 

Multiple linear regression is strictly a parametric supervised learning technique. A parametric technique is one which assumes that the variables conform to some distribution (often the Gaussian distribution) the properties of the distribution are assumed in the underlying statistical method. A non-parametric technique does not rely upon the assumption of any particular distribution. A supervised learning method is one which uses information about the dependent variable to derive the model. An unsupervised learning method does not. Thus cluster analysis, principal components analysis and factor analysis are all examples of unsupervised learning techniques. 

Understanding the distribution allows us to calculate the expected values of random variables that are normally and independently distributed. In least squares multiple regression, or in calibration work in general, there is a basic assumption that the error in the response variable is random and normally distributed, with a variance that follows a ) distribution. 

The most commonly used form of linear regression is based on three assumptions (1) that any difference between the experimental data and the calculated regression line is due to indeterminate errors affecting the values of y, (2) that these indeterminate errors are normally distributed, and (3) that the indeterminate errors in y do not depend on the value of x. Because we assume that indeterminate errors are the same for all standards, each standard contributes equally in estimating the slope and y-intercept. For this reason the result is considered an unweighted linear regression. 

The example in Figure 3 is as complex as is usually possible to analyze. There are seven unknowns, if no indices of refracdon are being solved for in the regression analysis. If correlation is a problem, then a less complex model must be assumed. For example, the assumption that and are each fixed at a value of 0.5 might reduce correlation. The five remaining unknowns in the regression analysis would then be and 3. In practice one first assumes the simplest possible model,... 

Parameters, errors and deviations are given in Tables 7-9 and one representative plot for every correlation is shown by Figures 2, 3 and 4. Moreover, data of Tables 7 and 8 indicates that both b and f is very small positive or negative number which equals to zero within the range of the experimental errors. To verify this assumption, fittings were repeated using linear regression without intercept (b, d=0)... 

Table 4.12. Accepted and Target Impurity Concentrations (Target Concentrations for Impurities, Under Assumption of the Regression Line in Fig. 4.7 (B a = 0.92, b = -0.743, m = 1) If the LOQ of the Method were 0.03%, the Target Concentration in the Last Line (0.011) Would he Inaccessible to Measurement ...

Calibration Each of the solutions is injected once and a linear regression is calculated for the five equidistant points, yielding, for example, Y = -0.00064 + 1.004 X, = 0.9999. Under the assumption that the software did not truncate the result, an r of this size implies a residual standard deviation of better than 0.0001 (-0.5% CV in the middle of the LO range use program SIMCAL to confirm this statement ) the calibration results are not shown in Fig. 4.39. 

A first evaluation of the data can be done by running nonparametric statistical estimation techniques like, for example, the Nadaraya-Watson kernel regression estimate [2]. These techniques have the advantage of being relatively cost-free in terms of assumptions, but they do not provide any possibility of interpreting the outcome and are not at all reliable when extrapolating. The fact that these techniques do not require a lot of assumptions makes them... 

Another classification technique is logistic regression [76], which is based on the assumption that a sigmoidal dependency exists between the probability of group membership and one or more predictor variables. It has been used [72] to model eye irritation data. 

Once soil samples have been analyzed and it is certain that the corresponding results reflect the proper depths and time intervals, the selection of a method to calculate dissipation times may begin. Many equations and approaches have been used to help describe dissipation kinetics of organic compounds in soil. Selection of the equation or model is important, but it is equally important to be sure that the selected model is appropriate for the dataset that is being described. To determine if the selected model properly described the data, it is necessary to examine the statistical assumptions for valid regression analysis. 

There are two statistical assumptions made regarding the valid application of mathematical models used to describe data. The first assumption is that row and column effects are additive. The first assumption is met by the nature of the smdy design, since the regression is a series of X, Y pairs distributed through time. The second assumption is that residuals are independent, random variables, and that they are normally distributed about the mean. Based on the literature, the second assumption is typically ignored when researchers apply equations to describe data. Rather, the correlation coefficient (r) is typically used to determine goodness of fit. However, this approach is not valid for determining whether the function or model properly described the data. 

---