Robust regression methods

These and other robust techniques are sure to find increasing use in analytical chemistry in future. One area where their use is already recommended is in interlaboratory comparisons (see Chapter 4), and many of the methods discussed in previous chapters, such as regression (see below) and ANOVA, can be robustified with the aid of suitable software. 

The problems caused by possible outliers in regression calculations have been outlined in Sections 5.13 and 6.9, where rejection using a specified criterion and non-parametric approaches respectively have been discussed. It is clear that robust approaches will be of value in regression statistics as well as in the statistics of repeated measurements, and there has indeed been a rapid growth of interest in robust regression methods amongst analytical scientists. A summary of two of the many approaches developed must suffice. 

In Section 6.9 it was noted that a single suspect measurement has a considerable effect on the a and b values calculated for a straight line by the normal least squares method, which seeks to minimize the sum of the squares of the y-residuals. This is 

---

Typical robust regression methods are linear methods. 

The method is not as useful with multisubstrate systems however, Cornish-Bowden and EndrenyE have presented a robust regression method to treat data with more parameters. 

However, no book on experimental design of this scope can be considered exhaustive. In particular, discussion of mathematical and statistical analysis has been kept brief Designs for factor studies at more than two levels are not discussed. We do not describe robust regression methods, nor the analysis of correlations in responses (for example, principle components analysis), nor the use of partial least squares. Our discussion of variability and of the Taguchi approach will perhaps be considered insufficiently detailed in a few years. We have confined ourselves to linear (polynomial) models for the most part, but much interest is starting to be expressed in highly non-linear systems and their analysis by means of artificial neural networks. The importance of these topics for pharmaceutical development still remains to be fully assessed. 

Least squares multi-linear regression is by far the most common method for estimating "best values" of the coefficients, but it is not the only method, and is not always the best method. So-called "robust" regression methods exist and may be useful. These reduce the effect on the regression line of outliers, or apparently aberrant data points. They will not be discussed here, and least squares regression is used in the examples throughout this book. 

It is quite rare for there to be enough data to show deviations from normality. Least squares regression is usually adequate and cases requiring "robust" regression methods are infrequent. 

Finally we note that, just as in the treatment of outliers in replicate measurements, non-parametric and robust methods can be very effective in handling outliers in regression robust regression methods have proved particularly popular in recent years. These topics are covered in the next chapter. 

The quantity Y (at the points x ) shows a normal distribution (see Fig. 2, left) and is outlier-free (the latter situation is tested with the Dixon test [13], [21] if the assumption is not confirmed, a robust regression method can be used, for example)... 

S. C. Rutan and P. W. Carr, Anal. Chim. Acta, 215, 131 (1989). Comparison of Robust Regression Methods Based on Least-Median and Adaptive Kalman Filtering Approaches Applied to Linear Calibration Data. 

An extensive introduction into robust statistical methods is given in Ref. 134 a discussion of non-linear robust regression is found in Ref. 135. An example is worked in Section 3.4. 

Hoskuldsson A (1988) PLS regression methods. Chemom 2 211 Huber PJ (1981) Robust statistics. Wiley, New York... 

For robust regression, the objective function is changed. While for OLS regression, the sum of all squared residuals is minimized in robust regression, another function of the residuals is minimized. Three methods for robust regression are mentioned here ... 

QSPR models have been developed by six multivariate calibration methods as described in the previous sections. We focus on demonstration of the use of these methods but not on GC aspects. Since the number of variables is much larger than the number of observations, OLS and robust regression cannot be applied directly to the original data set. These methods could only be applied to selected variables or to linear combinations of the variables. 

The aim of multivariate calibration methods is to determine the relationships between a response y-variable and several x-variables. In some applications also y is multivariate. In this chapter we discussed many different methods, and their applicability depends on the problem (Table 4.6). For example, if the number m of x-variables is higher than the number n of objects, OLS regression (Section 4.3) or robust regression (Section 4.4) cannot be applied directly, but only to a selection... 

Outliers or inhomogeneous data can affect traditional regression methods, hereby leading to models with poor prediction quality. Robust methods, like robust regression (Section 4.4) or robust PLS (Section 4.7.7), internally downweight outliers but give full weight to objects that support the (linear) model. Note that to all methods discussed in this chapter robust versions have been proposed in the literature. 

Shapiro Wilks W-test for normal data Shapiro Wilks W-test for exponential data Maximum studentlzed residual Median of deviations from sample median Andrew s rho for robust regression Classical methods of multiple comparisons Multivariate methods... 

Vanden Branden, K. and Hubert, M., Robustness properties of a robust PLS regression method, Anal. Chim. Acta, 515, 229-241, 2004. 

Inspired by the work of statisticians, chemists have also placed the use of the robust methods on the agenda. Phillips and Eyring [8] were among the first ones who applied an M-estimator, the biweight function of Tukey, in regression analysis of analytical data. They concluded that the efficiency of the robust regression was about the same or superior to the least squares... 

Also non-linear regression, that is, using quadratic terms such as (logP) and cross terms, may be used. However, as described in detail, there are a number of pitfalls to this method. A statistically more robust method which could be used instead of MLR is the PLS regression method. 

---