Robust statistics

In recent years, the use of robust statistics has become the favoured approach for obtaining sound estimates of the average value and spread of a data set. The advantage of robust statistics is that no rejection of suspect data is required. 

The simplest estimate of a robust average is the median of the data - when data points are arranged in order of magnitude, the median is the middle value 

To calculate the robust standard deviation for this data set, you first have to calculate the absolute difference between each result and the median, x, — median, and then find the median of these values. The median absolute deviation (MAD) is 0.02 %abv. This is converted to a standard deviation equivalent (MADe) by multiplying by 1.483  

For comparison, the mean of the data set is 40.02 %abv and the standard deviation is 0.05 %abv. You can see that the robust standard deviation is substantially smaller than the standard deviation. The use of robust statistics has reduced the influence of the extreme values in the data set. 

The target range for this proficiency testing scheme has been set at 0.03 %abv, based on fitness for purpose criteria. Using an assigned value (X) of 40.04 %abv 

---

The standard deviation, Sj, is the most commonly used measure of dispersion. Theoretically, the parent population from which the n observations are drawn must meet the criteria set down for the normal distribution (see Section 1.2.1) in practice, the requirements are not as stringent, because the standard deviation is a relatively robust statistic. The almost universal implementation of the standard deviation algorithm in calculators and program packages certainly increases the danger of its misapplication, but this is counterbalanced by the observation that the consistent use of a somewhat inappropriate statistic can also lead to the right conclusions. 

An extensive introduction into robust statistical methods is given in Ref. 134 a discussion of non-linear robust regression is found in Ref. 135. An example is worked in Section 3.4. 

Thompson, M., Robust Statistics and Functional Relationship Estimation for Comparing the Bias of Analytical Procedures over Extended Concentration Ranges, Anal. Chem. 61, 1989, 1942-1945. 

These results were evaluated according to the AOAC protocol and ISO 5725-5 robust statistics. Both kits meet the Japanese acceptance criteria. 

Porter PS, Rao ST, Ku J-Y, Poirot RL, Dakins M (1997) Small sample properties of nonparametric bootstrap t confidence intervals. J Air Waste Management Assoc 47 1197-1203 Powell R, Hergt J, Woodhead J (2002) Improving isochron calcttlatiorrs with robust statistics and the bootstrap. Chem Geol 185 191- 204... 

Hampel FR, Roncetti EM, Rousseeuw PJ, Stahel WA (1986) Robust statistics the approach based on influence functions. Wiley, New York... 

In robust statistics, rather than assuming an ideal distribution, an estimator is constructed that will give unbiased results in the presence of this ideal distribution, but that will try to minimize the sensitivity to deviations from ideality. Several approaches are described here ... 

Robust system identification and estimation has been an important area of research since the 1990s in order to get more advanced and robust identification and estimation schemes, but it is still in its initial stages compared with the classical identification and estimation methods (Wu and Cinar, 1996). With the classical approach we assume that the measurement errors follow a certain statistical distribution, and all statistical inferences are based on that distribution. However, departures from all ideal distributions, such as outliers, can invalidate these inferences. In robust statistics, rather than assuming an ideal distribution, we construct an estimator that will give unbiased results in the presence of this ideal distribution, but will be insensitive to deviation from ideality to a certain degree (Alburquerque and Biegler, 1996). 

Quantile probability plots (QQ-plots) are useful data structure analysis tools originally proposed by Wilk and Gnanadesikan (1968). By means of probability plots they provide a clear summarization and palatable description of data. A variety of application instances have been shown by Gnanadesikan (1977). Durovic and Kovacevic (1995) have successfully implemented QQ-plots, combining them with some ideas from robust statistics (e.g., Huber, 1981) to make a robust Kalman filter. 

The use of graphics in one form or another in statistics is the single most effective and robust statistical tool and at the same time, one of the most poorly understood and improperly used. 

The validity of the results is a central issue, and it is confirmed by comparing traditional methods with their robust counterparts. Robust statistical methods are less common in chemometrics, although they are easy to access and compute quickly. Thus, several robust methods are included. 

Peter Filzmoser was bom in 1968 in Weis, Austria. He studied applied mathematics at the Vienna University of Technology, Austria, where he wrote his doctoral thesis and habilitation, devoted to the field of multivariate statistics. His research led him to the area of robust statistics, resulting in many international collaborations and various scientific papers in this area. His interest in applications of robust methods resulted in the development of R software packages. J ( He was and is involved in the organization of several y scientific events devoted to robust statistics. Since... 

Maronna, R., Martin, D., Yohai, V. Robust Statistics Theory and Methods. Wiley, Toronto, ON, Canada, 2006. 

R library (robustbase) base library for robust statistics data (ash,package="chemometrics") load ash-data... 

Accommodation. The philosophy of this strategy is to include the outlying observations in the analysis. Methods are then used to define the final actions which are only slightly influenced by the presence of outliers (Figure le). Such statistical methods are developed under the name of "robust statistics. ... 

Methods for robust statistics have been developed that deliver good results (i.e. estimation of the population mean) even with a relatively large number of outliers or with a skewed distributiom For more detailed descriptions of these methods please refer to the relevant textbooks. 

Robust Statistics use trimmed data for the calculation of the estimated values. That means, that a part of the data set in the tails is excluded or modified prior to or during the calculation. An easy example is the use of the interquartile range (the range between the first and the third quartile) instead of the whole data set. 

In the last years the robust statistical methods got more and more important. In all new relevant standards the use of these methods are now highly recommended. 

The standard deviation is caicuiated after exclusion of outlier or with robust statistics... 

Due to plate-to-plate variations from different days or runs a normalizing step is necessary to render the data comparable across entire screens. We have developed several KNIME nodes for popular normalization methods in HTS such as POC, normalized percentage inhibition (NPI), standard score (z-score), and 5-score (26). For all nodes, robust statistics, grouping, negative control, and parameters can be chosen. The method chosen for normalization is dependent on the screening results and the normality of the data. A fiill discussion on this issue is beyond the scope of this chapter and the reader is referred to excellent reviews (27, 28). 

AMC (2001) Robust statistics a method of coping with outliers. AMC Technical Brief No. 6 (April 2001). Analytical... 

Create a centralized database that would lead to more robust statistical evaluations and improved prediction models (PMs). 

---