Outlier testing

Over the years an abundance of outlier tests have been proposed that have some theoretical rationale at their roots. ° Such tests have to be carefully adjusted to the problem at hand because otherwise one would either not detect true outliers (false negatives) in every case, or then throw out up to 50% of the good measurements as well (false positivesj. o Robust methods have been put forward to overcome this. Three tests will be described ... 

The situation of the pharmaceutical industry is today governed by the Barr ruling. The foregoing suggestions concerning the use of outlier tests are expressly aimed only at those users and situations not subject to the Barr ruling. 

Since the 1993 court decision against Barr Laboratories, 5 tjjg elimination of outliers has taken on a decidedly legal aspect in the U.S. (any non-U.S. company that wishes to export pharmaceuticals or preciwsor products to the U.S. market must adhere to this decision concerning out-of-specifica-tion results, too) the relevant section states that ... An alternative means to invalidate an individual OOS result... is the (outlier test). The court placed specific restrictions on the use of this test. (1) Firms cannot frequently reject results on this basis, (2) The USP standards govern its use in specific areas, (3) The test cannot be used for chemical testing results. ... A footnote explicitly refers only to a content uniformity test, 5 but it appears that the rule must be similarly interpreted for all other forms of inherently precise physicochemical methods. For a possible interpretation, see Section 4.24. 

Regarding the residuals, many an investigator would be tempted to cast out outliers the reader is advised to consult Section 1.5.5. If values are grouped (i.e. several values y, are measured at the same x), outlier tests can be applied to the individual group, however, blind reliance on a rule, such as y ,ean 2 5j, is Strongly discouraged. 

Note on GMPs The assays are conducted on individual dosage units (here tablets) and not on composite samples. The CU test serves to limit the variability from one dosage unit to the next (the Dissolution Rate test is the other test that is commonly used). Under this premise, outlier tests would be scientific nonsense, because precisely these outliers contain information on the width of the distribution that one is looking for. The U.S. vs. Barr Laboratories Decision makes it illegal to apply outlier tests in connection with CU and DR tests. This does not mean that the distribution and seemingly or truly atypical results should not be carefully investigated in order to improve the production process. 

Example 53 If the standard deviation before elimination of the purported outlier is not much higher than the upper CLf method), as in the case = 0.358 < CL(/(0.3) 0.57 factor Chu/sx 1.9 for = 9, see program MSD), an outlier test should not even be considered both for avoiding fruitless discussions and reducing the risk of chance decisions, the hurdle should be set even higher, say at p < 0.01, so that CLu/sx > 2.5. 

With small data sets or if there is reason to suspect deviations from the Gaussian distribution, a robust outlier test should be used. 

The FDA did not include outlier tests in the USP for chemical assays, but allowed the practice for biological tests. The reason for this could be that because of the high precision, n is usually small in chemical testing with n < 3, outlier tests cannot be conducted. It appears that Judge Wolin followed this recommendation when deliberating his decision. 

Since the U.S. vs. Barr decision in 1993 (relevant to pharmaceuticals and related fields, rules applied by the Federal Food Drug Administration, FDA), outlier tests may no longer be applied to physicochemical tests, under the assumption that such test methods, having been optimized and validated for the particular set of circumstances, rarely produce outliers. These tests may not be applied to CU results at all. Good manufacturing practices mandate that operators work according to pre-set procedures and write down any observed irregularities as they... 

MAD median absolute deviation, Huber s outlier test... 

Bolton, S., Outlier Tests and Chemical Assays, Appendix V in Pharmaceutical Statistics, Practical and Clinical Applications, Bolton, Sanford, 3rd ed., Marcel Dekker, New York, 1997. 

As explained in Section 33.2.1, one can prefer to consider each class separately and to perform outlier tests to decide whether a new object belongs to a certain class or not. The earliest approaches, introduced in chemometrics, were called SIMCA (soft independent modelling of class analogy) [27] and UNEQ [28]. 

As explained already, SIMCA can be applied as an outlier test, similarly to the multivariate QC tests referred to earlier. Feam et al. [44] have described certain properties of SIMCA in this respect and compared it with some alternatives. 

Other authors used a simple 2 standard deviation criteria or an outlier test (F-test) to check for significant differences between within-bottle and between-bottle results (Martin-Esteban et al. 1997 Quevauviller et al. 1995). The degree of homogeneity of elements and compounds in the materials tested in these studies does not seem to be adequately described and, hence, the asigned uncertainties in the mean values may represent only the bias between the analytical methods used in the certification. 

G Test statistic (estimate) of Grubb s outlier test (4.36)... 

Ga,n Significance limits of Grubb s outlier test for a risk of error a and n individual measurements ... 

The AO AC (cited in Ref. [11]) described the precision acceptance criteria at different concentrations within or between days, the details of which are provided in Table 2. Other parameters that should be tested in the precision study are the David-, Dixon- or Grubbs-, and Neumann-tests. The David-test is performed when determining whether the precision data are normally distributed. Outlier testing of the data is performed by the Dixon-test (if n < 6-8) or by the Grubbs-test (if n > 6-8), while trend testing of the data is performed by Neumann-test. Detailed methods have been described in the book written by Kromidas [29]. 

It is important to appreciate that the statistical significance of the results is wholly dependent on the quality of the data obtained from the trial. Data that contain obvious gross errors should be removed prior to statistical analysis. It is essential that participants inform the trial co-ordinator of any gross error that they know has occurred during the analysis and also if any deviation from the method as written has taken place. The statistical parameters calculated and the outlier tests performed are those used in the internationally agreed Protocol for the Design, Conduct and Interpretation of Collaborative Studies.14... 

We must not accept outliers in a calibration data set. We also can test this with an statistical outlier test. 

Thereby we have to consider that the outlier test assumes the chosen approach for the regression function to be correct. First we should have a look on the plot of the residual analysis, because from there we can recognise potential outliers. We calculate the regression both with and without the potential outlier. Then we can apply either the F-test or the t-test... 

The outlier test assumes that the chosen regression approach is correct... 

If a consensus value is used as the assigned value there are different possibilities to calculate it. If the arithmetic mean is used, an outlier test is required. But in many eases these tests are not very satisfaetoiy, espeeially if several outliers are present. If the tests are strietly used, they ean only be apphed to normally distributed data, whieh is usually not the case in trace analysis. 

Outlier tests assume normal distribution which is normally not true in trace analysis... 

Some statistical tests are specific for evaluation of normality (log-normality, etc., normality of a transformed variable, etc.), while other tests are more broadly applicable. The most popular test of normality appears to be the Shapiro-Wilk test. Specialized tests of normality include outlier tests and tests for nonnormal skewness and nonnormal kurtosis. A chi-square test was formerly the conventional approach, but that approach may now be out of date. 

If the data as a whole appear normally distributed but there is concern that an extreme point is an outlier, it is not necessary to apply the Rankit procedure. The Grubbs s outlier test (1950) is now recommended for testing single outliers, replacing Dixon s Q-test. After identifying a single outlier, which, of course, must be either the maximum or minimum data value, the G statistic is calculated ... 

The normality of the distribution of results is checked by an appropriate test, such as the Kolmogorov-Smirnov test, and outlier tests are performed on... 

As mentioned in the previous section, analytical methods intended for interlaboratory use should be subjected to a collaborative trial using the lUPAC protocol. This is a prescriptive procedure and outlier testing/removal is mandatory. Before the process begins the data need to be examined to ensure that only valid data are input to the calculation process. This process is best shown as a flow chart (Figure 36). 

Table A2 Critical values for Grubbs extreme deviation outlier tests ...

Nowadays the DIXON test alone is recommended by standardizing organisations [ISO 5725]. The general formula for the DIXON outlier test is ... 

Finally we should mention here that any distribution tends to produce outliers. Hence, our advice is to perform parameter tests before outlier tests and deletion. For normal distribution we can perform simple tests of skewness and excess using the /-test Skewness is tested against the critical value ... 

---