Outliers ordering

Note on GMPs The assays are conducted on individual dosage units (here tablets) and not on composite samples. The CU test serves to limit the variability from one dosage unit to the next (the Dissolution Rate test is the other test that is commonly used). Under this premise, outlier tests would be scientific nonsense, because precisely these outliers contain information on the width of the distribution that one is looking for. The U.S. vs. Barr Laboratories Decision makes it illegal to apply outlier tests in connection with CU and DR tests. This does not mean that the distribution and seemingly or truly atypical results should not be carefully investigated in order to improve the production process. 

To construct the reference model, the interpretation system required routine process data collected over a period of several months. Cross-validation was applied to detect and remove outliers. Only data corresponding to normal process operations (that is, when top-grade product is made) were used in the model development. As stated earlier, the system ultimately involved two analysis approaches, both reduced-order models that capture dominant directions of variability in the data. A PLS analysis using two loadings explained about 60% of the variance in the measurements. A subsequent PCA analysis on the residuals showed that five principal components explain 90% of the residual variability. 

The measured values are sorted in ascending or descending order, depending on whether the suspected outlier value yf deviates to higher or lower values. The test statistic is formed depending on the data set... 

After inspecting the tabular and graphic data, the operator is allowed to remove runs which appear to be outliers. Any run can be deleted or restored in any order, and the comparative statistics are recalculated with each operation. By comparing the standard deviation before and after deleting a run, the effect of that run can by determined. The editing process can continue indefinitely until the operator is satisfied with the validity of his results. 

A multivariate normal distribution data set was generated by the Monte Carlo method using the values of variances and true flowrates in order to simulate the process sampling data. The data, of sample size 1000, were used to investigate the performance of the robust approach in the two cases, with and without outliers. 

Robust system identification and estimation has been an important area of research since the 1990s in order to get more advanced and robust identification and estimation schemes, but it is still in its initial stages compared with the classical identification and estimation methods (Wu and Cinar, 1996). With the classical approach we assume that the measurement errors follow a certain statistical distribution, and all statistical inferences are based on that distribution. However, departures from all ideal distributions, such as outliers, can invalidate these inferences. In robust statistics, rather than assuming an ideal distribution, we construct an estimator that will give unbiased results in the presence of this ideal distribution, but will be insensitive to deviation from ideality to a certain degree (Alburquerque and Biegler, 1996). 

Let us consider the same chemical reactor as in Example 11.1 (Chen et al., 1998). Monte Carlo data for y were generated according to in order to simulate process sampling data. A window size of 25 was used here, and to demonstrate the performance of the robust approach two cases were considered, with and without outliers. 

A robust measure for the central value—much less influenced by outliers than the mean—is the median xm (x median). The median divides the data distribution into two equal halves the number of data higher than the median is equal to the number of data lower than the median. In the case that n is an even number, there exist two central values and the arithmetic mean of them is taken as the median. Because the median is solely based on the ordering of the data values, it is not affected by extreme values. 

They were also typical when the regression model chosen was first order. Mean-level bandwidths greater than 20-30% are probably indicative that errors have been made in the analysis process that should not be tolerated. In this case techniques would be carefully scrutinized to find errors, outliers, or changing chromatographic conditions. These should be remedied and the analysis repeated whenever possible. Certain manipulation can be done to reduce the bandwidth values. For example, they would be... 

In order to check the results of the analysis, K-Nearest Neighbor distances were computed for the scaled data set Including the cadmium results. The median of the distances from a given laboratory to the three nearest neighbors ranged from 0.26 to 1.24 with the median distance between members of the cluster (1,2,3,5,6,7) equal to 0.79. The median distances of Laboratories 4 and 8 from members of this cluster were 1.24 and 1.22, respectively, supporting the view that these laboratories are outliers. 

As noted in the last section, the correct answer to an analysis is usually not known in advance. So the key question becomes How can a laboratory be absolutely sure that the result it is reporting is accurate First, the bias, if any, of a method must be determined and the method must be validated as mentioned in the last section (see also Section 5.6). Besides periodically checking to be sure that all instruments and measuring devices are calibrated and functioning properly, and besides assuring that the sample on which the work was performed truly represents the entire bulk system (in other words, besides making certain the work performed is free of avoidable error), the analyst relies on the precision of a series of measurements or analysis results to be the indicator of accuracy. If a series of tests all provide the same or nearly the same result, and that result is free of bias or compensated for bias, it is taken to be an accurate answer. Obviously, what degree of precision is required and how to deal with the data in order to have the confidence that is needed or wanted are important questions. The answer lies in the use of statistics. Statistical methods take a look at the series of measurements that are the data, provide some mathematical indication of the precision, and reject or retain outliers, or suspect data values, based on predetermined limits. 

GRUBBY TEST for rejection of an observation is applied in order to determine if one of the observations should be rejected as being an outlier. The following equation was used for the test ... 

Equation (4.20) was proposed by Hoskuldsson [65] many years ago and has been adopted by the American Society for Testing and Materials (ASTM) [59]. It generalises the univariate expression to the multivariate context and concisely describes the error propagated from three uncertainty sources to the standard error of the predicted concentration calibration concentration errors, errors in calibration instrumental signals and errors in test sample signals. Equations (4.19) and (4.20) assume that calibrations standards are representative of the test or future samples. However, if the test or future (real) sample presents uncalibrated components or spectral artefacts, the residuals will be abnormally large. In this case, the sample should be classified as an outlier and the analyte concentration cannot be predicted by the current model. This constitutes the basis of the excellent outlier detection capabilities of first-order multivariate methodologies. 

Tables Va, b, and c attempt to answer these questions. In Table Va percent copper is ranked in increasing order along with the laboratory numbers and the methods. First, are any of the values outliers Should all the values be taken as representative of the copper content, or are some in error Assuming that the data are normally distributed (i.e., that they follow the Gaussian or bell-curve distribution) the V test can be applied to reject anomalous values this was done, and four values were rejected. Three come from one laboratory, and the fourth came from a laboratory which did not detect or take Zn into account on sample 3. Therefore, their calculation for copper was too high.

Let us illustrate the benefits of higher order on a concrete analytical example measurements of concentration of Mg2+ with an ISE and with an optical sensor. After linearization of the potentiometric signal, the two experiments can be displayed as a bilinear plot (Fig. 10.2). Contained in this plot is an unusual sample point S, which clearly falls out of the linear correlation because it lies outside the statistically acceptable 3a noise level. This outlier is an indication of the presence of an interferant. Its presence is clearly identified in this bilinear plot from combined ISE and optical measurement, although it would be undetected in a first-order sensor alone. 

Fig. 10.2 Bilinear plot obtained from second-order sensor. In this example, linearized ISE response is plotted against absorbance from fiberoptic sensor. Point S represents an outlier ...

The training and test sample errors (Fig. 45.9) are visualized in /View/Example Errors or clicking the errors icon. This graphic allows to identify outliers in the case they exist, presenting the samples ordered according to their relative error and highlighting the appropriate bar in red if it surpasses the previously defined error limit. 

---