Protocol outliers

The second perspective is closer to the mark Measurements that apparently do not fit model or experience should always be investigated in the light of all available information. While there is the distinct possibility of a discovery about to be made, the other outcome of a sober analysis of the circumstances is more probable a deviation from the experimental protocol. If this is documented, all the better the probable outlier can, in good conscience, be rejected and replaced by a reliable repeat result. 

It is important to appreciate that the statistical significance of the results is wholly dependent on the quality of the data obtained from the trial. Data that contain obvious gross errors should be removed prior to statistical analysis. It is essential that participants inform the trial co-ordinator of any gross error that they know has occurred during the analysis and also if any deviation from the method as written has taken place. The statistical parameters calculated and the outlier tests performed are those used in the internationally agreed Protocol for the Design, Conduct and Interpretation of Collaborative Studies.14... 

System installation in a permanent location may require a sample conditioning system featuring some degree of automation, such as automatic cleaning (the system illustrated above features such a system) and outlier sample collection and the need to interface to an existing control system process computer. The latter may require that the system operates with a standardized communications protocol, such as Modbus, for the chemical industry. Certain specialized industries use different protocols, such as the semiconductor industry, which uses SECS and SEC-11 protocols. A standardized approach designated the Universal Fieldbus is another method/protocol for process analyzers which is being supported by certain hardware manufacturers. 

For example, a single estimate for total PCB s has been historically collected in the NHATS program. Current advances in chemical analysis protocols now allow for the determination of isomer specific resolution of PCB s. Given the 209 PCB s that are now possible to detect, an adequate evaluation of the data without the use of pattern recognition techniques seems impossible. From a QA/QC perspective, these methods can facilitate the detection of outliers and aid in the interpretation of human chemical residue data. The application of statistical analysis must keep abreast with these advances made in chemisty. To handle the complexity and quantity of such data, the use of more sophisticated statistical analyses is needed. 

If no procedure for dealing with outliers was foreseen in the trial protocol, one analysis with the actual values and at least one other analysis eliminating or reducing the outlier effect should be performed and differences between their results discussed. ... 

The calculations for the Youden matched pairs procedure are less complicated than those for the lUPAC protocol and do not involve outlier detection and removal. For the example, the results are shown in Table 27. 

As mentioned in the previous section, analytical methods intended for interlaboratory use should be subjected to a collaborative trial using the lUPAC protocol. This is a prescriptive procedure and outlier testing/removal is mandatory. Before the process begins the data need to be examined to ensure that only valid data are input to the calculation process. This process is best shown as a flow chart (Figure 36). 

Eighteen spectroscopic samples paired with fingerstick reference measurements were collected from an individual over 2 days with a time-randomized protocol. A c alibration model was built by fitting a line through the plot of INU versus reference glucose concentration. The model was then applied to 31 samples collected over the next 14 weeks (7 additional samples from the same individual and 24 from different individuals). After rejecting 11 outliers ( 35%), they obtained a correlation coefficient (r) of 0.8 and a standard deviation of 1.2mM. 

Violations of protocol must be identified and characterized, and some member of the team should be dispatched to give the needed explanations and corrective actions. Note especially adherence to enrollment criteria, the drug administration regimen, including the duration of washout periods (if any), patient compliance, and the specified clinical and laboratory observations. The periodic computer checks can identify the site(s), if any, that are outliers from the average of other sites. 

The provision of six samples to seven laboratories might seem to be excessive, but in several preliminary round-robin intercomparison measurements, outliers (results differing from the mean by many standard deviations) had turned up, and could only be discarded with certainty because there were enough other results with which to compare (24), If only two or three laboratories had produced those results, it would have been difficult or impossible to identify the outlying data. The whole basis of the protocol was the expressed desire of the religious authorities to produce the best possible scientific result and the effort of a number of 14C scientists to meet that desire. 

Bosque-Sendra, J.M. Nechar, M. Cuadros Redriguez, L. Decision protocol for checking robustness with previous outlier detection in the validation of analytical methods. Fresenius J. Anal Chemie 1999, 365, 480-488. 

A data quality plan is a tool to aid in the implementation of data quality. The plan should be developed as soon as the protocol is finalized. Data quality is a shared responsibility across all functions. For example, the monitor assures quality by source document verification (SDV), and the clinician reviews listings of individual patient profiles and study outliers. ... 

The data were assessed by the ISO 5725-2 protocol [3] implemented in the software package Prolab98 (Dr Uhlig, Quo Data, Dresden, Germany), which is routinely used by the German Federal Environmental Agency for evaluation of laboratory proficiency tests. Outliers were rejected by use of Grubbs test (P=l%) and Cochran s test (P=l%). 

For quantitative methods at least five materials must be included, a requirement which can be met by a combination of samples and levels of analyte. At least eight laboratories must take part and follow the interlaboratory study protocol. For reason of possible outliers, it is recommended that more than this number of laboratories should take part. 

An erroneous value can be traced to a gross deviation from the prescribed procedure for establishment of reference values. Such values may either deviate significantly from the proper reference values (outliers) or be hidden in the reference distribution. Only a strict experimental protocol, with adequate controls at each step, can eliminate the latter type of erroneous values. 

Methods for identifying and handling of possible outlier data should be specified in the protocol. Medical or pharmacokinetic explanations for such observations should be sought and discussed. As outliers may be indicative of product failure, post hoc deletion of outlier values is generally discouraged. An approach to dealing with data containing outliers is to apply distribution-free (non-parametric), statistical methods (72). 

The data should be reported as specified in the protocol with the requested significant figures. Valid data (those free of gross errors and produced following the protocol) should be submitted to various statistical treatment for outlier detection of mean and variance, and an ANOVA treatment to establish the repeatability and reproducibility figures. All these treatments and their sequence are specified in the lUPAC protocol [2]. The final report should contain all individual and statistical data additional graphical representation e.g. Youden-plots, bar-graphs etc may also be added. 

In the development of an electroanalytical protocol, comparisons need to be made with a well-established and accepted accurate or gold standard procedure. The two important statistical analysis methods to allow such comparisons are the t-test and F-t st. Before the application of these, the g-test needs to be performed, that is, the obtained data should be tested for potential outliers (anomalies), which are data values which appear to be unreasonably distant (or outlying) from the others comprising the data set, and to do this objectively, involves the implementation of the 2-test [5]. 

To achieve the best accuracy and precision for quantitative analysis, samples must be analyzed in—at least—triphcate.This accomplishes two objectives (1) The agreement between the three repUcates provides a basis for editing the data to reject outliers and (2) a measure of the determination of variance can be calculated from the three repHcates. Generally, by making a minimum of three replicate determinations, reruns, dictated by quality assurance protocols (see Chapter 10) will not be required or will be necessary only in unusual situations. Reporting results as a mean and standard deviation ( error term) provides data that have more interpretation value. 

---