Collaborative test

These data are adapted from Steiner, E. H. Planning and Analysis of Results of Collaborative Tests published in Statistical Manual of the Association of Official Analytical Chemists, Association of Official Analytical Chemists Washington, DC, 1975. 

The goal of a collaborative test is to determine the expected magnitude of ah three sources of error when a method is placed into general practice. When several analysts each analyze the same sample one time, the variation in their collective results (Figure 14.16b) includes contributions from random errors and those systematic errors (biases) unique to the analysts. Without additional information, the standard deviation for the pooled data cannot be used to separate the precision of the analysis from the systematic errors of the analysts. The position of the distribution, however, can be used to detect the presence of a systematic error in the method. 

The design of a collaborative test must provide the additional information needed to separate the effect of random error from that due to systematic errors introduced by the analysts. One simple approach, which is accepted by the Association of Official Analytical Chemists, is to have each analyst analyze two samples, X and Y, that are similar in both matrix and concentration of analyte. The results obtained by each analyst are plotted as a single point on a two-sample chart, using the result for one sample as the x-coordinate and the value for the other sample as the -coordinate. ... 

In Examples 14.7 and 14.8 we have seen how a collaborative test using a pair of closely related samples can be used to evaluate a method. Ideally, a collaborative test should involve several pairs of samples, whose concentrations of analyte span the... 

In the two-sample collaborative test, each analyst performs a single determination on two separate samples. The resulting data are reduced to a set of differences, D, and a set of totals, T, each characterized by a mean value and a standard deviation. Extracting values for random errors affecting precision and systematic differences between analysts is relatively straightforward for this experimental design. 

An alternative approach for collaborative testing is to have each analyst perform several replicate determinations on a single, common sample. This approach generates a separate data set for each analyst, requiring a different statistical treatment to arrive at estimates for Grand and Csys-... 

Collaborative testing provides a means for estimating the variability (or reproducibility) among analysts in different labs. If the variability is significant, we can determine that portion due to random errors traceable to the method (Orand) and that due to systematic differences between the analysts (Osys). In the previous two sections we saw how a two-sample collaborative test, or an analysis of variance can be used to estimate Grand and Osys (or oJand and Osys). We have not considered, however, what is a reasonable value for a method s reproducibility. 

The last step in establishing a standard method is to validate its transferability to other laboratories. An important step in the process of validating a method is collaborative testing, in which a common set of samples is analyzed by different laboratories. In a well-designed collaborative test, it is possible to establish limits for the method s precision and accuracy. 

Agency. A second example of an external method of quality assessment is the voluntary participation of the laboratory in a collaborative test (Chapter 14) sponsored by a professional organization such as the Association of Official Analytical Chemists. Finally, individuals contracting with a laboratory can perform their own external quality assessment by submitting blind duplicate samples and blind standard samples to the laboratory for analysis. If the results for the quality assessment samples are unacceptable, then there is good reason to consider the results suspect for other samples provided by the laboratory. 

Color and Appearance Collaborative Reference Program, Collaborative Testing Services, Inc., Herndon, Va., previously MCCA-NBS Collaborative Reference Program on Color and Color Differences, National Institute of Standards and Technology, Washiagton, D. C., 1991. 

A collaborative test programme covering low-alloy and high-alloy steels was carried out by the Central Electricity Generating Board and various steelmakers. Samples were exposed in specially constructed chambers held at 566°C, 593°C and 621 °C fed with power-station steam at a pressure of 3-45 MN/m for times of up to 16 286 h. In the assessment of the results both metal lost from the surface and subsurface penetration were measured. The results have been reported by King, Robinson, Howarth and Perry in a C.E.G.B. report. Selected data are shown in Fig. 7.32, in which the broken lines have been obtained by extrapolation of the experimental results. 

In contrast to the requirements for enforcement methods, validation of a previously collaboratively tested method, which is used to generate data, should be validated for new laboratory conditions. Also, where published methods are submitted, validation is required, when applied to the relevant sample matrix and laboratory conditions. 

This set of articles presents the computational details and actual values for each of the statistical methods shown for collaborative tests. These methods include the use of precision and estimated accuracy comparisons, ANOVA tests, Student s t-testing, The Rank Test for Method Comparison, and the Efficient Comparison of Methods tests. From using these statistical tests the following conclusions can be derived ... 

Recently there has been progress towards a universal acceptance of collaboratively tested methods and collaborative trial results and methods, no... 

Youden, W.J. "Statistical Techniques for Collaborative Tests" Association of Official Analytical Chemists Washington, D.C., 1969 p 37.Sagan, C. The Washington Post... 

Other classes of reference materials now in existence include secondary reference materials. These are materials produced commercially for reference purposes, but whose guarantee rests soley with the producer. "Analyzed" materials such as geological materials obtained from the United Staes Geological Survey, represent test samples that complement the variety available from the previously mentioned sources. However, the "accepted" analyses reported for these materials are based upon consensus values obtained from large scale interlaboratory collaborative tests (round robins). Analysis of these materials can provide a useful means of comparing performance with other laboratories, but it does not ensure accuracy. In addi-... 

Collaborative Testing. A second approach to assessing accuracy, when no certified reference material is available, may be used in conjunction with analysis by independent methods and in-house materials. Sample exhanges with other laboratories can help establish the existence or absence of systematic errors in a method. Collaborative tests are most useful in this regard when some of the participating laboratories use different sample preparation and quantification. The utility of independent analysis methods and comparisons between destructive and non-destructive analysis is again emphasized here. 

W.J. Youden and E.H. Steiner, Statistical Manual of the AOAC. Statistical Techniques for Collaborative Tests. Planning and Analysis of Results of Collaborative tests, AOAC, Washington,DC, (1975) 33-121. 

Colovos, G. Eaton, W.S. Ricci, G.R. Shepard, L.S., Wang, H. Collaborative Testing of NIOSH Atomic Absorption Method , DHEW (NIOSH) Publication No. 79-1 Wt, 1978. 

One of the key concerns of analytical science is how good are the numbers produced . Even with an adequately developed, optimised and collaboratively tested method which has been carried out on qualified and calibrated equipment the question remains. Recently it has become fashionable to extend the concepts of the physical metrology into analytical measurements and to quantify confidence in terms of the much more negative uncertainty.It is based on the bottom-up principle or the so called error budget approach. This approach is based on the theory that if the variance contributions of all sources of error involved in analytical processes then it is possible to calculate the overall process... 

Youden, W. J. Statistical Techniques for Collaborative Tests The Association of Official Analytical Chemists Arlington, VA, 1967. 

JWG Porter, DR Westgarth, AP Williams. A collaborative test of ion-exchange chromatographic methods for determining amino acids. Br J Nutr 22 437-450, 1968. 

Collaboratively tested in 12 laboratories using 8 samples reproducibility CV 12-35%. 

Gas chromatography and capillary column GC represent by far the most developed and most used analytical techniques for the determination of multiple trichothecenes, especially in view of the possibility of employing MS for confirmation of the peaks. The available methods, such as the Scott and Ware methods (90,91), differ in sample extraction, sample cleanup, and the derivatization step. Heptafluorobutyryl (HFB), trimethylsilyl (TMS), and trifluoroacetyl derivatives are frequently used, coupled with electron capture detection. A GC method (92) has recently been collaboratively tested in barley and malt and accepted by the American Society of Brewing... 

---