Evaluation of interlaboratory

Davies, P. L., Statistical Evaluation of Interlaboratory Tests, Fresenius Z. Anal. Chem. 331, 1988, 513-519. 

In general, the evaluation of interlaboratory studies can be carried out in various ways (Danzer et al. [1991]). Apart from z-scores, multivariate data analysis (nonlinear mapping, principal component analysis) and information theory (see Sect. 9.2) have been applied. 

Evaluation of data and validation multivariate data analysis (MULTI-VAR, Wienke et al. [1991]), evaluation of interlaboratory studies (INTERLAB, Wienke et al. [1991]), ruggedness expert system (RES, van Leeuwen et al. [1991]). 

Currie, L. A., Benner, B. A., et al. (2002). A critical evaluation of interlaboratory data on total, elemental, and isotopic carbon in the carbonaceous particle reference material, NIST SRM 1649a. /. Res. NIST 107, 279-298. 

P. Robouch, N. Younes, P. Vermaercke, The Naji plot, a simple graphical tool for the evaluation of interlaboratory comparisons, PTB-TT-10 149 (2003). 

Before a procedure can provide useful analytical information, it is necessary to demonstrate that it is capable of providing acceptable results. Validation is an evaluation of whether the precision and accuracy obtained by following the procedure are appropriate for the problem. In addition, validation ensures that the written procedure has sufficient detail so that different analysts or laboratories following the same procedure obtain comparable results. Ideally, validation uses a standard sample whose composition closely matches the samples for which the procedure was developed. The comparison of replicate analyses can be used to evaluate the procedure s precision and accuracy. Intralaboratory and interlaboratory differences in the procedure also can be evaluated. In the absence of appropriate standards, accuracy can be evaluated by comparing results obtained with a new method to those obtained using a method of known accuracy. Chapter 14 provides a more detailed discussion of validation techniques. 

The principle of insignificancy , enabling use of the given principle for any level of probability is substantiated. The systematic application of the given principle results in developing metrological criteria for pharmaceutical reference substance, analytical validation, evaluation of results of interlaboratory testing and suitability of the analytical equipment for the phamiaceutical analysis. 

Bauer, C. R, Grant, C. L., and Jenkins, T. R, Interlaboratory Evaluation of High-Performance Liquid Chromatographic Determination of Nitroorganics in Munition Plant Wastewater, Ana/. Chem. 58, 1986, 176-182. 

Butler OT, Howe AM (1999) Development of an international standard for the determination of metals and metalloids in workplace air using ICP-AES evaluation of sample dissolution procedures through an interlaboratory trial, f Environ Monit 1 23-32. 

Sakai, S., Adachi, R., Akiyama, H., Teshima, R., Morishita, N., Matsumoto, T., and Urisu, A. (2009). Interlaboratory evaluation of an enzyme-linked immunosorbent assay kit for the determination of soybean protein in processed foods. /. AOAC Int. 93, 243-248. 

The FDA requires [FDC Act, Section 512 (b)(1)(G)] that methods used for the detection and confirmation of drug residues in animal products be practicable. Overseeing the reliability of these methods is the responsibility of the FDA CVM. The methods are corroborated using an interlaboratory evaluation of the method known as a method trial. The method trial is used to demonstrate that the method is suitable for use to detect and confirm drug residues and can be performed by a trained analytical chemist. 

Danzer K, Wank U, Wienke D (1991) An expert system for the evaluation and interpretation of interlaboratory comparisons. Chemom Intell Lab Syst 12 69... 

The Kerridge-Bongard model of information is of great importance in quality assurance, in particular for the assessment of interlaboratory studies. Examples of the information-theoretical evaluation of analytical results within the context of interlaboratory comparisons have been given by Danzer et al. [1987, 2001], Wienke et al. [1991] and Danzer [1993]. 

Jacobs TW, Gown AM, Yaziji H, et al. HER-2/neu protein expression in breast cancer evaluated by immunohistochemistry. A study of interlaboratory agreement. Am. J. Clin. Pathol. 2000 113 251-258. 

Brooke, D., Nielsen, I., de Bruijn, J., Hermens, J. (1990) An interlaboratory evaluation of the stir-flask method for the determination of octanol-water partition coefficients (log Pow). Chemosphere 21, 119-133. 

The previous chapters of this book have discussed the many activities which laboratories undertake to help ensure the quality of the analytical results that are produced. There are many aspects of quality assurance and quality control that analysts carry out on a day-to-day basis to help them produce reliable results. Control charts are used to monitor method performance and identify when problems have arisen, and Certified Reference Materials are used to evaluate any bias in the results produced. These activities are sometimes referred to as internal quality control (IQC). In addition to all of these activities, it is extremely useful for laboratories to obtain an independent check of their performance and to be able to compare their performance with that of other laboratories carrying out similar types of analyses. This is achieved by taking part in interlaboratory studies. There are two main types of interlaboratory studies, namely proficiency testing (PT) schemes and collaborative studies (also known as collaborative trials). 

This chapter has considered two of the types of interlaboratory comparison exercise in which your laboratory may participate. It is important to remember that proficiency testing schemes and collaborative studies have different aims. The former is a test of the performance of the laboratory, whereas the latter is used to evaluate the performance of a particular analytical method. Laboratories should participate in proficiency testing schemes (where an appropriate scheme is available) as this provides an independent check of the laboratory s performance. This chapter has described the key features of proficiency testing schemes and explained how the results from participation in a scheme should be interpreted. 

Ladies, G.S., et al., Phase two of an interlaboratory evaluation of the quantification of rat splenic lymphocyte subtypes using immunofluorescent staining and flow cytometry, Toxicol. Methods, 8, 87, 1998. 

Basketter, D.A. et al., Interlaboratory evaluation of the local lymph node assay with 25 chemicals and comparison with guinea pig test data. Toxicol. Meth., 1, 30, 1991. 

Basketter, D.A., Scholes, E.W., Kimber, I., Botham, RA., Hilton, J., Miller, K., Robbins, M.C., Harrison, P.T.C. and Waite, S.J. (1991). Interlaboratory evaluation of the local lymph node assay with 25 chemicals and comparison with guinea pig test data. Toxicol. Methods 1 30-43. 

From the available literature it becomes clear that method evaluation studies do not surpass the level of within-laboratory performances. Although several of these (see Table 3.3.1) reveal satisfactory levels of quality and environmentally relevant limits of detection, a genuine quality assurance of these methods is still lacking. There are no reports of interlaboratory studies and certified reference materials for surfactants are not available on the market yet. It can therefore be concluded that there remains much to be done in the field of improving and evaluating quality of analytical measurements of surfactants in biota. 

Additionally, the LAS interlaboratory study III resulted in data sets from five laboratories. For three of these, individual C LAS data were made available. One participant did not comply with the interlaboratory protocol and their results could not be included in the evaluation. Laboratory 3 only supplied total LAS results (discussed below), while laboratory 5 supplied individual C LAS data which was too late for inclusion in the final evaluation. However, their total LAS results have been included. The results of interlaboratory study III for LAS are shown in Fig. 4.5.3 for the single congeners (Ci0, Cn, C12 and C13LAS), grouped according to cartridges. In Fig. 4.5.4 the total LAS (sum of Ci0-Ci3LAS) results for each laboratory are shown. 

The investigation of the presence of marine biotoxins in water, phytoplankton, and food has been achieved by several in vitro assays. However, alternatives to the animal bioassay for marine toxins have not been sufficiently evaluated in interlaboratory studies needed to demonstrate their scientific validity. In addition, these methods continue to be time consuming and expensive for intensive monitoring programs, and present some difficulties for their automation. 

Bryce, S.M., Avlasevich, S.L., Bemis, J.C., Lukamowicz, M Elhajouji, A., Van Goethem, F., De Boeck, M Beerens, D. et al. (2008) Interlaboratory evaluation of a flow cytometric, high content in vitro micronucleus assay. Mutation Research, 650, 181-195. 

Several overall conclusions can be drawn based on the statistical evaluation of the data submitted by the participants of the DR CALUX intra-and interlaboratory validation study. First, differences in expertise between the laboratories are apparent based on the results for the calibration curves (both for the curves as provided by the coordinator and for the curves that were prepared by the participants) and on the differences in individual measurement variability. Second, the average results, over all participants, are very close to the true concentration, expressed in DR CALUX 2,3,7,8-TCDD TEQs for the analytical samples. Furthermore, the interlaboratory variation for the different sample types can be regarded as estimates for the method variability. The analytical method variability is estimated to be 10.5% for analytical samples and 22.0% for sediment extracts. Finally, responses appear dependent on the dilution of the final solution to be measured. This is hypothesized to be due to differences in dose-effect curves for different dioxin responsive element-active substances. For 2,3,7,8-TCDD, this effect is not observed. Overall, based on bioassay characteristics presented here and harmonized quality criteria published elsewhere (Behnisch et al., 2001a), the DR CALUX bioassay is regarded as an accurate and reliable tool for intensive monitoring of coastal sediments. 

Baker YA, Harries HM, Waring JF et al. Clofibrate-induced gene expression changes in rat liver a cross-laboratory analysis using membrane cDNA arrays. Environ Health Perspect 2004 112 428-438. Waring JF, Ulrich RG, Flint N et al. Interlaboratory evaluation of rat hepatic gene expression changes induced by methapyrilene. Environ Health Perspect 2004 112 439 48. 

Precision plays a central role in collaborative studies. Wood [84] defines a collaborative trial as a procedure whereby the precision of a method of analysis may be assessed and quantified. Precision is the objective of interlaboratory validation studies, and not trueness or whichever other method performance parameter. Evalu-... 

Piersma AH, Attenon P, Bechter R, Govers MJAP, Krafft N, Schmid BP, Stadler J, Verhoef A, Verseil C (1995) Interlaboratory evaluation of embryotoxicity in the postimplantation rat embryo culture. Reprod Toxicol, 9 275-280. 

The evaluation of reproducibility results often focuses more on measuring bias in results than on determining differences in precision alone. Statistical equivalence is often used as a measure of acceptable interlaboratory results. An example of reproducibility criteria for an assay method could be that the assay results obtained in multiple laboratories will be statistically equivalent or the mean results will be within 2% of the value obtained by the primary testing laboratory. 

Coefficient of variation of interlaboratory results as a function of sample concentration (expressed as g analyte/g sample). The shaded region has been referred to as the Horwitz trumpet" because Of the way it flares open. [From W. Horwitz, Evaluation of Analytical Methods Used for Regulation of Foods and Drugs." Anal. Chem. 1982, 54.67A]... 

McCleary, B.V., Gibson, T.S., Solah, V., and Mug-ford, D.C. 1994. Total starch measurement in cereal products Interlaboratory evaluation of a rapid enzymic test procedure. Cereal Chem. 71 501-505. 

---