Collaborative trial

As we have seen, proficiency testing schemes allow the competence of laboratories to be monitored, compared and perhaps improved. By contrast a collaborative trial (CT) aims to evaluate the precision of an analytical method, and sometimes its ability to provide results free from bias. It is normally a one-off experiment involving expert or competent laboratories, all of which by definition use the 

When the seven differences for factors A-G have all been calculated in this way, it is easy to identify any factors that have a worryingly large effect on the results. It may be shown that any difference that is more than twice the standard deviation of replicate measurements is significant and should be further studied. This simple set of experiments, technically known as an incomplete factorial design, has the disadvantage that interactions between the factors cannot be detected. This point is further discussed in Chapter 7. 

In recent years international bodies have moved towards an agreement on how CTs should be performed. At least eight laboratories (k 8) should be involved. Since the precision of a method usually depends on the analyte concentration, it should be applied to at least five different levels of analyte in the same sample matrix, with duplicate measurements ( = 2) at each level. A crucial requirement of a CT is that it should distinguish between the repeatability standard deviation, s and the reproducibility standard deviation, Sr. At each analyte level these are related by the equation  

The first assumption is then tested using Grubbs test (Section 3.7) which is applied first as a test for single outliers, and then (since each laboratory makes duplicate measurements) in a modified form as a test for paired outliers. In both 

Youden plots provide a good deal of information in an immediately accessible form, but we still need methods for calculating the variances and sf The following example shows how this can also be done in a simple way. 

---

Brompton Hospital/Medical Research Council Collaborative Trial, Br. Med. J. 4, 383 (1972). 

Tollefson GD, Beasley CM, Tran PV (1997). Olanzapine versus haloperidol in the treatment of schizophrenia and schizoaffective and schizophreniform disorders results of an international collaborative trial. Am J Psychiatry 154, 457-65. 

Criticisms of the KS test An extensive collaborative trial was carried out on the KS test and the conclusion (Cowen, 1978) was that the test was suitable for white and clear, soluble disinfectants providing due care was taken in interpreting the pass concentration. Further modification ofthe test is necessary before it can be applied to... 

The third approach is also known as a collaborative study or a collaborative trial. Both names underpin the joint effort of the coordinator and participants to characterize the reference material. In any case, the measurement methods used in the characterization should be traceable to what is called stated references , and preferably to SI. The aspect of traceability of measurement results goes well beyond the actual measurements it also includes the transformation of the sample from the state of the reference material to the state in which it can be measured. An example of such a transformation is the destruction of the sample. 

The European Pharmacopoeia prepares a protocol which must be strictly followed by the participants of the collaborative trial to assign the content. The protocol usually requires ... 

Assuming an analytical error of 2 % which is based on the analysis of the results of a number of proficiency tests and collaborative trials (Daas and Miller 1998), a maximum uncertainty of 0.24% would imply a 0.1% probability of rejecting a good result. Given that the uncertainty of a content to be assigned is below a predetermined value, the results of the collaborative trial are acceptable otherwise it is recommended to repeat the trial in whole or in part. 

An example for the application of this calculation is given in Table 5.4 which shows the results of a collaborative trial for the establishment of ciprofloxacin hydrochloride CRS 2. The uncertainty was calculated to be o.ii %. 

Fig. 5-9 Decision tree used to accept or reject results from collaborative trials to establish the potencies of antibiotics to be used as chemical reference substances for microbiological assay standards.

White, J. W. (1992). Internal standard stable carbon isotope ratio method for determination of C-4 plant sugars in honey Collaborative trial study, and evaluation of improved protein preparation procedure. J. Assoc. Ojfic. Anal. Chem. 75,543-548. 

The previous chapters of this book have discussed the many activities which laboratories undertake to help ensure the quality of the analytical results that are produced. There are many aspects of quality assurance and quality control that analysts carry out on a day-to-day basis to help them produce reliable results. Control charts are used to monitor method performance and identify when problems have arisen, and Certified Reference Materials are used to evaluate any bias in the results produced. These activities are sometimes referred to as internal quality control (IQC). In addition to all of these activities, it is extremely useful for laboratories to obtain an independent check of their performance and to be able to compare their performance with that of other laboratories carrying out similar types of analyses. This is achieved by taking part in interlaboratory studies. There are two main types of interlaboratory studies, namely proficiency testing (PT) schemes and collaborative studies (also known as collaborative trials). 

If the analytical method used by participants in the proficiency testing round has been validated by means of a formal collaborative trial, then the repeatability and reproducibility data from the trial can be used. The repeatability standard deviation gives an estimate of the expected variation in replicate results obtained in a single laboratory over a short period of time (with each result produced by the same analyst). The reproducibility standard deviation gives an estimate of the expected variation in replicate results obtained in different laboratories (see Chapter 4, Section 4.3.3 for further explanation of these terms). 

Loveless, S.E. et al., Further evaluation of the local lymph node assay in the final phase of an international collaborative trial, Toxicology, 108, 141, 1996. 

Arlett, C.E and Cole, J. (1990). The third United Kingdom Environmental Mutagen Society collaborative trial overview, a summary and assessment. Mutagenesis 5 (Suppl.) 85-88. 

Loveless, S.E., Ladies, G.S., Greherick, G.F., Ryan, C.A., Basketter, D.A., Scholes, E.W., House, R.V, Hilton, J., Dearman, R.J. and Kimber I. (1996). Further evaluation of the local lymph node assay in the final phase of the international collaborative trial. Toxicology 108 141-152. 

The precision values referred to in 1 (iii) shall be obtained from a collaborative trial which has been conducted in accordance with an internationally recognised protocol on collaborative trials (e.g. International Organisation of Standardization Precision of Test Methods )17 The repeatability and reproducibility values shall be expressed in an internationally recognised form (e.g. the 95% confidence intervals as defined by ISO 5725/1981). The results from the collaborative trial shall be published or be freely available. 

Consideration of the above requirements confirms that in future all methods must be fully validated if at all possible, i.e. have been subjected to a collaborative trial conforming to an internationally recognised protocol. In addition this, as described above, is now a legislative requirement in the food sector of the European Union. The concept of the valid analytical method in the food sector, and its requirements, is described below. 

Precision is defined as the closeness of agreement between independent test results obtained under prescribed conditions.19 In a standard method the precision characteristics are obtained from a properly organised collaborative trial, i.e. a trial conforming to the requirements of an International Standard (the AOAC/ISO/IUPAC Harmonised Protocol or the ISO 5725 Standard). Because of the importance of collaborative trials, and the resource that is now being devoted to the assessment of precision characteristics of analytical methods before their acceptance, they are described in detail below. 

As seen above, all official methods of analysis are required to include precision data. These may be obtained by subjecting the method to a collaborative trial conforming to an internationally agreed protocol. A collaborative trial is a procedure whereby the precision of a method of analysis may be assessed and quantified. The precision of a method is usually expressed in terms of repeatability and reproducibility values. Accuracy is not the objective. 

Recently there has been progress towards a universal acceptance of collaboratively tested methods and collaborative trial results and methods, no... 

There is concern in the food analytical community that although methods should ideally be validated by a collaborative trial, this is not always feasible for economic or practical reasons. As a result, IUPAC guidelines are being developed for single laboratory method validation to give information to analysts on the acceptable procedure in this area. These guidelines should be finalised by the end of 2001. 

Has the method been validated by collaborative trial (i.e. externally) ... 

Studies by Schneier et al. (F. Schneier, personal communication, June 1994) and two large, unpublished, multicenter international collaborative trials involving several hundred patients have yielded more modest effects, with less striking but still clinically meaningful superiority of moclobemide over placebo. 

Kirkwood, J.M., J.E. Harris, R. Vera, S. Sandler, D.S. Fischer, J. Khandekar, M.S. Ernstoff, L. Gordon, R. Lutes, P. Bonomi, et al., A randomized study of low and high doses of leukocyte alpha-interferon in metastatic renal cell carcinoma the American Cancer Society collaborative trial Cancer Res, 1985. 45(2) 863-71. 

Validation of a new analytical method is typically done at two levels. The first is the level of prevalidation, aiming at fixing the scope of the validation. The second level is an extensive, full validation performed through a collaborative trial or interlaboratory study. The objective of full validation, involving a minimum number of laboratories, is to demonstrate that the method performs as was stated after the prevalidation. 

Use of Validated Methods In-Home Versus Interlaboratory Validation Wherever possible or practically achievable, a laboratory should use methods which have been fully validated through a collaborative trial, also called interlaboratory study or method performance study. Validation in collaborative studies is required for any new analytical method before it can be published as a standard method (see below). However, single-laboratory validation is a valuable source of data usable to demonstrate the fitness for purpose of an analytical method. In-house validation is of particular interest in cases where it is inconvenient or impossible for a laboratory to enter into or to organize itself a collaborative study [4,5]. 

On the one hand, even if an in-house vahdated method shows good performance and reliable accuracy, such a method cannot be adopted as a standard method. In-house validated methods need to be compared between at least eight laboratories in a collaborative trial. On the other hand, a collaborative study should not be conducted with an unoptimized method [58]. Interlaboratory studies are restricted to precision and trueness while other important performance characteristics such as specificity and LOD are not addressed [105]. For these reasons, single-laboratory validation and interlaboratory validation studies do not exclude each other but must be seen as two necessary and complementary stages in a process, presented in Figure... 

The terms PT schemes and collaborative trials are often confused with each other, as in both QA measures, a number of different laboratories are involved. However, there is a clear distinction between both. The mean differences with respect to the objective and application, the results, used method, test materials, and participating laboratories are summarized in Table 9. It is important to note also that the results obtained from PT schemes, as well as those from collaborative performance studies, can be used for assessing the MU (see Section 8.2.2). 

Validation attempts in tlie field of drug residue analysis have demonstrated that die requirement for a full collaborative trial at the ideal level, while desirable, is sometimes impractical. Limiting factors for completing ideal multilaboratory validation studies are usually the high cost, lack of sufficient expert laboratories willing to participate in such studies, and overall time constraints. Hence, a three-laboratory validation study is often applied (16). 

The objective for any analytical procedure is to enable consistent and reliable data of the appropriate quality to be generated by laboratories. Such procedures should be sufficiently well-defined and robust to ensure the best use of resources and to minimise the possibility of expensive large-scale collaborative trials yielding unsatisfactory results through lack of application of best practices. As part of achieving the objectives of the AMC it was felt that such a handbook would enable a consistency of approach to the work of the sub-committees. 

Provision of a unified and disciplined framework that covers all aspects of the validation process from sample and method selection to full collaborative trial. 

---