Inter-laboratory precision

This simple function is most useful in setting acceptance criteria for analytical method inter-laboratory precision. Note that the concentration, C, is relative so that 100% has a value of 1, i.e. 10 . Table 19 lists values in decreasing powers of 10. 

M. Thompson, Recent trends in inter-laboratory precision at ppb and sub-ppb concentrations in relation to fitness for purpose criteria in proficiency testing, Analyst, 125 (2000), 385-386. 

If known pharmacological agents are available, then the screening assay is validated across multiple assays. This is more difficult if the target of interest is novel and no pharmacological tools are available. It requires rigorous attention to detail to achieve inter-laboratory precision even down to the way the compounds are solubilized and diluted, the types of equipment, and the detection technology. 

Analysis of filtered Baltic Sea samples of about 0.5 pmol/L spiked to nominally I, 2 and 3/tmol/L ammonia and analysed in three independent analytical runs resulted in a mean standard deviation of 0.092 pmol/L or 2.7 % Hansen and Johannsen, unpublished). Similar results have been reported by Riley et al. (1972) and Solorzano (1969). The recent ICES intercomparison exercise (Aminot and Kirkwood, 1995) showed an overall relative standard deviation of more than 20 %, indicating that, despite good precision of ammonia measurements within one laboratory, the inter-laboratory precision is comparatively poor. This is probably due to the ease of contamination in preparations of zero water and standards as well as in the handling of samples for ammonia determinations. 

The precision of an analytical procedure expresses the closeness of agreement (degree of scatter) between a series of measurements obtained from multiple sampling of the same homogeneous sample under the prescribed conditions. Precision may be considered at three levels repeatability (within run) intermediate precision (over time) and reproducibility (inter-laboratory). 

Reference methods are generally arrived at by consensus and fairly extensive testing by a number of laboratories. For example, the flame atomic absorption method for Ca in serum developed under the leadership of the agency fondly remembered as NBS, now NIST (Cali et al. 1972), was established after several inter-laboratory comparison exercises. The results were evaluated after each exercise and the procedure was changed as necessary. After five exercises, it was felt that the state-of-the-art had been reached, with the reference method being capable of measuring Ca in serum with an accuracy of 2% of the true value determined by IDMS (note that attainment of high accuracy and precision is not only a matter of the method, but is a function of both the method and analyst expertise). 

Even if most examples and procedures presented apply to in-house validation, the procedure does not distinguish between validations conducted in a single laboratory and those carried out within inter-laboratory method performance studies. A preference for inter-laboratory studies can be concluded from the statement that laboratories should always give priority to methods which have been tested in method performance studies. Within the procedure a profound overview of different categories of analytical methods according to the available documentation and previous external validation is given. For example, if a method is externally validated in a method performance study, it should be tested for trueness and precision only. On the other hand, a full validation is recommended for those methods which are published in the scientific literature without complete presentation of essential performance characteristics (Table 9). 

If analytical methods are validated in inter-laboratory validation studies, documentation should follow the requirements of the harmonized protocol of lUPAC. " However, multi-matrix/multi-residue methods are applicable to hundreds of pesticides in dozens of commodities and have to be validated at several concentration levels. Any complete documentation of validation results is impossible in that case. Some performance characteristics, e.g., the specificity of analyte detection, an appropriate calibration range and sufficient detection sensitivity, are prerequisites for the determination of acceptable trueness and precision and their publication is less important. The LOD and LOQ depend on special instmmentation, analysts involved, time, batches of chemicals, etc., and cannot easily be reproduced. Therefore, these characteristics are less important. A practical, frequently applied alternative is the publication only of trueness (most often in terms of recovery) and precision for each analyte at each level. No consensus seems to exist as to whether these analyte-parameter sets should be documented, e.g., separately for each commodity or accumulated for all experiments done with the same analyte. In the latter case, the applicability of methods with regard to commodities can be documented in separate tables without performance characteristics. 

Method precision refers to the variability in measurement of the same sample. There are three main components of method precision repeatability (also known as system or intraassay precision), intermediate precision (also known as inter-assay or intra-laboratory precision), and reproducibility precision (also known as ruggedness, overall or inter-laboratory... 

Precision Repeatability Ruggedness Reproducibility Sample prep replication Injection replication Factorial designs Intra-laboratory study Inter-laboratory study... 

It is readily apparent that the HORRAT ratio of 2.89 indicates that the method is not sufficiently precise for inter-laboratory work. However, inspection of the ANOVA table shows a large value for the mean square due to the samples, MSs. This is an indication that they may not have been homogeneous and that the method may in fact be satisfactory. This demonstrates a major advantage in partitioning the variances between laboratory, sample and error rather than the traditional within- and between-laboratories method. 

ISO uses two terms, trueness and precision , to describe the accuracy of a measured value. Trueness refers to the closeness of agreement between the average value of a large number of test results and the true or accepted reference value. Precision refers to the closeness of agreement of test results, or in other words the variability between repeated tests. The standard deviation of the measured value obtained by repeated determinations under the same conditions is used as a measure of the precision of the measurement procedure. The repeatability limit r (an intra-laboratory parameter) and the reproducibility limit R (an inter-laboratory parameter) are calculated as measures of precision. Again, precision and trueness together describe the accuracy of an analytical method. 

An alternative method for performance evaluation is to participate in inter-laboratory comparisons, often known as round-robins . Usually this involves sending sub-samples of a selection of appropriate samples to a number of independent laboratories, to be analysed either by a fully specified procedure and technique, or to be analysed by whatever method each laboratory choses. Obviously, these two approaches test different things. The former indicates the precision attainable using a specified procedure, and tests both the adequacy of the specification and the competence of participating laboratories. Only the... 

Precision refers to the degree of repeatability under the stated conditions of the method. It is expressed as percent relative standard deviation (% RSD) for a statistically significant number of analyses of samples. Precision provides a measure of day to day, analyst to analyst and instrument to instrument variation on a routine basis. The precision data provided in support are standard deviation, % RSD, confidence intervals and may also include inter laboratory variations. 

The precision of an analytical method is the closeness of a series of individual measurements of an analyte when the analytical procedure is applied repeatedly to multiple aliquots of a single homogeneous volume of biological matrix [16], The precision is calculated as coefficient of variation (C.V.), i.e., relative standard deviation (RSD). The measured RSD can be subdivided into three categories repeatability (intra-day precision), intermediate precision (inter-day precision) and reproducibility (between laboratories precision) [16, 78, 79, 81],... 

Precision Instrument repeatability - ten replicate injections. RSD<1% Intra-assay precision. Multiple analysis of aliquots of a sample during one day. RSD < 2% Intermediate precision. Multiple analysts, instruments, days in one laboratory. Reproducibility by inter-laboratory studies to detect bias. 

Once a method is established, precision may be determined by suitable replicated experiments. However it is in inter-laboratory trails that the problems with environmental methods often show up. It is accepted that for trace analysis RSD values of tens of percent are likely. In studies conducted in Western Australia on pesticide residues in bovine fat RSD values for dieldrin were 12% and for dia-zonium were 28%. It is typical to see a quarter of the laboratories in such a trial producing values that could be termed outliers. In the previously mentioned study, 5 laboratories out of 26 had z> 3 for aldrin. In a parallel study RSD values for petroleum in dichloromethane and water were 40% and 25%, respectively. The conclusions of these studies was that there was poor comparability because of the different methods used, that accreditation apparently made no difference to the quality of results, and that a lack of understanding of definitions of the quantities to be analysed (for example gasoline range organics ) caused non-method errors. In relation to methods, this is contrary to the conclusion of van Nevel et al. who asserted that the results of the IMEP round of analyses of trace elements in natural and synthetic waters showed no dependence on method [11]. If properly validated methods do yield comparable results, then one conclusion from the range of studies around the world is that many environmental methods are not validated. It may be that validated methods are indeed used, but not for exactly the systems for which they were validated. 

ISO 5725 (1986) Precision of test methods - Determination of repeatability and reproducibility for a standard test method by inter-laboratory tests. 

---