Repeatability and reproducibility

Precision is a measure of the spread of data about a central value and may be expressed as the range, the standard deviation, or the variance. Precision is commonly divided into two categories repeatability and reproducibility. Repeatability is the precision obtained when all measurements are made by the same analyst during a single period of laboratory work, using the same solutions and equipment. Reproducibility, on the other hand, is the precision obtained under any other set of conditions, including that between analysts, or between laboratory sessions for a single analyst. Since reproducibility includes additional sources of variability, the reproducibility of an analysis can be no better than its repeatability. 

Are Repeatable and Reproducibility (R R) studies available for the test procedure ... 

Figure 1.6. Repeatability and reproducibility are defined using historical data. The length of the time interval over which the parameter is reviewed is critical the shorter it is, the better defined the experimental boundary conditions tend to be the repeatability sets the limit on what could potentially be attained, the reproducibility defines what is attained in practice using a given set of instrumentation and SOPs.

When judging by communications available in the open literature, none of effects 1 through 5 could so far be observed repeatedly and reproducibly under rigorously controlled conditions. Provisionally, all instances of published experimental confirmation can be placed into two groups (1) the observation of sporadic sufficiently pronounced manifestations, and (2) the observation of more invariant but very weak effects (as a rule, at the level of background noise, particularly in the detection of neutrons and tritium). However, there were far fewer confirmations than infirmations (i.e., work in which the successful experiments could be carefully reproduced or the method used to determine the products was analyzed and shown to be in error). Such work has been of exceptional value in the area of advancing the methods and techniques used in experimental studies. 

TABLE 4.15 Recovery, repeatability, and reproducibility for milk detection"... 

The precision of recovery is determined under repeatability and reproducibility conditions. The more important between-laboratory reproducibility is calculated as relative standard deviation (RSDr) and compared with the RSDr, which is estimated from the Horwitz equation using the same analyte concentration. For good methods this ratio should be about 1, but a method will usually be accepted if the ratio is not larger than 2. 

Prior to conducting the DOE (design of experiments) described in Table 3, it was established that no reaction took place in the absence of a catalyst and that the reactions were conducted in the region where chemical kinetics controlled the reaction rate. The results indicated that operating the reactor at 1000 rpm was sufficient to minimize the external mass-transfer limitations. Pore diffusion limitations were expected to be minimal as the median catalyst particle size is <25 pm. Further, experiments conducted under identical conditions to ensure repeatability and reproducibility in the two reactors yielded results that were within 5%. 

Analytical measurement Pros - Results obtained reflect well reality - Repeatability and reproducibility of results (at least between good qualified labs) - Measurements are independent of information/data sources - Multipurpose analytical methods can cover many compounds on a single run - Even the best model will ultimately need to be experimentally checked - Discovery of new emerging contaminants is possible... 

Random deviations (errors) of repeated measurements manifest themselves as a distribution of the results around the mean of the sample where the variation is randomly distributed to higher and lower values. The expected mean of all the deviations within a measuring series is zero. Random deviations characterize the reliability of measurements and therefore their precision. They are estimated from the results of replicates. If relevant, it is distinguished in repeatability and reproducibility (see Sect. 7.1)... 

It is a well-known fact that the precision in trace analysis decreases with diminishing concentration in a similar way as it does with decreasing sample weight (Sect. 2.1). The dependency of the repeatability and reproducibility standard deviation on the concentration of analytes has been investigated systematically at first by Horwitz et al. [1980] on the basis of thousands of pieces of interlaboratory data (mostly from food analysis). The result of the study has been represented in form of the well-known Horwitz trumpet which is represented in Fig. 7.3. 

Kele, M., Guiochon, G. (2002). Repeatability and reproducibility of retention data and band profiles on six batches of monolithic columns. J. Chromatogr. A 960, 19 -49. 

Typical flow rates in FTA vary between 0.5 and 5.0 ml/min per channel, although higher values have also been used. Most of the published work on CL with HA is based on equal flow rates for every stream entering the manifold. Nevertheless, if different rates must be used, deterioration of repeatability and reproducibility might appear due to incomplete mixing and anomalous hydrodynamic characteristics. These problems can be avoided if the general rule that the ratio of the fastest to the slowest flow rate should not exceed the value of 3 or 4 is followed. 

A range of 11 different materials, representative of building products commonly used in Europe, has been distributed and will be tested in each of the laboratories, on each test method, and the results will be assessed for repeatability and reproducibility. 

Precision is the closeness of agreement between independent test results obtained under stipulated conditions. Precision depends only on the distribution of random errors and does not relate to the true value. It is calculated by determining the standard deviation of the test results from repeat measurements. In numerical terms, a large number for the precision indicates that the results are scattered, i.e. the precision is poor. Quantitative measures of precision depend critically on the stipulated conditions. Repeatability and reproducibility are the two extreme conditions. 

Clearly, precision studies should mirror the operating conditions used during the routine use of the method. Individual sources of imprecision, such as change of operator or instrument type, can be studied but two types of overall precision are commonly estimated, i.e. repeatability and reproducibility. 

Two other terms that you will come across when working with chemical data are repeatability and reproducibility. Again, these two terms can easily be confused and you should learn to distinguish between them. They are both measures of precision. 

Write down your definitions of repeatability and reproducibility. 

Repeatability and reproducibility are discussed in more detail in Chapter 4. 

If the analytical method used by participants in the proficiency testing round has been validated by means of a formal collaborative trial, then the repeatability and reproducibility data from the trial can be used. The repeatability standard deviation gives an estimate of the expected variation in replicate results obtained in a single laboratory over a short period of time (with each result produced by the same analyst). The reproducibility standard deviation gives an estimate of the expected variation in replicate results obtained in different laboratories (see Chapter 4, Section 4.3.3 for further explanation of these terms). 

Accuracy (Trueness and Precision) of Measurement Methods and Results - Part 2 Basic Method for the Determination of Repeatability and Reproducibility of a Standard Measurement Method , ISO 5725-2 1994, International Organization for Standardization (ISO), Geneva, Switzerland, 1994. 

A. The distinction between repeatable and reproducible was clarified much later by the present author (152). 

The ability to provide accurate and reliable data is central to the role of analytical chemists, not only in areas like the development and manufacture of drugs, food control or drinking water analysis, but also in the field of environmental chemistry, where there is an increasing need for certified laboratories (ISO 9000 standards). The quality of analytical data is a key factor in successfully identifying and monitoring contamination of environmental compartments. In this context, a large collection of methods applied to the routine analysis of prime environmental pollutants has been developed and validated, and adapted in nationally or internationally harmonised protocols (DIN, EPA). Information on method performance generally provides data on specificity, accuracy, precision (repeatability and reproducibility), limit of detection, sensitivity, applicability and practicability, as appropriate. 

When the test programme spans long times, the situation is more akin to reproducibility conditions than to repeatability, even although the measurements are made in the same laboratory. The increase in uncertainty can be illustrated by comparing the repeatability and reproducibility figures given in precision statements. 

The precision values referred to in 1 (iii) shall be obtained from a collaborative trial which has been conducted in accordance with an internationally recognised protocol on collaborative trials (e.g. International Organisation of Standardization Precision of Test Methods )17 The repeatability and reproducibility values shall be expressed in an internationally recognised form (e.g. the 95% confidence intervals as defined by ISO 5725/1981). The results from the collaborative trial shall be published or be freely available. 

As seen above, all official methods of analysis are required to include precision data. These may be obtained by subjecting the method to a collaborative trial conforming to an internationally agreed protocol. A collaborative trial is a procedure whereby the precision of a method of analysis may be assessed and quantified. The precision of a method is usually expressed in terms of repeatability and reproducibility values. Accuracy is not the objective. 

The precision of a test method is the variability between test results obtained on the same material using a specific test method (ASTM, 2004 Patnaik, 2004). The precision of a test is usually unrelated to its accuracy. The results may be precise, but not necessarily accurate. In fact, the precision of an analytical method is the amount of scatter in the results obtained from multiple analyses of a homogeneous sample. To be meaningful, the precision study must be performed using the exact sample and standard preparation procedures that will be used in the final method. Precision is expressed as repeatability and reproducibility. 

The repeatability and reproducibility values have important implications for quality. As the demand for clear product specifications, and hence control over product consistency, grows, it is meaningless to establish product specifications that are more restrictive than the reproducibility/repeatability values of the specification test methods. 

Kele, M. and Guiochon, G. Repeatability and Reproducibility of Retention Data and Band Profiles on Six Batches of Monolithic Columns,/. Chromatogr., 960 19—49,2002. 

---