Interlaboratory error

Currie, L. A., Polach, H. A., Exploratory Analysis of the International Radiocarbon Cross-Calibration Data Consensus Values and Interlaboratory Error, Proceedings of the 10th International Radiocarbon Conference, Radiocarbon. 22, 933... 

There were several reports that estimated the accuracy of the aqueous solubility data. Analysis of experimental solubility values for 411 compounds provided an average standard deviation of 0.6 units [21], In a more recent study [22] data for 1031 compounds that had at least two or more experimental values from the AQUASOL database [15] had a standard deviation of 0.5log units for measurements at the same temperature. The large interlaboratory error masked the influence of temperature on the solubility. Differences as large as AT = 30°C did not increase the error. Thus the accuracy of models for diverse sets of compounds can be hardly below 0.5 to 0.6 log units. 

Figure 8-3 Interlaboratory error as a function of analyte concentration. Note that the relative standard deviation dramatically increases as the analyte concentration decreases. In the ultratrace range, the relative standard deviation approaches 100%. (From W. Horowitz, dna/. Chem.. 1982, 54. 67A-76A.)...

Reanalysis and improvement of the Potts and Guy Equation 7.2 has been performed. Through reanalysis of the data, Equation 7.3 confirms the validity of the model published by Potts and Guy. This has been improved in terms of statistical quality (e.g., goodness of fit, etc.) through the use of improved data (Equation 7.4) and the removal of suspected poor-quality data (Equation 7.5). Importantly, the coefficients on the descriptors and the constant in the model do not change from Equation 7.2 to Equation 7.5. This is a good indicator of the stability of the model. Also, although the statistical fit of the model is not perfect (75% of the variance of the data is explained by Equation 7.5), it is realistic in terms of the data being modeled, that is, a diverse set of values with considerable interlaboratory error. The standard error of about 0.65 log units is acceptable and relates well to the error that may be expected from experimental measurement. 

Viscosity versus shear rate curves for two different compositions of a hydroxy benzoic acid/poly(ethylene terephthalate) copolymer (HBA/PET) at various temperatures were used [84] in the formation of the master rheogram for the liquid-crystalline polymer shown in Fig. 4.37. The data for 80 mole% of HBA at six different temperatures ranging from 275°C to 330 C were taken from Ref. 42. In order to eliminate operator and interlaboratory errors, the data on the same composition at one representative temperature of 275°C was also used from another source [85]. The other composition of HBA/PET chosen was 60 mole% and again from two different sources [42,86]. From one source [42], the viscosity data for 60 mole% of HBA at three different temperatures between 210°C and 300 C was used, where as from the other [86], three different temperatures between 260°C and 285 C was used. A total of 51 data points covering a shear-rate range from 2 to 8000/s have been included (Table B3 of Appendix B) in the unification process to form the master rheogram in Fig. 4.37. 

Two technical papers recognized as significant early contributions in the discussion of the limitations of analytical accuracy and uncertainty include those by Horwitz of the U.S. FDA [1, 2], For this next series of articles we will be discussing both the topic and the approaches to this topic taken by the classic papers just referenced. The determination and understanding of analytical error is often approached using interlaboratory collaborative studies. In this book we have previously delved into that subject in Chapters 34-39. 

In general terms, the variation from laboratory to laboratory (between-laboratory) was greater than that attributed to the analytical error displayed within laboratories (intralaboratory). There are many reasons for the interlaboratory variation that can be attributed to operational parameters such as mobile phase flowrate, mobile phase and buffer composition, vaporiser temperature, tip temperature and source temperature. 

Traeness is a property related to systematic errors. It is the closeness of agreement between the average value obtained from a large set of test results and an accepted reference value. It can be checked with reference materials or in interlaboratory comparisons. 

The laboratory bias is either a systematic error—if the laboratory is considered on its own—or a random error—if the laboratory is considered as one of a group, as is the case in interlaboratory studies. 

Reproducibility uncertainty is derived from reproducibility precision (interlaboratory tests) and counts for the repeatability error, run, and laboratory... 

If data are normally distributed, the mean and standard deviation are the best description possible of the data. Modern analytical chemistry is often automated to the extent that data are not individually scrutinized, and parameters of the data are simply calculated with a hope that the assumption of normality is valid. Unfortunately, the odd bad apple, or outlier, can spoil the calculations. Data, even without errors, may be more or less normal but with more extreme values than would be expected. These are known has heavy-tailed distributions, and the values at the extremes are called outliers. In interlaboratory studies designed to assess proficiency, the data often have outliers, which cannot be rejected out of hand. It would be a misrepresentation for a proficiency testing body to announce that all its laboratories give results within 2 standard deviations (except the ones that were excluded from the calculations). 

The results of these interlaboratory studies are reported in USEPA Method Validation Studies 14 through 24 (14). The data were reduced to four statistical relationships related to the overall study 1, multilaboratory mean recovery for each sample 2, accuracy expressed as relative error or bias 3, multilaboratory standard deviation of the spike recovery for each sample and 4, multilaboratory relative standard deviation. In addition, single-analyst standard deviation and relative standard deviation were calculated. 

Interlaboratory tests are routinely used to validate new analytical procedures—especially those intended for regulatory use. Typically, 5 to 10 laboratories are given identical samples and the same written procedure. If all results are similar, and there is no serious systematic error, then the method is considered reliable. ... 

Recent interlaboratory comparisons of HPLC and microbiological methods for vitamin B6 revealed significant variability among laboratories (42,70). The extraction and hydrolysis of the B6 vitamers, especially the pyridoxine-/ -glucoside (PNG) in plant-based foods, were cited as problem areas. Other sources of analytical error included HPLC (mis)identification of the individual B6 vitamers, and vitamer interconversion during extraction and analysis. 

One reason for interlaboratory differences in the reproducibility of immunostaining results is the use of microwave ovens with significant differences in age, power, construction, and design. Individual laboratories should optimize wattages and duration of heating as well as durations of each of the steps mentioned above some of them will need to be determined by trial and error. 

Accuracy can also be demonstrated through participation in properly conducted interlaboratory studies, which are also useful to detect systematic errors (Gtinzler 1996) related to, e.g. sample pretreatment (e.g. extraction, clean-up), final measurement (e.g. calibration error, spectral interference) and laboratory competence. As described below, interlaboratory studies are organised in such a way that several laboratories analyse a common material which is distributed by a central laboratory responsible for the data collection and evaluation. 

The sources of error that are likely to occur in speciation analysis have been discussed extensively in technical meetings over the past few years and recent publications describe in detail the various pitfalls that were detected in the context of interlaboratory studies (Quevauviller et al., 1992a 1996a Quevauviller,... 

Statistics should follow the technical scrutiny, not the other way round. A statistical analysis of data of an interlaboratory study cannot explain deviating results nor can alone give information on the accuracy of the results. Statistics only treat a population of data and provide information on the statistical characteristics of this population. The results of the statistical treatment may give rise to discussions on particular data not belonging to the rest of the population, but outlying data can sometimes be closer to the true value than the bulk of the population (Griepink et al., 1993). If no systematic errors affect the population of data, various statistical tests may be applied to the results, which can be treated either as individual data or as means of laboratory means. When different methods are applied, the statistical treatment is usually based on the mean values of replicate determinations. Examples of statistical tests used for certification purposes are described elsewhere (Horwitz, 1991). Together with the technical evaluation of the results, the statistical evaluation forms the basis for the conclusions to be drawn and the possible actions to be taken. 

The comparisons may take place every time a measurement is made (e.g., calibration of an analytical measurement using a standard solution), periodically (e.g., calibration of the balance), or infrequently (e.g., validation of a method). The reference value is used to either calibrate the process or to check its calibration or validity. The number of steps in the chain of comparisons should be kept to a minimum as each additional step introduces additional errors and increases the overall uncertainty. Interlaboratory comparisons provide evidence of comparability and provide confidence in traceability claims they do not, however, provide traceability directly. 

Interlaboratory studies that have been made on a common sample have shown that the errors associated with quantitative analysis by chromatography are much larger than expected. One fairly recent study is typical.4 Two samples were prepared and sent to 78 labs for analysis by reverse phase LC. When the data were analyzed, it was found that ... 

The largest commercially available datasets are the Physical Properties (PHYSPROP) and AQUASOL databases ca. 6000 compounds in each database). The AQUASOL database has been published as a book. Furthermore, two relatively large sets of aqueous solubility data models were used in many other studies.Data from the AQUASOL database had an interlaboratory variation of about a = 0.49 log-units (as estimated for A=1031 molecules).Moreover, large inter-laboratory errors mask the influence of temperature, and differences as large as AT = 30 °C do not increase this error. In-house models developed at pharmaceutical companies could be based on similar or even larger numbers of measurements. For example, about 5000 molecules were used to develop a model at Bayer Healthcare AG. " ... 

An interlaboratory study was made on six samples of rocks and pottery analyses were performed in Berkeley and Jerusalem. Table I gives an abbreviated list of the results in which the error limits shown are the counting errors alone. From a practical viewpoint, the agreements among data are quite satisfactory. However, in terms of ultimate possibilities, there are still minor problems. For example, the scandium values for KERMl have counting errors of only about 0.25%, yet the difference between the values is 2%. 

The Z-scores achieved by the participants are displayed in Fig. 4 for all the samples investigated. Eleven laboratories completed the interlaboratory comparison successfully (80% of Z-scores within Z < 2). The between-labo-ratory standard deviations are better than the target values for tolerable error set by PLC-4 (Table 2). Unsatisfactory Z-scores of individual laboratories were mainly a consequence of low recoveries, in particular for samples containing interfering substances. As a consequence, correction of analytical results for recovery, regularly determined by analysis of spiked water samples, will be conducted within the PLC-4 programme. 

Laessig RH, Ehrmeyer SS. Use of computer modefing to predict the magnitude of ultralaboratory error tolerated by proposed CDC interlaboratory proficiency testing performance criteria. Clin Chem 1988 34 1849-53. 

---