Precision, Bias and Accuracy

The concepts of precision, bias and accuracy were introduced in Chapter 4. However, as they are important in the context of evaluating measurement uncertainty it is worth revisiting them. 

Precision is the closeness of agreement between independent test results obtained under stipulated conditions. The precision tells us by how much we can expect the results of repeated measurements to vary. The precision of a set of measurement results will depend on the magnitude of the random errors affecting the measurement process. Precision is normally expressed as a standard deviation or relative standard deviation (see Section 6.1.3). 

Bias is the difference between the mean value of a number of test results and an accepted reference value. The magnitude of the bias will depend on the size and direction of systematic errors. 

Bias is a measure of trueness . It tells us how close the mean of a set of measurement results is to an assumed true value. Precision, on the other hand, is a measure of the spread or dispersion of a set of results. Precision applies to a set of replicate measurements and tells us how the individual members of that set are distributed about the calculated mean value, regardless of where this mean value lies with respect to the true value. 

Accuracy is the closeness of the agreement between the result of a measurement and the true value of the quantity being measured. Accuracy is the property of a single measurement result. It tells us how close a single measurement result is to the true value and therefore includes the effect of both precision and bias. 

---

To be aware of the meaning of the terms uncertainty , error , precision , bias and accuracy . 

Figure 6.12 illustrates the difference between precision, bias and accuracy. In examples (a) and (b), there is no bias. However, the precision in case (b) is better than in case (a) (i.e. the dispersion of results is smaller). Individual results obtained in case (b) would therefore be considered to be more accurate than those... 

Figure 6.12 Illustration of precision, bias and accuracy (a) not biased, not precise (b) not biased, precise, accurate (c) biased, not precise (d) biased, precise.

Cjood precision, which has been defined as the close agreement of measured values, is indicated b> a low standard deviation. It is an essential requirement. Good precision and low or /cri> bias equate to good or high accuracy. See Annex C (to be discussed later) for more on precision, bias, and accuracy. The term imcenainty is also frequently used as a surrogate for accuracy in an inverse sense, i.e.. low or acceptable uncertainty is equivalent to high accuracy. Section 8 contains an expanded discussion on these concepts. 

If the random errors are higher than can be tolerated to meet the goals of the test, the errors can be compensated for with rephcate measurements and a commensurate increase in the laboratory resources. Measurement bias can be identified through submission and analysis of known samples. Establishing and justifying the precision and accuracy reqrtired by the laboratory is a necessary part of estabhshing confidence. 

Definition and Uses of Standards. In the context of this paper, the term "standard" denotes a well-characterized material for which a physical parameter or concentration of chemical constituent has been determined with a known precision and accuracy. These standards can be used to check or determine (a) instrumental parameters such as wavelength accuracy, detection-system spectral responsivity, and stability (b) the instrument response to specific fluorescent species and (c) the accuracy of measurements made by specific Instruments or measurement procedures (assess whether the analytical measurement process is in statistical control and whether it exhibits bias). Once the luminescence instrumentation has been calibrated, it can be used to measure the luminescence characteristics of chemical systems, including corrected excitation and emission spectra, quantum yields, decay times, emission anisotropies, energy transfer, and, with appropriate standards, the concentrations of chemical constituents in complex S2unples. 

Validate routine methods, i.e., define the conditions under which the assay results are meaningful.115 To do that, one must select samples that are truly representative of the product stream. This may be a difficult task when the process is still under development and the product stream variable. The linearity of detector response should be defined over a range much broader than that expected to be encountered. Interference from the sample matrix and bias from analyte loss in preparation or separation often can be inferred from studies of linearity. Explicit detection or quantitation limits should be established. The precision (run-to-run repeatability) and accuracy (comparison with known standards) can be estimated with standards. Sample stability should be explored and storage conditions defined. 

The ISO recommendation [1993] should be followed and accuracy used only as a qualitative term. In case of quantitative characterization (by means of the bias), a problem may appear which is similar to that of precision, namely that a quality criterion is quantified by a measure that has a reverse attribute regarding the property which have to be characterized. If the basic idea of measures can be accepted, which is that a high quality becomes a high value and vice versa, bias is an unsuited measure of accuracy (and trueness). In this sense, accuracy could be defined by means of a measure proposed in the next paragraph. 

In this chapter we use the terms precision and accuracy in relation to the finite sampling variance and bias, respectively. Also, we describe the overall quality of an estimator - the mean square error - by the term reliability. Note the difference between our terminology and that in some statistics literature where accuracy is used to describe the overall quality (i.e., the reliability in this chapter). The decomposition of the error into the variance and bias allows us to use different approaches for studying the behavior of each term. 

The primary goal of this series of chapters is to describe the statistical tests required to determine the magnitude of the random (i.e., precision and accuracy) and systematic (i.e., bias) error contributions due to choosing Analytical METHODS A or B, and/or the location/operator where each standard method is performed. The statistical analysis for this series of articles consists of five main parts as ... 

Samples 1 and 2, but not for Samples 4 and 5. This indicates that the difference between precision and accuracy is large enough to indicate a bias inherent within the analytical method(s). Since these are the same methods and locations tested, further evaluation is required to determine if a bias actually exists. 

The section following shows a statistical test (text for the Comp Meth MathCad Worksheet) for the efficient comparison of two analytical methods. This test requires that replicate measurements be made on two different samples using two different analytical methods. The test will determine whether there is a significant difference in the precision and accuracy for the two methods. It will also determine whether there is significant systematic error between the methods, and calculate the magnitude of that error (as bias). 

Accuracy is often used to describe the overall doubt about a measurement result. It is made up of contributions from both bias and precision. There are a number of definitions in the Standards dealing with quality of measurements [3-5]. They are only different in the detail. The definition of accuracy in ISO 5725-1 1994, is The closeness of agreement between a test result and the accepted reference value . This means it is only appropriate to use this term when discussing a single result. The term accuracy , when applied to a set of observed values, describes the consequence of a combination of random variations and a common systematic error or bias component. It is preferable to express the quality of a result as its uncertainty, which is an estimate of the range of values within which, with a specified degree of confidence, the true value is estimated to lie. For example, the concentration of cadmium in river water is quoted as 83.2 2.2 nmol l-1 this indicates the interval bracketing the best estimate of the true value. Measurement uncertainty is discussed in detail in Chapter 6. 

A quantitative procedure should be validated for selectivity, calibration model, stability, accuracy (bias, precision), linearity, and limit of quantification (LOQ). Additional... 

Once initial analyses are completed, random samples are sent from SGS to ActLabs for check assays, to establish precision (repeatability) and analytical bias. Additionally, coarse sample rejects are chosen at random and sent to ActLabs for preparation and analysis, to check the accuracy and repeatability of the original sample preparation. A further check on SGS Lab precision is conducted by renumbering pulps and re-submission from ActLab to SGS for analysis. Tournigan monitors quality assurance by plotting and analyzing the data, as received, and activates re-assaying of sample batches that do not meet predetermined standards. 

Interlaboratoiy tests are a test for accuracy. Inaccuracy arises from systematic and random effects that are related to bias and precision respectively. As a result of a PT the laboratory should be able to determine, whether imprecision or bias is the reason for its inaccuracy. 

Define Quality Control, Quality Assurance, sample, analyte, validation study, accuracy, precision, bias, calibration, calibration curve, systematic error, determinate error, random error, indeterminate error, and outlier. 

Precision and Accuracy. Statistical requirements for precision and accuracy have been established to ensure that the method is reproducible and free from bias. 

The ISO Guide 3534-1 [57] defines accuracy as "the closeness of agreement between a test result and the accepted reference value", with a note stating that the term accuracy, when applied to a set of test results, involves a combination of random components and a common systematic error or bias component". Accuracy is expressed, then, as two components trueness and precision. Trueness is defined as the closeness of agreement between the average value obtained from a large set of test results and an accepted reference value" and it is normally expressed in terms of bias. Finally, precision is defined as the closeness of agreement between independent test results obtained under stipulated conditions". 

Calibrations were carried out for the GC/PID or the GC/MS daily. Calibration standards were prepared based on standard reference materials obtained from Supelco Chromatography products. A check standard was analyzed every ten samples to assure calibration and accuracy. A reagent blank was included in each analytic batch of samples. Blanks were made from reagent or make-up water and matrix similar to the sample. A spiked sample was analyzed every twenty samples. This was done by splitting an appropriate sample into two subsamples and adding a known quantity of TCE to one of the split samples. The purpose of a spiked sample is to determine the extent of matrix bias or interference on TCE recovery and sample to sample precision. Accuracy was assessed by analysis of external reference standards (separate from calibration standards) and by percent recoveries of spiked samples. Precision was assessed by means of replicate sample analysis. It is expressed as relative percent difference (RPD) in the case of duplicates or relative standard deviation (RSD) for triplicate (or more) analyses. Recovery was 96% or more for all spiked samples, and RPD/RSD are less than 7% for all samples. 

The determination of precision and accuracy is an important part of environmental analysis because it indicates the degree of bias or any error in the measurement. 

Fig. 2 Results of manufacturer comparisons for total cholesterol performed from January 2000 through March 2003. Sixteen manufacturers performed 79 comparisons. Bias versus the Cholesterol Reference Method Laboratory Network (CRMLN) laboratory is plotted on the a-ax is and the coefficient of variation (CV) (%) is plotted on the y-axis. Vertical lines at -3% and +3% bias and the horizontal line at 3% CV indicate the National Cholesterol Education Program recommendations for accuracy and precision for clinical laboratories...

Apart from the fact that a linear calibration can be performed, bracketing offers excellent precision and accuracy. With the determination of serum cholesterol as an example, Cohen et al. (1980) showed that the replication error on five different serum pools was characterized by a CV of 0.17% with a set-to-set variability of 0.32%. For each serum average, a standard error (considering all causes of variability combined) of 0.16% CV was obtained. The undetected systematic error (bias) in this study was estimated to be smaller than 0.5%, while White et al. (1982), using two different IDMS methods, found serum glucose concentrations to agree within 1%. 

To optimize or compare measurement protocols, an index of quality must be quantified for each estimated parameter. For a parameter taken separately, the performance of estimation is a function of both the variance (or precision) and the bias (or accuracy) of the estimate. The covariance matrix of each unbiased estimator 0 checks the inequality of Frechet-Cramer-Rao 24... 

Accuracy is often used instead of bias and trueness. It can be seen from Figure 2 that it involves bias and precision. 

Accuracy is a term that is used loosely in daily conversation. It has particular meanings in analytical measurement. For example, under the current ISO definition, accuracy is a property of a result and comprises bias and precision. 

---