Trueness, estimation

ISO/TS 21748, 2004. Guidance for the use of repeatability, reproducibility and trueness estimates in measurement uncertainty estimation. 

International Standards Organization, ISO 21748, Guidance for the Use of Repeatability, Reproducibility and Trueness Estimates in Measurement Uncertainty Estimation, Geneva, 2004. 

Trueness. Absence of systematic errors can be tested traditionally by means of recovery functions see Sect. 6.1.2, Fig. 6.3C Burns et al. [2002]. For this reason the concentration estimated by the model, x, is compared with the true concentration value, xtrue, by a regression model... 

If a Reference Material (RM), without a certified value or if an in-house material is used, there is no certified, traceable value. However, the procedure is the same. The value assigned to the material by the laboratory itself is the best available estimate for the tme value. Of course the determination of trueness is not as reliable as with a CRM. 

After the (im)precision part of the uncertainty we now want to estimate the trueness (bias) part. For that we... 

Trueness or exactness of an analytical method can be documented in a control chart. Either the difference between the mean and true value of an analyzed (C)RM together with confidence limits or the percentage recovery of the known, added amount can be plotted [56,62]. Here, again, special caution should be taken concerning the used reference. Control charts may be useful to achieve trueness only if a CRM, which is in principle traceable to SI units, is used. All other types of references only allow traceability to a consensus value, which however is assumed not to be necessarely equal to the true value [89]. The expected trueness or recovery percent values depend on the analyte concentration. Therefore, trueness should be estimated for at least three different concentrations. If recovery is measured, values should be compared to acceptable recovery rates as outlined by the AOAC Peer Verified Methods Program (Table 7) [56, 62]. Besides bias and percent recovery, another measure for the trueness is the z score (Table 5). It is important to note that a considerable component of the overall MU will be attributed to MU on the bias of a system, including uncertainties on reference materials (Figures 5 and 8) [2]. 

As in traditional methods that use univariate calibrations, the description of a method of analysis that uses multivariate calibration must also include the corresponding estimated figures of merit, including accuracy (trueness and precision), selectivity, sensitivity, linearity, limit of detection (LOD), limit of quantification (LOQ) and robustness. In this chapter, only the most common figures of merit are described. For a more extensive review, see [55]. Also, for a practical calculation of figures of merit in an atomic spectroscopic application, see [12]. 

In the case of Cd in drinking water the performance criteria are clearly speci-bed as 10 percent for both trueness and precision. Therefore, it is rather surprising to see PT providers stating assigned standard deviations of 5, 7, 10, or even 14 percent for Cd in water, in compliance with the WFD. This is because some PT providers only consider trueness, others, in turn, combine trueness and precision. Some consider precision properly, others take it as a direct estimate of the normalization factor. To improve such situations, the European Union initiated the CoEPT project to study the differences and similarities in the operation of PT schemes and the evaluation of PT results in view of implementing a harmonized approach to provide a basis for the comparability of the PT schemes operated on the market [70]. 

Abstract A protocol has been developed illustrating the link between validation experiments, such as precision, trueness and ruggedness testing, and measurement uncertainty evaluation. By planning validation experiments with uncertainty estimation in mind, uncertainty budgets can be obtained from validation data with little additional effort. The main stages in the uncertainty estimation process are described, and the use of true-... 

The estimates obtained by applying Eqs. 8-10 are intended to give a first estimate of the measurement uncertainty associated with a particular parameter. If such estimates of the uncertainty are found to be a significant contribution to the overall uncertainty for the method, further study of the effect of the parameters is advised, to establish the true relationship between changes in the parameter and the result of the method. However, if the uncertainties are found to be small compared to other uncertainty components (i.e. the uncertainties associated with precision and trueness) then no further study is required. 

The individual sources of uncertainty, evaluated through the precision, trueness, ruggedness and other studies are combined to give an estimate of the standard uncertainty for the method as a whole. Uncertainty contributions identified as being proportional to analyte concentration are combined using Eq. (11) ... 

The main disadvantage of this approach is that it may not readily reveal the main sources of uncertainty for a particular method. In previous studies we have typically found the uncertainty budget to be dominated by the precision and trueness terms [14]. In such cases, if the combined uncertainty for the method is too large, indicating that the method requires improvement, further study may be required to identify the stages in the method which contribute most to the uncertainty. However, the approach detailed here will allow the analyst to obtain, relatively quickly, a sound estimate of measurement uncertainty, with minimum experimental work beyond that required for method validation. 

In Part 1 [1] we described a protocol for the evaluation of measurement uncertainty from validation studies such as precision, trueness and ruggedness testing. In this paper we illustrate the application of the protocol to a method developed for the determination of the dyes Cl solvent red 24 and Cl solvent yellow 124, and the chemical marker quinizarin (1,4-dihydroxyanthra-quinone) in road fuel. The analysis of road fuel samples suspected of containing rebated kerosene or rebated gas oil is required as the use of rebated fuels as road fuels or extenders to road fuels is illegal. To prevent illegal use of rebated fuels, HM Customs and Excise require them to be marked. This is achieved by adding solvent red 24, solvent yellow 124 and quinizarin to the fuel. A method for the quantitation of the markers was developed in this laboratory [2]. Over a period of time the method had been adapted to improve its performance and now required re-validation and an uncertainty estimate. This paper describes the experiments under-... 

In addition, the results obtained from the replicate analysis of a sample of BP diesel, prepared for the trueness study (see below), were used in the estimate of uncertainty associated with method precision. 

In evaluation of the performance characteristics of a candidate method, precision, accuracy (trueness), analytical range, detection limit, and analytical specificity are of prime importance. The sections in this chapter on method evaluation and comparison contain a detailed outline of these concepts and their assessment. The estimated performance parameters for a method can then be related to quality goals that ensure acceptable medical use of the test results (see section on Analytical Goals), From a practical point of view, the ruggedness of the method in routine use is of importance. Reliable performance when used by different operators and with different batches of reagents over longer time periods is essential. 

CRMs to finalise the method development, to validate analytical procedures and finally control in time the accuracy of procedures, are rare and valuable materials, in particular matrix CRMs. They should tell the analyst how his entire measurement procedure is performing. He will receive information on precision as well as on trueness. CRMs are primarily developed to check for trueness, which is the most difficult property to verify. Precision can be tested on RMs or can be estimated from published data e.g. the performance required by a standard method, whereas the evaluation of trueness is possible only with external help a CRM or a properly organised interlaboratory study. Having a CRM allows one to perform the verification of trueness whenever the operator wants it. The analyst should never forget that only when accurate results (precise and true) are achieved, comparability in space and over time is guaranteed. But to exploit to a maximum the information on trueness delivered by the CRM, the precision must also be sufficient and verified. 

As already mentioned, the main advantage of CRMs lies in the availability of the true value . In other words, it is a reference value that is considered as the best estimate analytical sciences can give for the real content of the substance in the particular material. As such CRMs represent the only way to check trueness easily. The verification of trueness will consist in a statistical comparison of the value determined by the operator on the CRM and the certified value. Again it must be stressed that this has no sense if the method s precision is too large. The trueness of a method can be verified within the laboratory or through an interlaboratory study, in particular when a reference standard or an official method is concerned. 

As already discussed in Chapter 4, CRMs can be used to assess the precision and trueness of a method. Precision has already been largely studied and improved by using proper internal validation procedures and can be maintained through appropriate control charts. For all those activities, simple RMs are sufllcient. To assess trueness, the analyst has to look for external help. One simple way is to purchase a CRM. This CRM will help him to solve his accuracy problems if two conditions are fulfilled. First, he must choose an adapted CRM — representative of the daily routine samples secondly, it must be properly certified — the certified value must be a good estimate of the true value [1] ... 

The primary performance measures of a ligand-binding assay are bias/trueness and precision. These measures along with the total error are then used to derive and evaluate several other performance characteristics such as sensitivity (LLOQ), dynamic range, and dilutional linearity. Estimation of the primary performance measures (bias, precision, and total error) requires relevant data to be generated from a number of independent runs (also termed as experiments or assay s). Within each run, a number of concentration levels of the analyte of interest are tested with two or more replicates at each level. The primary performance measures are estimated independently at each level of the analyte concentration. This is carried out within the framework of the analysis of variance (ANOVA) model with the experimental runs included as a random effect [23]. Additional terms such as analyst, instmment, etc., may be included in this model depending on the design of the experiment. This ANOVA model allows us to estimate the overall mean of the calculated concentrations and the relevant variance components such as the within-run variance and the between-run variance. 

These experiments are designated as phases II and III of a method validation or an analyst familiarization for those using the approach recommended in the USDA/FSIS Chemistry Laboratories Guidebook (see QA section of the posted methods)7 The results provide an assessment of the recovery (trueness) and the analyst precision attained with the method under routine conditions of use. In addition, the data generated may be used to calculate statistical estimates of the reliability of the results, including estimates of MU. i ... 

Recovery and Its Uncertainty. The trueness of the method must be considered in the calculation of the uncertainty estimate to cover uncertainties due to method bias. Barwick and Ellison describe several possibilities for estimating uncertainty related to trueness, including the use of data from the analysis of a representative certified reference material, comparison with a reference or standard method, and spiking recovery studies. Since no representative certified reference material was available for this analyte-matrix combination, the trueness was estimated using spiking studies. As stated in the previous section, the spiked samples in this example were assumed to be representative of the incurred tissue. 

Estimation of trueness and precision The EU directive (Council Directive 98/83/EC, 1998) refers to the definitions for the determination of trueness and precision given in ISO 5725-1 (ISO, 2002). According to this standard, the estimation of the trueness requires a large series of replicate determinations of test samples. But what does Targe series mean Neither ISO 5725-1 nor the EU directive resolve this question. The determination of precision also requires a number of replicate determinations of a test sample. Information on a recommended number of replicates is also missing in ISO 5725-1. 

For the estimation of trueness, ENV-ISO 13530 recommends regular participation in external quality procedures such as interlaboratory trials and proficiency schemes for the control of trueness (bias). For internal routine action, the use of control charts, based on the mean, spiking recovery, and analysis of blanks, is recommended. In addition, the standard recommends the use of a mean and/or a range control chart and the execution of a minimum of six replicate determinations of the test sample for the calculation of the standard deviation for the control of the precision. 

Examples for the estimation of laboratory internal performance data ENV-ISO 13530 (ISO/TR, 2003) can be applied to ascertain the laboratory internal values for precision, trueness and LOD aceording to the EU directive 98/83/EC. The examples in Table 1.4 m give an indication for the internal actions to be applied by the laboratory. 

The use of CRMs for validation purposes is, however, not limited to the above intralaboratory verification of trueness (checking the absence of significant systematic errors). They also enable the user to estimate the precision of a method (repeatability and reproducibility), which should actually represent one of the first steps of the method validation. In this respect, the evaluation will have to take into account specific characteristics of the CRM, in particular, possible sources of uncertainties linked to the material heterogeneity which should in principle be considered for the calculation of the uncertainty of the certified values. 

---