Trueness, method validation

Accuracy expresses the closeness of a result to the true value. Accuracy = trueness + precision. Under specific conditions it is quantified by the measurement uncertainty. Measurement uncertainty may vary under changing conditions and method validation determines the degree. 

Method validation seeks to quantify the likely accuracy of results by assessing both systematic and random effects on results. The properly related to systematic errors is the trueness, i.e. the closeness of agreement between the average value obtained from a large set of test results and an accepted reference value. The properly related to random errors is precision, i.e. the closeness of agreement between independent test results obtained under stipulated conditions. Accnracy is therefore, normally studied as tmeness and precision. 

The purpose of an analytical method is the deliverance of a qualitative and/or quantitative result with an acceptable uncertainty level. Therefore, theoretically, validation boils down to measuring uncertainty . In practice, method validation is done by evaluating a series of method performance characteristics, such as precision, trueness, selectivity/specificity, linearity, operating range, recovery, LOD, limit of quantification (LOQ), sensitivity, ruggedness/robustness, and applicability. Calibration and traceability have been mentioned also as performance characteristics of a method [2, 4]. To these performance parameters, MU can be added, although MU is a key indicator for both fitness for purpose of a method and constant reliability of analytical results achieved in a laboratory (IQC). MU is a comprehensive parameter covering all sources of error and thus more than method validation alone. 

If a method must be developed from scratch, or if an established method is changed radically from its original published form, then before the method is validated, the main task is simply to get the method to work. This means that the analyst is sure that the method can be used to yield results with acceptable trueness and measurement uncertainty (accuracy). When the analyst is satisfied that the method does work, then the essentials of method validation will also have been done, and now just need to be documented. If there is an aspect of the method that does not meet requirements, then further development will needed. Discovering and documenting that the method now does satisfy all requirements is the culmination of method validation. 

Due to the high workload of analysing such large series, trueness is usually not determined during method validation, but rather from the results of a great number of quality control (QC) samples during routine application or in interlaboratory studies. 

Key words Measurement uncertainty Method validation Precision Trueness Ruggedness testing... 

The main disadvantage of this approach is that it may not readily reveal the main sources of uncertainty for a particular method. In previous studies we have typically found the uncertainty budget to be dominated by the precision and trueness terms [14]. In such cases, if the combined uncertainty for the method is too large, indicating that the method requires improvement, further study may be required to identify the stages in the method which contribute most to the uncertainty. However, the approach detailed here will allow the analyst to obtain, relatively quickly, a sound estimate of measurement uncertainty, with minimum experimental work beyond that required for method validation. 

We have applied this protocol to the evaluation of the measurement uncertainty for a method for the determination of three markers (Cl solvent red 24, Cl solvent yellow 124 and quinizarin (1,4-dihydroxyanthra-quinone)) in road fuel. The method requires the extraction of the markers from the sample matrix by solid phase extraction, followed by quantification by HPLC with diode array detection. The uncertainty evaluation involved four experimental studies which were also required as part of the method validation. The studies were precision, trueness (evaluated via the analysis of spiked samples) and ruggedness tests of the extraction and HPLC stages. The experiments and uncertainty calculations are described in detail in Part 2. A summary of the uncertainty budget for the method is presented in Fig. 3. 

Method validation makes use of a series of tests to determine its performance characteristics and to establish the method s acceptance for general use. The following are a list of criteria associated with the validation of a method selectivity and specificity, linearity and calibration, accuracy or trueness, range, precision, limit of detection, limit of quantification, ruggedness and application. 

These experiments are designated as phases II and III of a method validation or an analyst familiarization for those using the approach recommended in the USDA/FSIS Chemistry Laboratories Guidebook (see QA section of the posted methods)7 The results provide an assessment of the recovery (trueness) and the analyst precision attained with the method under routine conditions of use. In addition, the data generated may be used to calculate statistical estimates of the reliability of the results, including estimates of MU. i ... 

The use of CRMs for validation purposes is, however, not limited to the above intralaboratory verification of trueness (checking the absence of significant systematic errors). They also enable the user to estimate the precision of a method (repeatability and reproducibility), which should actually represent one of the first steps of the method validation. In this respect, the evaluation will have to take into account specific characteristics of the CRM, in particular, possible sources of uncertainties linked to the material heterogeneity which should in principle be considered for the calculation of the uncertainty of the certified values. 

Method validation makes use of a set of tests that both test any assumptions on which the analytical method is based and establish and document the performance characteristics of a method, thereby demonstrating whether the method is fit for a particular analytical purpose. Typical performance characteristics of analytical methods are applicability, selectivity, calibration, trueness, precision, recovery, operating range, limit of quantification, limit of detection, sensitivity. 

Characterisation of method performance involves a judgement as to whether the capabilities of the new method are sufficient to meet the needs of the end user (this is also known as method validation). Various options exist for characterisation of method performance. The trueness of a new method could be assessed against that of established methods, repeatability could be assessed using reference materials, and reproducibility through interlaboratory comparisons. In R D, many of these options may not readily be available. Validation tools may be limited to the use of in-house reference materials. 

The assessment of validation data of CEN methods does not differ significantly from other validation schemes. The most important quantitative performance characteristics are trueness and precision. Additionally, some information about sensitivity... 

Komit6 for Levnedsmidler (NMKL)]. The standard presents a universal validation approach for chemical analytical methods in the food sector. This includes methods for the main constituents and also for trace components. Therefore, the NMKL procedure focuses on primary validation parameters, such as specificity, calibration, trueness, precision, LOD or LOQ and does not refer to special requirements of pesticide residue analysis. 

Even if most examples and procedures presented apply to in-house validation, the procedure does not distinguish between validations conducted in a single laboratory and those carried out within inter-laboratory method performance studies. A preference for inter-laboratory studies can be concluded from the statement that laboratories should always give priority to methods which have been tested in method performance studies. Within the procedure a profound overview of different categories of analytical methods according to the available documentation and previous external validation is given. For example, if a method is externally validated in a method performance study, it should be tested for trueness and precision only. On the other hand, a full validation is recommended for those methods which are published in the scientific literature without complete presentation of essential performance characteristics (Table 9). 

Verification implies that the laboratory investigates trueness and precision in particular. Elements which should be included in a full validation of an analytical method are specificity, calibration curve, precision between laboratories and/or precision within laboratories, trueness, measuring range, LOD, LOQ, robustness and sensitivity. The numbers of analyses required by the NMKL standard and the criteria for the adoption of quantitative methods are summarized in Table 10. 

In summary, official German analytical methods for pesticide residues are always validated in several laboratories. These inter-laboratory studies avoid the acceptance of methods which cannot readily be reproduced in further laboratories and they do improve the ruggedness of analytical procedures applied. The recently introduced calibration with standards in matrix improves the trueness of the reported recovery data. Other aspects of validation (sample processing, analyte stability, extraction efficiency) are not considered. 

If analytical methods are validated in inter-laboratory validation studies, documentation should follow the requirements of the harmonized protocol of lUPAC. " However, multi-matrix/multi-residue methods are applicable to hundreds of pesticides in dozens of commodities and have to be validated at several concentration levels. Any complete documentation of validation results is impossible in that case. Some performance characteristics, e.g., the specificity of analyte detection, an appropriate calibration range and sufficient detection sensitivity, are prerequisites for the determination of acceptable trueness and precision and their publication is less important. The LOD and LOQ depend on special instmmentation, analysts involved, time, batches of chemicals, etc., and cannot easily be reproduced. Therefore, these characteristics are less important. A practical, frequently applied alternative is the publication only of trueness (most often in terms of recovery) and precision for each analyte at each level. No consensus seems to exist as to whether these analyte-parameter sets should be documented, e.g., separately for each commodity or accumulated for all experiments done with the same analyte. In the latter case, the applicability of methods with regard to commodities can be documented in separate tables without performance characteristics. 

II the difference approach, which typically utilises 2-sided statistical tests (Hartmann et al., 1998), using either the null hypothesis (H0) or the alternative hypothesis (Hi). The evaluation of the method s bias (trueness) is determined by assessing the 95% confidence intervals (Cl) of the overall average bias compared to the 0% relative bias value (or 100% recovery). If the Cl brackets the 0% bias then the trueness that the method generates acceptable data is accepted, otherwise it is rejected. For precision measurements, if the Cl brackets the maximum RSDp at each concentration level of the validation standards then the method is acceptable. Typically, RSDn> is set at <3% (Bouabidi et al., 2010),... 

One or more of these bias components are encountered when analyzing RMs. In general, RMs are divided into certified RMs (CRMs, either pure substances/solu-tions or matrix CRMs) and (noncertified) laboratory RMs (LRMs), also called QC samples [89]. CRMs can address all aspects of bias (method, laboratory, and run bias) they are defined with a statement of uncertainty and traceable to international standards. Therefore, CRMs are considered useful tools to achieve traceability in analytical measurements, to calibrat equipment and methods (in certain cases), to monitor laboratory performance, to validate methods, and to allow comparison of methods [4, 15, 30]. However, the use of CRMs does not necessarely guarantee trueness of the results. The best way to assess bias practically is by replicate analysis of samples with known concentrations such as reference materials (see also Section 8.2.2). The ideal reference material is a matrix CRM, as this is very similar to the samples of interest (the latter is called matrix matching). A correct result obtained with a matrix CRM, however, does not guarantee that the results of unknown samples with other matrix compositions will be correct [4, 89]. 

On the one hand, even if an in-house vahdated method shows good performance and reliable accuracy, such a method cannot be adopted as a standard method. In-house validated methods need to be compared between at least eight laboratories in a collaborative trial. On the other hand, a collaborative study should not be conducted with an unoptimized method [58]. Interlaboratory studies are restricted to precision and trueness while other important performance characteristics such as specificity and LOD are not addressed [105]. For these reasons, single-laboratory validation and interlaboratory validation studies do not exclude each other but must be seen as two necessary and complementary stages in a process, presented in Figure... 

Precision plays a central role in collaborative studies. Wood [84] defines a collaborative trial as a procedure whereby the precision of a method of analysis may be assessed and quantified. Precision is the objective of interlaboratory validation studies, and not trueness or whichever other method performance parameter. Evalu-... 

The Nordic Committee on Food Analysis has published a guideline on the Validation of Chemical Analytical Methods which differentiates between external validation work carried out on published methods and that work required to transfer it into the working laboratory to confirm its suitability for use. Table 20 is adapted from this document. This document is most easily accessed from a recently published book If certified reference materials are available they should be used to confirm the verification of trueness. These guidelines could also apply to intra-laboratory training programmes. 

Anklam et al. [7] as well as Ahmed [8] recently published a comprehensive overview of different PCR assays that have been published in the literature. The authors tried to include performance data adding to the value of the review articles. The validation of PCR methods and thus the establishment of such performance criteria is still the subject of much debate. H bner et al. [9] suggested an approach for the validation of PCR assays. In general, it is currently the view of most researchers that validation of a PCR assay should not differ essentially from the validation of other analytical methods. Thus, all principles outlined in the ISO standard 17025 General requirements for the competence of testing and calibration laboratories, ISO standard 5725 Accuracy (trueness and precision) of measurement methods and results as well as the principles as laid down by Codex Alimentarius (http //www.co-dexalimentarius.net), are applicable to PCR. 

If the laboratory develops the validation method in-house, there always needs to be some sample to be used for this purpose a sample that best mimics routine samples is the most suitable. The usual practice is that a routine sample is used for this purpose as knowledge of the true value is not a critical issue at this stage. Next, the trueness of a method is usually determined by analyzing an appropriate CRM and/or participating in an ILC, one with an externally defined reference value.10... 

There are no official guidehnes on the sequence of validation experiments and the optimal sequence can depend on the method itself. A potentially useful sequence for a liquid chromatographic method is 1) Selectivity of standards (optimizing separation and detection of standard mixtures) 2) precision of retention times and peak areas 3) linearity, limit of quantitation, hmit of detection, range 4) selectivity with real samples 5) trueness or accuracy, at different concentrations 6) ruggedness. 

In Part 1 [1] we described a protocol for the evaluation of measurement uncertainty from validation studies such as precision, trueness and ruggedness testing. In this paper we illustrate the application of the protocol to a method developed for the determination of the dyes Cl solvent red 24 and Cl solvent yellow 124, and the chemical marker quinizarin (1,4-dihydroxyanthra-quinone) in road fuel. The analysis of road fuel samples suspected of containing rebated kerosene or rebated gas oil is required as the use of rebated fuels as road fuels or extenders to road fuels is illegal. To prevent illegal use of rebated fuels, HM Customs and Excise require them to be marked. This is achieved by adding solvent red 24, solvent yellow 124 and quinizarin to the fuel. A method for the quantitation of the markers was developed in this laboratory [2]. Over a period of time the method had been adapted to improve its performance and now required re-validation and an uncertainty estimate. This paper describes the experiments under-... 

Results verification is totally different from results validation. Results validation (point 4.7.5. and 5.9. of NBN-EN-ISO-CEI 17025 standard) shows, each year, or when it is judged necessary, that a given laboratory has the capacity to apply a particular method, repetitively, in respect of obtained data during initial validation. Trueness and statistical dispersion of results are the basis of the definition of the uncertainty of the standard of measurement [16] and, in some cases, the basis for the definition of the limit of detection and quantification. Management of data from validation results, as control card, could permit the detection and control of eventual deviation. Validation of results is the internal quality control procedure which verifies the stability of performance of the methods for which accreditation is sought, in the limited-scope procedural context. 

So, the methods characteristic of each test, comprising taken together a type of tests, must undergo validation testing of their results. This is the implementation of the method, and the establishment of a standard for its performance. For the standardization of quantitative methods, this consists at a minimum of a determination of trueness when blank utilization, certified reference materials (or reference materials, or spiking materials) or collaborative trials are used, repeatability (r) with repetition,... 

The determination of the property of interest in a simple solution will indicate to the analyst his working range in terms of sensitivity of the signal. The degree of trueness and precision are properties quantified in the validation process. Robustness or ruggedness concerns the ability of the method to remain unaffected by environmental changes (analyst, time, fluctuations in supplies, etc,). All six characteristics have to be established and quantified in the validation of the method (section 2.3). 

The validation has the objective to identify, during the method development process, all sources of error and eliminate them or to quantify their contribution to the total uncertainty of the determination. For trace organic determinations particular attention must be given to the quantitative extraction and clean-up of all PCBs. Several types of adapted materials must be prepared to test all steps of the process (from simple calibrant solutions or mixtures, spiked extracts, to spiked soil material). CRMs should be used for validating trueness. Laboratory RMs must be prepared for the establishment of control charts when the method is under statistical control. 

---