Method Performance Characteristics

The recommendations on design of method validation experiments contained in this chapter represent the authors assessment of current best practices, and these should be considered in the context of any subsequent guidance that may be provided by lUPAC or other authoritahve scientific bodies. As guidance is constantly evolving, it is important that a check be undertaken to ensure that the most recent relevant guidance is consulted before planning any method validation studies. 

The current list of performance characterishcs or related factors (discussed in greater detail in Sechon 8.7) to be considered in planning a method validation according to recom-mendahons from the Codex Alimentarius Commission includes  

Analyte stability (see Sections 8.5.4, 8.5.6, 8.5.7) Ruggedness testing/robustness (see Section 8.5.8) Calibration curve (see Section 8.7.1) 

Method comparisons, CRMs (see Section 8.7.4) Limit of detection (see Section 8.7.9) 

The EC requirements do not include limits of detection and quantification or MU, but instead require the determination of other statistical indicators of result reliability, that is, the decision limit (CCa) and the detection capability (CCP) as discussed in Section 8.7.10. 

---

There is current discussion on an international basis whereby the present procedure by which specific methods of analysis are incorporated into legislation are replaced by one in which method performance characteristics are specified. This is because by specifying a single method ... 

Approaches for Evaluating Acceptable Methods of Analysis Method Performance Characteristics and Criteria Approach Analytical Quality Assurance... 

A simplified approach to assess MU is the JUriess-for-purpose approach, defining a single parameter called the fitness function. This fitness function has the form of an algebraic expression u=f(c) and describes the relationship between the MU and the concentration of the analyte. For example, = 0.05c means that the MU is 5% of the concentration. Calculation of the MU will hereby rely on data obtained by evaluating individual method performance characteristics, mainly on repeatability and reproducibility precision, and preferably also on bias [21,40,41]. This approach can more or less be seen as a simplification of the step-by-step protocol for testing the MU, as described by Eurachem [14]. 

The purpose of an analytical method is the deliverance of a qualitative and/or quantitative result with an acceptable uncertainty level. Therefore, theoretically, validation boils down to measuring uncertainty . In practice, method validation is done by evaluating a series of method performance characteristics, such as precision, trueness, selectivity/specificity, linearity, operating range, recovery, LOD, limit of quantification (LOQ), sensitivity, ruggedness/robustness, and applicability. Calibration and traceability have been mentioned also as performance characteristics of a method [2, 4]. To these performance parameters, MU can be added, although MU is a key indicator for both fitness for purpose of a method and constant reliability of analytical results achieved in a laboratory (IQC). MU is a comprehensive parameter covering all sources of error and thus more than method validation alone. 

The literature gives a wide range of practical guidelines for the evaluation of method performance characteristics [58]. Besides the diversity of approaches, also the terminology and way of reporting results vary widely. Differences may occur depending on the purpose and the application field of the method, and validation studies may become more difficult as the complexity of the analysis increases [86]. In what follows, terms and formulas are taken from the accepted IUPAC nomenclature for the presentation of results of chemical analysis [66]. For each validation parameter, definitions, ways of expression, determination guidehnes, and acceptance criteria are reported in Table 5. 

Use of Standardized Methods The first level of AQA is the use of validated or standardized methods. The terms validated and standardized here refer to the fact that the method performance characteristics have been evaluated and have proven to meet certain requirements. At least, precision data are documented, giving an idea of the uncertainty and thus of the error of the analytical result. In both validated and standardized methods, the performance of the method is known. 

Due to the demand for reliable and comparable methods, performance requirements have been established at a national and international level for implementation of official methods, e.g. by European legislation, by the CEN or the Association of the Analytical Communities (AOAC) International, and worldwide by Codex Alimen-tarius (CAC). Thus any method proposed to be used for official purposes must be validated in a collaborative trial study, resulting in defined method performance characteristics [4], The framework for the design and conduct of such collaborative trial studies, as well as the statistical evaluation, are also defined in appropriate protocols [5]. Any method that has been successfully validated according to these protocols can be recognised as an official method for use in legal cases or for international trade control purposes. 

The purpose of this protocol is to define the method performance characteristics to be observed, the design and execution of the validation experiments, the data analysis, and the acceptance criteria for the validation of SOP 123 for use on Product W. 

To validate SOP 123 for measuring mass of Product W, the quantitative method performance characteristics of accuracy, precision, linearity, and range will be assessed using the validation assays shown in the design matrix over two days and using two operators. As per ICH guideline Q2A, the validation experiments will be run from 40 pg to 180 jUg. The test lot will be diluted out and concentrated up to specific expected masses using the mass cited on certificate of analysis for the lot of Product W selected for the validation. The points on the Product W curve will be as follows 40 /xg, 50 /xg, 70 /xg, 90 /xg, 110 /xg, 130 pg, 150 pg, and 180 pg. 

Other method performance characteristics should be addressed as appropriate. The reader is encouraged to refer to ICH Q2A... 

Scope. The scope of assay validation is to assess the essential test method performance characteristics of accuracy, reproducibility, repeatability, linear-ity, and limit of quantitation/detection [41]. A test method is evaluated for its readiness for assay validation against the following criteria (1) description of the test basis and scientific purpose, (2) case for relevance, (3) proposed practical application,... 

A method performance characteristic generally expressed in terms of the signal or measnrement (true) value... 

In the EU, and also in other countries that have adopted regulations or guidelines based on the EU approach, the interpretation of analytical results and regulatory decisions is made on the basis of those results that depend on the method performance characteristics known as the detection capability and the decision limit. The detection capability (CCfi) is defined in point 1.12 of the Annex to Commission Decision 2002/657/EC. CCP as the smallest content of the analyte that may be detected, identified, and/or quantified in a sample with an error probability of p. The P error is the probability that the tested sample is truly non-compliant even though a compliant measurement has been obtained. For screening tests the P error (the false-compliant rate) should be <5%. 

A sample or standard used either singly or in replicate to monitor method performance characteristics. 

Valcarcel, M. Tuque de Castro, M. D. A Hierarchical Approach to Analytical Chemistry, Trends Anal. Chem., 1995,14, 242-250. Further details on evaluating analytical methods may be found in Wilson, A. L. The Performance-Characteristics of Analytical Methods, Part l-Talanta, 1970, 17, 21-29 Part ll-Talanta, 1970, 17, 31M4 Part lll-Talanta, 1973, 20, 725-732 Part IV-Talanta, 1974,21, 1109-1121. 

Black Powder. Black powder is mainly used as an igniter for nitrocellulose gun propellant, and to some extent in safety blasting fuse, delay fuses, and in firecrackers. Potassium nitrate black powder (74 wt %, 15.6 wt % carbon, 10.4 wt % sulfur) is used for military appHcations. The slower-burning, less cosdy, and more hygroscopic sodium nitrate black powder (71.0 wt %, 16.5 wt % carbon, 12.5 wt % sulfur) is used industrially. The reaction products of black powder are complex (Table 12) and change with the conditions of initia tion, confinement, and density. The reported thermochemical and performance characteristics vary greatly and depend on the source of material, its physical form, and the method of determination. Typical values are Hsted in Table 13. 

Product standards may stipulate performance characteristics, dimensions, quaUty factors, methods of measurement, and tolerances and safety, health, and environmental protection specifications. These are introduced principally to provide for interchangeabiUty and reduction of variety. The latter procedure is referred to as rationalization of the product offering, ie, designation of sizes, ratings, etc, for the attribute range covered and the steps within the range. The designated steps may foUow a modular format or a preferred number sequence. 

Nondestmctive tests differ from methods of laboratory analysis and testing where specimens are generally sectioned, broken, damaged, or destroyed. Nondestmctive tests can be performed on materials, components, and stmctures or systems that actually are to be used. Thus, effective use of NDE requires engineering knowledge of the stmcture, the performance characteristics, and service environment, as well as the test method. More complete information on all of the topics discussed herein are available (1 6). 

The purpose of controller tuning is to choose the correct controller constants to obtain the desired performance characteristics. This usually means that the control variables should be restored in an optimal way to acceptable values, following either a change in the set point or the appearance of an input disturbance. Simulation examples TEMPCONT and CONTUN, provide exercises for controller tuning using the methods explained below. 

Validation may mean different things to different people, depending on the context and the application of analytical science. For food control and monitoring purposes, it is generally expected that validation includes the establishment of performance characteristics and evidence that the method fits the respective purpose. ... 

As discussed before, the efficiency of the extraction step is one of the fundamental performance characteristics of an analytical method. Unfortunately, the provisions regarding extraction efficiency in Council Directive 91/414/EEC (amended by Directive 96/68/EG) are listed in the metabolism section of the directive (Annex IIA 6.1 and 6.2). Nevertheless, results obtained in these studies are essential for the development of enforcement methods and must be reflected in this context. 

The assessment of validation data of CEN methods does not differ significantly from other validation schemes. The most important quantitative performance characteristics are trueness and precision. Additionally, some information about sensitivity... 

---