Validation ruggedness

Torbeck LD. Assay validation ruggedness and robustness with designed experiments. Pharm Technol 1996 20(3) 168-172. 

The first tests on the prototype demonstrated the efficiency of the system. Dysfunctional tests on the hardware and on the software made it possible to validate ruggedness and the high level of safety. The study also contributes towards controlling risks connected with hydrogen, the use of which is expanding. It shows the difficulty in certifying innovative products with present-day reference fi ames and standards. 

Of all the requirements that have to be fulfilled by a manufacturer, starting with responsibilities and reporting relationships, warehousing practices, service contract policies, airhandUng equipment, etc., only a few of those will be touched upon here that directly relate to the analytical laboratory. Key phrases are underlined or are in italics Acceptance Criteria, Accuracy, Baseline, Calibration, Concentration range. Control samples. Data Clean-Up, Deviation, Error propagation. Error recovery. Interference, Linearity, Noise, Numerical artifact. Precision, Recovery, Reliability, Repeatability, Reproducibility, Ruggedness, Selectivity, Specifications, System Suitability, Validation. 

In summary, official German analytical methods for pesticide residues are always validated in several laboratories. These inter-laboratory studies avoid the acceptance of methods which cannot readily be reproduced in further laboratories and they do improve the ruggedness of analytical procedures applied. The recently introduced calibration with standards in matrix improves the trueness of the reported recovery data. Other aspects of validation (sample processing, analyte stability, extraction efficiency) are not considered. 

Lastly, a laboratory not involved in the development process must validate the method. The independent laboratory validation study, or ruggedness trial, ensures that analysts unfamiliar with the method can successfully perform the method. The method developer should, therefore, strive to make all procedures as straightforward as possible to aid reproducibility of the method. 

Analytical methods, particularly those used by accredited laboratories, have to be validated according to official rules and regulations to characterize objectively their reliability in any special field of application (Wegscheider [1996] EURACHEM/WELAC [1993]). Validation has to control the performance characteristics of analytical procedures (see Chap. 7) such as accuracy, precision, sensitivity, selectivity, specificity, robustness, ruggedness, and limit values (e.g., limit of detection, limit of quantitation). 

A calibration procedure has to be validated with regard to general and specific requirements under which the calibration model has been developed. For this purpose, it is important to test whether the conditions represented in Fig. 6.6 are fulfilled. On the other hand, it is to assure by experimental studies that certain performance features (accuracy, precision, sensitivity, selectivity, specificity, linearity, working range, limits of detection and of quantification, robustness, and ruggedness, see Chap. 7) fulfil the expected requirements. 

Methods can only usefully applied in analytical practice when they are sufficiently robust and therefore insensitive to small variations in method conditions and equipment (replacement of a part), operator skill, environment (temperature, humidity), aging processes (GC- or LC columns, reagents), and sample composition. This demand makes robustness (ruggedness) to an important validation criterion that has to be proved by experimental studies. The concepts of robustness and ruggedness mostly have been described verbally where it must be stated that their use is frequently interchangeably and synonymously (e.g., Hendricks et al. [1996] Kellner et al. [1998] EURACHEM [1998] ICH [1994, 1996] Wunsch [1994] Wildner and Wunsch [1997] Valcarcel [2000] Kateman and Buydens [1993]). 

Evaluation of data and validation multivariate data analysis (MULTI-VAR, Wienke et al. [1991]), evaluation of interlaboratory studies (INTERLAB, Wienke et al. [1991]), ruggedness expert system (RES, van Leeuwen et al. [1991]). 

Typical performance characteristics that should be considered in the validation are precision, accuracy, limit of detection, limit of quantitation, selectivity, range, linearity, robustness, ruggedness... 

For non-compendial procedures, the performance parameters that should be determined in validation studies include specificity/selectivity, linearity, accuracy, precision (repeatability and intermediate precision), detection limit (DL), quantitation limit (QL), range, ruggedness, and robustness [6]. Other method validation information, such as the stability of analytical sample preparations, degradation/ stress studies, legible reproductions of representative instrumental output, identification and characterization of possible impurities, should be included [7], The parameters that are required to be validated depend on the type of analyses, so therefore different test methods require different validation schemes. 

Method validation is defined in the international standard, ISO/IEC 17025 as, the confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use are fulfilled. This means that a validated method, if used correctly, will produce results that will be suitable for the person making decisions based on them. This requires a detailed understanding of why the results are required and the quality of the result needed, i.e. its uncertainty. This is what determines the values that have to be achieved for the performance parameters. Method validation is a planned set of experiments to determine these values. The method performance parameters that are typically studied during method validation are selectivity, precision, bias, linearity working range, limit of detection, limit of quantitation, calibration and ruggedness. The validation process is illustrated in Figure 4.2. 

Precision estimates are key method performance parameters and are also required in order to carry out other aspects of method validation, such as bias and ruggedness studies. Precision is also a component of measurement uncertainty, as detailed in Chapter 6. The statistics that are applied refer to random variation and therefore it is important that the measurements are made to comply with this requirement, e.g. if change of precision with concentration is being investigated, the samples should be measured in a random order. 

Specific bias effects, which could be large and need to be understood and minimized, should have been studied during method development, e.g. the presence or absence of a potential interfering substance, or the effect of changing a deriva-tization reaction time. The task during validation is to study residual effects and this can be done as part of ruggedness studies, as discussed in Section 4.6.5. 

Once the appropriate dissolution conditions have been established, the method should be validated for linearity, accuracy, precision, specificity, and robustness/ruggedness. This section will discuss these parameters only in relation to issues unique to dissolution testing. All dissolution testing must be performed on a calibrated dissolution apparatus meeting the mechanical and system suitability standards specified in the appropriate compendia. 

Test methods used in the laboratory are generally derived from pharmacopoeias such as the US Pharmacopoeia, British Pharmacopoeia, or European Pharmacopoeia. For test methods that are not from recognized pharmacopoeias, validation of the analytical methods is required. The validation includes testing for accuracy, specificity, ruggedness, robustness, precision, detection limit, quantitation hmit, and range. A discussion of analytical methods vahda-tion is presented in Section 9.6.5. 

Method validation Basic method validation (short-term use, fit for purpose, little robustness), data will see expert eye prior to release Extended method validation, robustness and ruggedness tests important for unsupervised operation... 

This review describes the determination of robustness and ruggedness in analytical chemistry. The terms ruggedness and robustness as used in method validation are sometimes considered to be equivalent [1,2], In other publications a difference is made between the two terms [3]. In the following only the term ruggedness will be used. 

A ruggedness test is a part of method validation (Table 3.1) and can be considered as a part of the precision evaluation [2,4,5]. Ruggedness is related to repeatability and reproducibility. Some definitions for ruggedness come very close to those for reproducibility. Certain interpretation methods to identify the significant factors in a ruggedness test use criteria based on results for repeatability or reproducibility. These two items will be considered in Section 3.4.7. 

It should be noted that the same designs as those applied here are sometimes used in the optimization stage of the method [18,24-27], i.e. before the validation. In that case it makes sense to apply levels that are further apart than in ruggedness testing. 

Optimal values for the factors are selected from the tested levels for the factors (extremes or nominal) in function of a number of responses of the method (see also references [16,19]). When one changes the method conditions due to these results one has to be aware that a new method is defined. What is done here is in fact a simplistic way of optimizing a method. The optimization of a method however is a step that is expected to come much sooner in the method development than in the ruggedness testing. One also has to realize that when one defines a new method this requires a new full validation, including a ruggedness test. 

A number of software packages or expert systems for ruggedness testing has been developed. RES (commercialized under the name Shaiker ) is an expert system created by Van Leeuwen et al. [4,23] and has been validated and evaluated [42,43]. It uses fractional factorial and Plackett-Burman designs and allows to test the factors at two or three levels. The interpretation criteria used here are the predefined values (see Section 3.4.8). 

J.A. Van Leeuwen, L.M.C. Buydens, B.G.M. Vandeginste, G. Kateman, P.J. Schoenmakers, M. Mulholland, RES, an expert system for the set-up and interpretation of a ruggedness test in HPLC method validation. Part 1 The ruggedness test in HPLC method validatioa Chemometrics and Intelligent Laboratory systems, 10 (1991) 337-347. 

Ruggedness testing is one part of an overall method validation program and it is therefore important to begin this chapter by giving a brief outline of the levels of protocols for validation showing clearly where the ruggedness test is performed. 

---