Ruggedness testing

Ruggedness testing evaluates how small changes in the method conditions affect the measurement result, e.g. small changes in temperature, pH, flow rate, composition of mobile phase, etc. The aim is to identify and, if necessary, better control method conditions that might otherwise lead to variation in measurement results, when measurements are carried out at different times or in different laboratories. It can also be used to improve precision and bias. 

Experiment number Method parameter Observed result 

To find if changing factor A to a has an effect, Aa is calculated as shown in the following equation  

Experiments 1-4 had the factor at the nominal level, A , while experiments 5-8 had the factor at the alternative level, a . Inspection of Table 4.8 shows that with this combination the effect of the other factors cancels out. The seven factors can be dealt with in a similar way by grouping the nominal level for that factor and subtracting the alternative level. Changing factor B to b is examined by calculating Ab, as follows  

The next step is to arrange the seven differences, Aa to AG, in numerical order (ignoring the sign). To calculate if any of the differences are statistically significant, a statistical test (t-test) is applied. Equation (4.17) is used to compare the difference A with the expected precision of the method, s. The value of t used corresponds to the value obtained from statistical tables for the degrees of freedom appropriate for the estimation of s and the level of confidence used. For example, if the method standard deviation was obtained from ten results, i.e. nine degrees of freedom, t(95%) = 2.262. 

The process of evaluating a method to determine those factors for which a small change in value has a significant effect on the method s results. 

In many cases an optimized method may produce excellent results in the laboratory developing the method, but poor results in other laboratories. This is not surprising since a method is often optimized by a single analyst under an ideal set of conditions, in which the sources of reagents, equipment, and instrumentation remain the same for each trial. The procedure might also be influenced by environmental factors, such as the temperature or relative humidity in the laboratory, whose levels are not specified in the procedure and which may differ between laboratories. Finally, when optimizing a method the analyst usually takes particular care to perform the analysis in exactly the same way during every trial. 

An important step in developing a standard method is to determine which factors have a pronounced effect on the quality of the analytical method s result. The procedure can then be written to specify the degree to which these factors must be controlled. A procedure that, when carefully followed, produces high-quality results in different laboratories is considered rugged. The method by which the critical factors are discovered is called ruggedness testing.  

Ruggedness testing is often performed by the laboratory developing the standard method. Potential factors are identified and their effects evaluated by performing the analysis while the factors are held at two levels. Normally one level for each factor is that given in the procedure, and the other is a level likely to be encountered when the procedure is used by other laboratories. 

The experimental design for ruggedness testing is balanced in that each factor level is paired an equal number of times with the upper case and lower case levels 

---

Experimental Design for a Ruggedness Test Involving Seven Factors... 

Vander Heyden, Y., Khots, M. S., and Massart, D. L., Three-Level Screening Designs for the Optimization or the Ruggedness Testing of Analytical Procedures, Analytica Chimica Acta 276, 1993, 189-195. 

Vander Heyden, Y., Hartmann, C., Massart, D.L., Hollands, A.M.J., Nuyten, P. and Schoenmakers, R, Ruggedness testing of a size-exclusion chromatographic assay for low molecular-mass polymers, /. Chromatogr. A, 756,89,1996. 

Ruggedness can be determined by an interlaboratory study with a sufficient in > 8) participating laboratories following one and the same procedure as performed by different analysts, and using operational and environmental conditions that may differ but are still within the specified parameters of the assay [1, 41]. Detailed guidance for robustness and ruggedness testing is available [42, 43],... 

Ruggedness Test of the extent to which the results of an analytical procedure are affected by slight changes in the procedure. 

Joergensen K, Jacobsen L. Factorial design used for ruggedness testing of flow through cell dissolution method by means of Weibull transformed drug release profiles. Inti J Pharm 1992 88 23-29. 

ASTM El 169-89 Standard Guide for Conducting Ruggedness Tests (Plackett-Burman Design), American Society for Testing and Materials (ASTM), 100 Barr Harbor Drive, West Conshohocken, PA 19428-2959. 

To our knowledge, in the first approach described, Youden and Steiner introduced the term ruggedness test for a setup in which by means of an experimental design the influences of minor but deliberate and controlled changes in the method parameters or factors are evaluated in order to detect non-robust factors, i.e., with a large influence on the response. Controlling the latter factors avoids problems in the following interlaboratory study. 

Mulholland, M., and Waterhouse, J. (1987). Development and evaluation of an automated procedure for the ruggedness testing of chromatographic conditions in high-performance liquid chromatography./. Chromatogr. 395, 539-551. 

Mulholland, M. (1988). Ruggedness testing in analytical chemistry. Trends Anal. Chem. 7, 383-389. 

Mulholland, M. (1996). Ruggedness tests for analytical chemistry. In Robustness of Analytical Chemical Methods and Pharmaceutical Technological Products (M. W. B. Hendriks, J. H. de Boer, and A. K. Smilde, Eds), pp. 191-232, Elsevier, Amsterdam. 

Vander Heyden, Y., Luypaert, K., Hartmann, C., Massart, D. L., Hoogmartens, J., and De Beer, J. (1995). Ruggedness tests on the high-performance liquid chromatography assay of the United States Pharmacopeia XXII for tetracycline hydrochloride. A comparison of experimental designs and statistical interpretations. Anal. Chim. Acta 312, 245-262. 

Method validation Basic method validation (short-term use, fit for purpose, little robustness), data will see expert eye prior to release Extended method validation, robustness and ruggedness tests important for unsupervised operation... 

Reproducibility Standard Deviation Reproducibility Limit R Ruggedness Test Selectivity... 

A review of ruggedness testing methods is presented in Chapter 3 and in Chapter 5 examples are given. In these chapters procedures are described that test the robustness or ruggedness of existing methods. Hence, incorporating robustness explicitly in analytical techniques (see Section 1.1) is not discussed. 

Methods to deal with these problems are outlined in Chapter 5 and examples are given how to tackle these problems. Examples of ruggedness testing in HPLC are also given in Chapter 3. 

The experiments conducted to perform ruggedness tests of measurement procedures can also be viewed as experiments to investigate robust design see, for example, Wemimont [2], and Youden [3,4]. The objective of ruggedness tests is to determine a robust measurement procedure that is a procedure that will give a consistent (and correct) result under a range of measurement conditions. 

---