Robustness testing design

The analyst should avoid creating impossible factor combinations. This occurs, for instance, when choosing both the batch number and the manufacturer of the capillary as factors in a robustness test by means of a two-level design. It is impossible to define two unique batch numbers that exist for both manufacturers. The way to examine both factors is by using nested designs. ... 

The two-factor interaction effects and the dummy factor effects in FF and PB designs, respectively, are often considered negligible in robustness testing. Since the estimates for those effects are then caused by method variability and thus by experimental error, they can be used in the statistical analysis of the effects. Requirement is that enough two-factor interaction or dummy factor effects (>3) can be estimated to allow a proper error estimate (see Section VII.B.2.(b)). 

The robustness tests described in references 20 and 69-92 used an experimental design approach, but often not all information is provided to repeat. Several analysts rely on software packages to set up and interpret a robustness test. Applied software packages are Modde... 

In reference 20, a typical robustness test is not performed, but a study on the influence of peak measurement parameters is reported on the outcome. The study is special in the sense that no physicochemical parameter in the experimental runs is changed, but only data measurement and treatment-related parameters. These parameters can largely affect the reported results, as shown earlier, and in that sense they do influence the robusmess of the method. The different parameters (see above) were first screened in a two-level D-optimal design (9 factors in 10 experiments). The most important were then examined in a face-centered CCD, and conclusions were drawn from the response surfaces plots. 

In this chapter, the possibilities to set up and treat the results of a robustness test were reviewed (Sections I-VIII). Robusmess usually is verified using two-level screening designs, such as FF and PB designs. These designs allow examining the effects of several mixmre-related, quantitative, and qualitative factors, on one or several responses, describing either quantitative and/or qualitative aspects of the analytical method. 

Finally, a review of robustness testing of CE methods was made and the tests were critically discussed (Section IX). Some researchers use the OVAT procedure, which seems less appropriate for a number of reasons. Some use response surface designs, which also seems less preferable in this context. Another remarkable observation from the case smdies is that only in a minority the quantitative aspect of the method is considered in the responses smdied, even though that was the initial idea of proposing the robustness tests. 

Hund, E., Vander Heyden, Y, Haustein, M., Massart, D. L., and Smeyers-Verbeke, J. (2000). Gomparlson of several criteria to decide on the significance of effects in a robustness test with an asymmetrical factorial design. Anal. Chim. Acta 404, 257—271. 

Dejaegher, B., Capron, X., Smeyers-Verbeke, J., and Vander Heyden, Y. (2006). Randomization tests to identify significant effects in experimental designs for robustness testing. Anal. Chim. Acta 564, 184-200. 

Ragonese, R., Macka, M., Hughes, J., and Petocz, P. (2002). The use of the Box-Behnken experimental design in the optimisation and robustness testing of a capillary electrophoresis method for the analysis of ethambutol hydrochloride in a pharmaceutical formulation. J. Pharm. Biomed. Anal. 27, 995-1007. 

A simple example, focusing on the analytical procedure, will illustrate the type of experimental design used to investigate three key factors in an HPLC method. Detailed discussion of experimental designs for robustness testing can be found in Morgan and Hendriks et Riley and Rosanske provide an... 

Test failures were attributed to a number of causes as illustrated in Figure 17.3. Operator error while executing the test case accounted for 1% of test failures. These tests were repeated once the error was understood. Incorrect setup also accounted for 1% of test failures. These tests too were repeated with the correct setup once the error was understood. Clarity problems with the test method and acceptance criteria accounted for 40% of test failures. Only the remaining 58% of tests did what they should have done, which is detect system errors. That is, 42% of test failure processing was avoidable if a more robust test process was adopted. Of the errors identified, 37% were classed as significant, and 63% as not significant. Resolution of these errors impacted specification and design documents. 

In a robustness test the following steps can be identified (a) identification of the variables to be tested, (b) definition of the different levels for the variables, (c) selection of the experimental design, (d) definition of the experimental protocol, (e) definition of the responses to be determined, (f) execution of the experiments and determination of the responses of the method, (g) calculation of effects, (h) statistical and/or graphical analysis of the effects, and (i) drawing chemically relevant conclusions from the statistical analysis and, if necessary, taking measures to improve the performance of the method. A general overview of robustness testing can be found in [35). 

The levels selected in a robustness test are different from those at which factors are evaluated in method optimization. For optimization purposes the variables are examined in a broad interval. In robustness testing the levels are much less distant. They represent the (somewhat exaggerated) variations in the values of the variables that could occur when a method is transferred. For instance, in optimization the levels for pH would be several units apart, while in robustness testing the difference could be 0.2 pH units. The levels can for instance be defined based on the uncertainty with which a factor level can be set and re.set 36 and usually they are situated around the method (nominal) conditions if the method specifies pH 4.0, the levels would be 3.9 and 4.1. The experimental designs used are in both situations the same and comprise fractional factorial and Plackett-Burman designs. 

In robustness testing sometimes replicated experiments at the nominal conditions are executed regularly distributed among the design experiments. This allows verifying if the response studied is not affected by time effects, and occasionally to correct for it. 

Robustness Testing. Robustness testing studies the capacity of the method to remain unaffected by small, but deliberate variation in the method parameters. By means of a limited set of experiments (often using an experimental design approach), the critical parameters that may affect the ruggedness of the method can be identified, understood, and improvements made if necessary. 

Generally, if robustness is designed into the method development process, the methods should transfer more readily. The successful performance of a test method may be sensitive to the setting of some operational parameters. In robustness testing, a variety of parameters are evaluated to determine the extent to which they can be varied without affecting the performance of the method. In an HPLC experiment, the following representative parameters (factors) may be evaluated ... 

Peters PA, Paino TC. Robustness testing of an HPLC method using experimental design. Pharm Technol, Analytical Validation 1999. 

EXPERIMENTAL DESIGN IN METHOD OPTIMIZATION AND ROBUSTNESS TESTING... 

To determine the robustness of a method, several approaches exist. Basically, the situation for robustness testing is similar to that for screening during optimization, except for the range within which the factors are examined. The influence of small but deliberate changes in parameters on the response (s) is evaluated using either an OVAT or an experimental design approach (12). 

In fact, the definition of ruggedness by Youden and Steiner equals the USP and ICH definitions of robustness. It is also the most widely applied definition. Further in this chapter, only consequences related to this definition are considered, and only the term robustness is used. In such type of robustness testing, usually screening designs are applied. 

---