Ruggedness statistical methods

Ruggedness of an analytical method is the insensibility of the method for variations in the circumstance and the method variables during execution. It is important to note that statistically significant deviations are not always relevant. The purpose for which the measurement is made is a more important criterion for deciding on the ruggedness and the statistical method employed is merely a tool. 

To demonstrate the accuracy, two dust and two soil reference materials were analyzed with the described method. The mean value of the correlation coefficients between the certified and the analyzed amounts of the 16 elements in the samples is r = 0.94. By application of factor analysis (see Section 5.4) the square root of the mean value of the communahties of these elements was computed to be approximately 0.84. As frequently happens in the analytical chemistry of dusts several types of distribution occur [KOM-MISSION FUR UMWELTSCHUTZ, 1985] these can change considerably in proportion to the observed sample size. In the example described the major components are distributed normally and most of the trace components are distributed log-normally. The relative ruggedness of multivariate statistical methods against deviations from the normal distribution is known [WEBER, 1986 AHRENS and LAUTER, 1981] and will be tested using this example by application of factor analysis. 

Principles and Characteristics Whereas parameters most relevant to method development are considered to be accuracy, system precision, linearity, range, LOD, LOQ, sensitivity and robustness, method validation parameters are mainly bias, specificity, recovery (and stability of the analyte), repeatability, intermediate precision, reproducibility and ruggedness. However, method development and validation are highly related. Also, validation characteristics are not independent they influence each other. Acceptance criteria for validation parameters should be based on the specification limits of the test procedure. Quantitation and detection limits need a statement of the precision at their concentration levels. Procedures used for validation of qualitative methods are generally less involved than those for quantitative analytical methods. According to Riley [82], who has discussed the various parameters for validation of quantitative analytical methods, the primary statistical parameters that validate an analytical method are accuracy and precision. 

Tranffier of analytical method methodology. Continuation of method validation by (costly and lengthy) interlaboratory collaborative studies (ruggedness) statistical comparison of the validation results (e.g. for HPLC methods cfr. ref. [70]). 

Precision estimates are key method performance parameters and are also required in order to carry out other aspects of method validation, such as bias and ruggedness studies. Precision is also a component of measurement uncertainty, as detailed in Chapter 6. The statistics that are applied refer to random variation and therefore it is important that the measurements are made to comply with this requirement, e.g. if change of precision with concentration is being investigated, the samples should be measured in a random order. 

The relative standard deviation is not used in a statistical test [4]. It is only used to check if the repeatability of the method is good enough. If the relative standard deviation is larger than 1%, repeatability is considered too high for a HPLC method. In that case the reason for the large relative standard deviation has to be diagnosed preliminary to the interpretation of the main effects from the ruggedness test. 

Merck also proposed recently an expert system called Ruggedness Method Manager for ruggedness tests of chromatographic assay methods. The system uses fractional factorial designs. Besides the factors to be examined, interactions that possibly also could be relevant have to be defined by the user. The system then calculates a design in which the main effects are not confounded with one of the specified interactions. The interpretation criterion to identify statistically significant effects is not known to the authors of this chapter. 

The standard error reflects the statistical relevance of all the main effects (i.e. a main effect with a smaller value than the standard error is not statistically relevant). In this instance a main effect must be larger than -0.7 to be considered a real effect and not just a reflection of the overall precision of the method. The results for each factor are given in Table 5.18. The largest effect is around 3% and is due to the change in the acid type used to control the pH of the mobile phase. All observed effects were unlikely to cause a lack of method ruggedness as no effect caused a critical reduction in the plate count. 

As is indicated in Figure 8, this process is likely to be an iterative one. However, it is essential that good written records are kept during this phase so that, in the event of problems at subsequent levels, investigations may be more readily carried out. Alas, far too often the excuse of analytical creativity is cited for lack of such records. The most important outcome from this initial evaluation should be an assessment of robustness (or ruggedness) of the developed procedure. The AO AC Guide,Use of statistics to develop and evaluate analytical methods is an excellent source for a discussion of statistical procedures for both inter- and intra-laboratory studies. 

Collaborative testing. Once the method has been validated in one laboratory, it may be subjected to collaborative testing. Here, identical test samples and operating procedures are distributed to several laboratories. The results are analyzed statistically to determine bias and interlaboratory variability. This step determines the ruggedness of the method. 

The ruggedization of the analytical procedure was performed by applying statistical screening techniques to minimize the effort required and, therefore, reduce the time and the cost substantially. The statistical approaches used in this study were those first introduced by Plackett-Burman (3.) and Youden-Steiner (4). Both techniques reduce the required effort since they use balanced incomplete block design experiments which can clearly indicate the non-affecting parameters from those that may have an effect. In this study the important variables of the analytical method were identified by using the Plackett-Burman technique. 

The results of the ruggedness testing and bias evaluation should be published in full. This report should identify the critical parameters, including the materials within the scope of the method, and detail the effect of variations in these on the final result. It should also include the values and relevant uncertainties associated with bias estimations, including both statistical and reference material uncertainties. Since it is a requirement of the validation procedure that this information should be available before carrying out the collaborative study, publishing it would add little to the cost of validating the method and would provide valuable information for future users of the method. 

Evaluation of ruggedness and statistical control of the method (charts)... 

Carlson R (1992) Design and optimization of organic synthesis. Elsevier, Amsterdam Box GEP (1952) Statistical design in the study of analytical methods. Analyst 77 879-889 Cochran WG, Cox GM (1957) Experimental designs. WUey, New York Small TS, Fell AF, Coleman MW, Berridge JC (1995) Central composite design forthe rapid optimisation of ruggedness and chiral separation of amlodipine in capillary electrophoresis. Chirahty 7 226-234... 

---