Evaluating Robustness

Having calculated the result of the analysis, it can be informative to assess its robustness. In any combined analysis, some of the studies included will be larger than others, and sometimes a small percentage of included studies can be considerably larger than the majority of others. The nature of the calculations performed here mean that the larger trials tend to influence the result more, since they tend to have greater precision. 

It can therefore be helpful to assess the robustness of the overall conclusion by performing the analysis without the data from the largest study or studies to see if the results remain qualitatively the same. If they do, then the result of the primary overall analysis is deemed robust. If they do not, confidence in the overall result can be undermined. Moreover, if the results are considerably different, it simply may not be appropriate to present the combined result alone and make statements based on it. 

6 Meta-analysis, Group Sequential Study Designs, and other topics 

---

All aspects in the analytical process are equally important, and each step should be isolated in method development experiments and/or validation to ensure acceptable quality of results. A good way to evaluate robustness of a method is to alter parameters (e.g., solvent volumes, temperature, pH, sources of reagents) of each step to determine... 

HPLC methods can usually be transferred without many modifications, since most commercially available HPLC instruments behave similarly. This is certainly true when the columns applied have a similar selectivity. One adaptation, sometimes needed, concerns the gradient profiles, because of different instrumental or pump dead-volumes. However, larger differences exist between CE instruments, e.g., in hydrodynamic injection procedures, in minimum capillary lengths, in capillary distances to the detector, in cooling mechanisms, and in the injected sample volumes. This makes CE method transfers more difficult. Since robustness tests are performed to avoid transfer problems, these tests seem even more important for CE method validation, than for HPLC method validation. However, in the literature, a robustness test only rarely is included in the validation process of a CE method, and usually only linearity, precision, accuracy, specificity, range, and/or limits of detection and quantification are evaluated. Robustness tests are described in references 20 and 59-92. Given the instrumental transfer problems for CE methods, a robustness test guaranteeing to some extent a successful transfer should include besides the instrument on which the method was developed at least one alternative instrument. 

To determine Sb in marine sediments by ETAAS, a direct method was developed based on quantitating the analyte in the liquid phase of the slurries (prepared directly in autosampler cups). The variables influencing the extraction of Sb into the liquid phase and the experimental setup were set after a literature search and a subsequent multivariate optimisation procedure. After the optimisation, a study was carried out to assess robustness. Six variables were considered at three levels each (see Table 2.13). In addition, two noise factors were set after observing that two ions, which are currently present into marine sediments, might interfere in the quantitations. In order to evaluate robustness, a certified reference material was used throughout, BCR-CRM 277 Estuarine Sediment (guide value for Sb 3.5 0.4pgg ). Table 2.13 depicts the experimental setup. 

A combination of reversed-phase chromatography and ETAAS was proposed. A central composite design studied the chromatographic conditions. Response curves were deployed for each analyte in order to select the best experimental conditions and evaluate robustness... 

Note that the optimal production strategy does not depend on the recoverable volirmes. Hence, in order to evaluate robustness, we can focus on the scale parameters. In fact in the first two mrmerical examples the recoverable volumes will simply be assirmed to be known, an assumption that under no circirmstances is required. Still in order to simplify the discussion and focus on the important results, it is convenient to do this. As a result the only unknown parameters in the model are the scale parameters, i.e., D, ..., D . 

For a recent critical evaluation of situations where current DFT approaches experience difficulties, see Davidson E R 1998 How robust is present-day DFT ... 

On a different note, after some 50 years of intensive research on high-pressure shock compression, there are still many outstanding problems that cannot be solved. For example, it is not possible to predict ab initio the time scales of the shock-transition process or the thermophysical and mechanical properties of condensed media under shock compression. For the most part, these properties must presently be evaluated experimentally for incorporation into semiempirical theories. To realize the potential of truly predictive capabilities, it will be necessary to develop first-principles theories that have robust predictive capability. This will require critical examination of the fundamental postulates and assumptions used to interpret shock-compression processes. For example, it is usually assumed that a steady state is achieved immediately after the shock-transition process. However, due to the fact that... 

Using computer-aided numerical calculations, one can readily simulate and identify critical parameters for process validation. Thus, one can evaluate the robustness of the process during its design. To ensure performance, optimization of the process and evaluation of critical parameters can be determined before actual operating conditions. 

Testing the robustness of a (best) model (and second-best contenders) by evaluating sets of statistically similar data created with program SIMILAR if the derived decisions remain unaffected by measurement noise, the model is adequate. 

There is little or no capability to evaluate risk. The robustness of the plan to key uncertainties is rarely assessed. Important risks in R D develop-... 

Expert opinion is a source, frequently elicited by survey, that is used to obtain information where no or few data are available. For example, in our experience with a multicountry evaluation of health care resource utilization in atrial fibrillation, very few country-specific published data were available on this subject. Thus the decision-analytic model was supplemented with data from a physician expert panel survey to determine initial management approach (rate control vs. cardioversion) first-, second-, and third-line agents doses and durations of therapy type and frequency of studies that would be performed to initiate and monitor therapy type and frequency of adverse events, by body system and the resources used to manage them place of treatment and adverse consequences of lack of atrial fibrillation control and cost of these consequences, for example, stroke, congestive heart failure. This method may also be used in testing the robustness of the analysis [30]. 

Care for people with dementia is demanding of resources, while the outcomes of care are uncertain. To date, the economic analyses of care strategies have been limited to new drug therapies for people with Alzheimer s disease. Full economic evaluations to compare both the costs and consequences have only been conducted for one of these dmgs, donepezil. However, problems with the design and data used in these studies mean that they do not provide robust evidence to determine appropriate management strategies for dementia. 

---