Statistical evaluation systematic errors

The difficult analysis of the sources of error in free energy difference evaluations by computer simulation and the effect of particular implementations of free energy difference techniques has been the subject of a number of recent studies. The error inherent in any computer simulation can formally be categorized by statistical and systematic errors. In practice, these errors are often difficult to separate. 

The precision stated in Table 10 is given by the standard deviations obtained from a statistical analysis of the experimental data of one run and of a number of runs. These parameters give an indication of the internal consistency of the data of one run of measurements and of the reproducibility between runs. The systematic error is far more difficult to discern and to evaluate, which causes an uncertainty in the resulting values. Such an estimate of systematic errors or uncertainties can be obtained if the measuring method can also be applied under circumstances where a more exact or a true value of the property to be determined is known from other sources. 

Analytical quality control (QC) efforts usually are at level I or II. Statistical evaluation of multivariate laboratory data is often complicated because the number of dependent variables is greater than the number of samples. In evaluating quality control, the analyst seeks to establish that replicate analyses made on reference material of known composition do not contain excessive systematic or random errors of measurement. In addition, when such problems are detected, it is helpful if remedial measures can be Inferred from the QC data. 

The values of a and b were determined empirically by measuring the frequencies of six reference masses and fitting mass-frequency data to the form of the equation by means of a least-squares procedure. By using empirical values of a and b and the measured ion frequencies, the masses of the six ions were calculated. A statistical evaluation between the known masses and the experimentally determined masses was then made to determine the accuracy of the calibration law. Systematic errors were observed as is described in (40). 

Statistics should follow the technical scrutiny, not the other way round. A statistical analysis of data of an interlaboratory study cannot explain deviating results nor can alone give information on the accuracy of the results. Statistics only treat a population of data and provide information on the statistical characteristics of this population. The results of the statistical treatment may give rise to discussions on particular data not belonging to the rest of the population, but outlying data can sometimes be closer to the true value than the bulk of the population (Griepink et al., 1993). If no systematic errors affect the population of data, various statistical tests may be applied to the results, which can be treated either as individual data or as means of laboratory means. When different methods are applied, the statistical treatment is usually based on the mean values of replicate determinations. Examples of statistical tests used for certification purposes are described elsewhere (Horwitz, 1991). Together with the technical evaluation of the results, the statistical evaluation forms the basis for the conclusions to be drawn and the possible actions to be taken. 

This part of the chapter is concerned with the evaluation of nncertainties in data and in calculated results. The concepts of random errors/precision and systematic errors/accuracy are discussed. Statistical theory for assessing random errors in finite data sets is summarized. Perhaps the most important topic is the propagation of errors, which shows how the error in an overall calculated result can be obtained from known or estimated errors in the input data. Examples are given throughout the text, the headings of key sections are marked by an asterisk, and a convenient summary is given at the end. 

The evaluation study will determine the attributes (bias, precision, specificity, limits of detection) of the immunoassay. Bias testing (systematic error) will be conducted by measuring recoveries of the analyte added to matrices of interest. Replicate analysis will be performed on blind replicates or split levels (e.g., Youden pairs). A minimum number of replicates will be performed to provide statistically meaningful results. The number of replicates will be determined by the intended purpose of the immunoassay as well as the documented method performance of the comparative method. 

The technical evaluation may also lead to the comparison of the results obtained from different methods. It will allow participants to extract information by comparing and possibly discussing their performance and method with other participants applying similar procedures, i.e. it may allow to discover biases in methods. If several enriched materials have been prepared and analysed the organiser may produce Youden plots where trends and systematic errors can appear [10-12]. Such more elaborated data presentations have to be issued with sufficient explanations to avoid misunderstanding and wrong conclusions. More advanced data treatment require the application of suitable robust statistics which have to be carefully chosen to arrive at sound scientific conclusions. Their meaning should always be explained and documented. 

In the interpretation of the numerical results that can be extracted from Mdssbauer spectroscopic data, it is necessary to recognize three sources of errors that can affect the accuracy of the data. These three contributions to the experimental error, which may not always be distinguishable from each other, can be identified as (a) statistical, (b) systematic, and (c) model-dependent errors. The statistical error, which arises from the fact that a finite number of observations are made in order to evaluate a given parameter, is the most readily estimated from the conditions of the experiment, provided that a Gaussian error distribution is assumed. Systematic errors are those that arise from factors influencing the absolute value of an experimental parameter but not necessarily the internal consistency of the data. Hence, such errors are the most difficult to diagnose and their evaluation commonly involves measurements by entirely independent experimental procedures. Finally, the model errors arise from the application of a theoretical model that may have only limited applicability in the interpretation of the experimental data. The errors introduced in this manner can often be estimated by a careful analysis of the fundamental assumptions incorporated in the theoretical treatment. 

The details of the assessment of stability data are under intense discussion within the scientific community. A majority of laboratories evaluate data with acceptance criteria relative to the nominal concentration of the spiked sample. The rationale for this is that it is not feasible to introduce more stringent criteria for stability evaluations than that of the assay acceptance criterion. Another common approach is to compare data against a baseline concentration (or day zero concentration) of a bulk preparation of stability samples established by repeated analysis, either during the accuracy and precision evaluations, or by other means. This evaluation then eliminates any systematic errors that may have occurred in the preparation of the stability samples. A more statistically acceptable method of stability data evaluations would be to use confidence intervals or perform trend analysis on the data [24]. In this case, when the observed concentration or response of the stability sample is beyond the lower confidence interval (as set a priori), the data indicate a lack of analyte stability under the conditions evaluated. 

Thus the present section does not refer extensively to the statistical considerations summarized in Sections 8.2 and 8.3 rather we are dealing here with the realities of different situations in which the analyst can find himself/herself when confronted with situations that can be appreciably less than ideal, e.g., how to caUhrate the measuring instrument when no analytical standard or internal standard or blank matrix is available. It is understood that the statistical considerations of Sections 8.1-8.3 would then be apphed to the data thus obtained. Also, note that none of the statistical evaluations of random errors, discussed in this chapter, would reveal any systematic errors. These aspects are addressed in this section and build upon the preceding discussion of analytical standards (Section 2.2), preparation of caUbration solutions (Section 2.5) and the brief simplified introduction (Section 2.6) to cahbration and measurement... 

The method evaluation ensures that the experimental standard deviation (a) is a valid measure of the combined uncertainty and that the basic assumptions for the statistical tests for possible systematic errors are fulfilled. The systematic errors are zero point errors if a is significantly deviating from zero and proportional errors, if P is significantly deviating from 1 [6,19,26,28]. 

Much of the remainder of this book will deal with the evaluation of random errors, which can be studied by a wide range of statistical methods. In many cases we shall assume for convenience that systematic errors are absent (though methods which test for the occurrence of systematic errors will be described). But first we must discuss systematic errors in more detail - how they arise, and how they may be countered. The titration example above shows that systematic errors cause the mean value of a set of replicate measurements to deviate from the true value. It follows that (a) in contrast to random errors, systematic errors cannot be revealed merely by making repeated measurements, and that (b) unless the true result of the analysis is known in advance - an unlikely situation - very large systematic errors might occur, but go entirely undetected unless suitable precautions are taken. In other words, it is all too easy to overlook substantial sources of systematic error. A few examples will clarify both the possible problems and their solutions. 

One of the most important properties of an analytical method is that it should be free from systematic error. This means that the value which it gives for the amount of the analyte should be the true value. This property of an analytical method may be tested by applying the method to a standard test portion containing a known amount of analyte (Chapter 1). However, as we saw in the last chapter, even if there were no systematic error, random errors make it most unlikely that the measured amount would exactly equal the standard amount. In order to decide whether the difference between the measured and standard amounts can be accounted for by random error, a statistical test known as a significance test can be employed. As its name implies, this approach tests whether the difference between the two results is significant, or whether it can be accounted for merely by random variations. Significance tests are widely used in the evaluation of experimental results. This chapter considers several tests which are particularly useful to analytical chemists. 

Judging from the available statistical evaluations, MNDO can often provide results of useful accuracy. It should be kept in mind, however, that errors in semiempirical methods are generally less systematic than in ab initio or density functional methods and therefore harder to anticipate. It is thus important to know about the qualitative deficiencies of a given semiempirical method. In the case of MNDO, these include ... 

---