Data Interpretation Errors

Users expect certified values to be correct - with a probability of 95 % - within the stated uncertainty intervals. They assume, perhaps naively, that all statements of uncertainty are the same. In practice the stated uncertainties may have quite different meanings because they have been based on quite diflferent principles. This issue is discussed in more detail below. But for most users neither the differences nor the consequences of the differences are always evident, or understood. 

ISO Guide 33 (1989) recommends that CRMs are used on a regular basis to ensure reliable measurements . In reality, the expression to ensure reliable measurements can have a wide range of interpretations, including  

Although the user will require differing types of information from the producer to properly use the CRM for each applications, there is a tendency to provide only a certified value and an uncertainty value, which is generally said to be a 95 % confidence interval, or something similar. The relevance of this was made clear by Jor-hem (1998), but it is not always evident from the supplied documentation. 

One of the most common complaints from the inexperienced user is that the result obtained in the routine laboratory does not fall in the confidence interval. Pau-wels (1999) makes considerable reference to this problem, which he calls the Jor-hem Paradox . Even though Pauwels goes on to explain this paradox, in doing so he highlights the problem when he states two results (the certified value and the subsequent laboratory determination) which both claim to contain the most probable mean value of the material with a probability of 95 % do, effectively, not overlap . 

How can this situation arise It is because most certification bodies are not in a position to consider other uncertainty components than those associated with the certification process. 

---

Data interpretation error—incorrect analytical data interpretation producing false positive or false negative results... 

Representativeness Sampling design error Field procedure error Data interpretation error Sample management error Data management error... 

The primary source of data interpretation error in elemental analysis is laboratory contamination affecting the method blank and the samples. Method blank is a volume of analyte-free water prepared and analyzed in the same manner as the samples. Method blank is also called analytical blank or preparation blank. 

Contaminated samples and method blanks and instrument memory effects (carryover) are a major source of false positive results. To determine whether data interpretation errors may have produced false positive results, the chemist examines the instrument and method blank data and answers the following questions ... 

A complete knowledge of the data quality that arises only from Level 4 validation enables the data user to make project decisions with the highest level of confidence in the data quality. That is why Level 4 validation is usually conducted for the data collected to support decisions related to human health. Level 4 validation allows the reconstruction of the entire laboratory data acquisition process. It exposes errors that cannot be detected during Level 3 validation, the most critical of which are data interpretation errors and data management errors, such as incorrect computer algorithms. 

Methods for measurement of kp have been reviewed by Stickler,340 41 van Herk Vl and more recently by Beuermann and Buback.343 A largely non critical summary of values of kp and k, obtained by various methods appears in the Polymer Handbook.344 Literature values of kp for a given monomer may span two or more orders of magnitude. The data and methods of measurement have heen critically assessed by IUPAC working parties"45"01 and reliable values for most common monomers are now available. 43 The wide variation in values of kp (and k,) obtained from various studies docs not reflect experimental error but differences in data interpretation and the dependence of kinetic parameters on chain length and polymerization conditions. 

Used either as prelaboratory preparation for related laboratory activities or to expose students to additional laboratory activities not available in their program, these modules motivate students to learn by proposing real-life problems in a virtual environment. Students make decisions on experimental design, observe reactions, record data, interpret these data, perform calculations, and draw conclusions from their results. Following a summary of the module, students test their understanding by applying what they have learned to new situations or by analyzing the effect of experimental errors. 

However, the amount of error in the data is not generally the limiting factor in data interpretation. Rather, the locations at which the data are taken most severely hinder progress toward a mechanistic model. Reference to Fig. 1 indicates that the decision between the dual- and single-site models would be quite difficult, even with very little error of measurement, if data are taken only in the 2- to 10-atm range. However, quite substantial error can be tolerated if the data lie above 15 atm total pressure (assuming data can be taken here). Techniques are presented that will seek out such critical experiments to be run (Section VII). 

Second, there is no unique scheme of data interpretation. The process of inference always remains arbitrary to some extent. In fact, all the existing DDT data combined still allow for an infinite number of models that could reproduce these data, even if we were to disregard the measurement uncertainties and take the data as absolute numbers. Although this may sound strange, it is less so if we think in terms of degrees of freedom. Let us assume that there are one million measurements of DDT concentration in the environment. Then a model which contains one million adjustable parameters can, in principle, exactly (that is, without residual error) reproduce these data. If we included models with more adjustable parameters than observa-... 

Non-sampling errors can be categorized into laboratory error and data management error, with laboratory error further subdivided into measurement, data interpretation, sample management, laboratory procedure and methodology errors. 

We can easily quantify measurement error due to existence of a well-developed approach to analytical methods and laboratory QC protocols. Statistically expressed accuracy and precision of an analytical method are the primary indicators of measurement error. However, no matter how accurate and precise the analysis may be, qualitative factors, such as errors in data interpretation, sample management, and analytical methodology, will increase the overall analytical error or even render results unusable. These qualitative laboratory errors that are usually made due to negligence or lack of information may arise from any of the following actions ... 

In the course of sample tracking, data evaluation, and interpretation, field sample IDs may be entered into several different field forms, spreadsheets, and data bases, and appear on maps and figures as identifiers for the sampling points. Because the field records and computer data entry during sample receiving at the laboratory are done for the most part manually, errors in sample ID recording are common. To reduce data management errors, sample numbers must be simple, short, and consecutive. 

Much more difficult to detect are data interpretation and judgment errors, such as the unrecognized false positive and false negative results and the incorrect interpretation of mass spectra and chromatographic patterns. The detection and correction of these errors is made possible through internal review by experienced analysts. 

Tier 2 Review may be also documented in a checklist, which becomes part of the project file. Any errors in calculations and data interpretation discovered during Tier 2 Review may be easily corrected at this point. 

Tier 2 and Tier 3 Reviews require a great deal of time and experience in analysis and data interpretation and should be conducted by laboratory staff with appropriate qualifications. Due to time and budget constraints at commercial laboratories, internal review is sometimes looked upon as an unnecessary time- and resourceconsuming process. At such laboratories, perpetually busy analysts and supervisors pay cursory attention to internal review, a practice that is evident in their error-ridden laboratory reports. 

The major advantage of microscopic methods is their direct measurement of particle size. In many of the alternative methods, at least one automated data-interpretation or calculation step is inserted between the instrumental analysis and establishing the estimate of particle size. This reduces the subjectivity of the measurement while increasing the likelihood of interpretive errors. 

The requirements for automatic interpretation of SOPs mentioned already leads us to another approach that is a general use for any steps in a laboratory workflow. If we look at the operator-entering data, we have to keep several critical sources of errors in mind typing errors, data type errors, formats errors, and data limit errors. One valuable solution based on expert system technology is a system that verities the data entered by the operator on the basis of rules. 

In the interpretation of the numerical results that can be extracted from Mdssbauer spectroscopic data, it is necessary to recognize three sources of errors that can affect the accuracy of the data. These three contributions to the experimental error, which may not always be distinguishable from each other, can be identified as (a) statistical, (b) systematic, and (c) model-dependent errors. The statistical error, which arises from the fact that a finite number of observations are made in order to evaluate a given parameter, is the most readily estimated from the conditions of the experiment, provided that a Gaussian error distribution is assumed. Systematic errors are those that arise from factors influencing the absolute value of an experimental parameter but not necessarily the internal consistency of the data. Hence, such errors are the most difficult to diagnose and their evaluation commonly involves measurements by entirely independent experimental procedures. Finally, the model errors arise from the application of a theoretical model that may have only limited applicability in the interpretation of the experimental data. The errors introduced in this manner can often be estimated by a careful analysis of the fundamental assumptions incorporated in the theoretical treatment. 

The authors have completed a new approach to the kinetics of thermal grooving and have concluded that Mullins theory is correct, but errors have been made in data interpretation that have resulted in incorrect conclusions and these errors have been self-perpetuating. The conventional practice of... 

Another main source of uncertainty is the uncertainty in input data. Data uncertainties may be caused by, for example, measurement errors, interpretation errors, or the uncertainty involved in extrapolation when the parameter varies in space or in time. Conceptual uncertainty, data uncertainty and spatial variability may all be related (Figure 2). Our second Part 2 paper (Hudson and Andersson, 2003) will elaborate further on data uncertainty and spatial variability. As with conceptual uncertainty, the judgement whether the data uncertainty needs to be reduced, must be weighted against its impact on performance. 

FIGURE 4.24 (a) Calculated profiles for HNO, NO2, and NO (solid lines) and N2O5 (dashed line) for the ATMOS simulation at 47° S, sunrise, assuming gas-phase chemistry only (McElroy et al., 1992). ATMOS data are indicated by the circles, which have been connected by dotted lines for convenience of interpretation. Error bars represent the lo- estimate of the measurement uncertainty, (b) Same as for part (a) except the simulation includes the heterogeneous hydrolysis of N2O, proceeding with an efficiency of y = 0.06. Reprinted from McElroy et al. (1992) with kind permission from Elsevier Science Ltd., The Boulevard, Langford Lane, Kidlington 0X5 1GB. UK. 

---