Distribution of data

Example 6.3 Distribution of Data in Quantum Chemistry Applications 

Some data structures commonly encoimtered in quantum chemistry applications are listed in Table 6.1. For a problem size of n (where n is the number of basis functions or the number of atoms), there could be data structures whose storage requirements grow as 0(n), O(n ), O(n ), and O(n ). If arrays of 0(n ) are stored, they must be distributed or serious 

Some data structures commonly encoimtered in quantum chemistry methods. The problem size (that is, the number of atoms or the number of basis functions) is denoted n 

0(n) basis set, molecular coordinates, molecular orbital energies 

O(n ) Fock matrix, density matrix, single-substitution amplitudes O(n ) subsets of triple-substitution amplitudes 0(n ) two-electron integrals, double-substitution amplitudes 

---

The degree of data spread around the mean value may be quantified using the concept of standard deviation. O. If the distribution of data points for a certain parameter has a Gaussian or normal distribution, the probabiUty of normally distributed data that is within Fa of the mean value becomes 0.6826 or 68.26%. There is a 68.26% probabiUty of getting a certain parameter within X F a, where X is the mean value. In other words, the standard deviation, O, represents a distance from the mean value, in both positive and negative directions, so that the number of data points between X — a and X -H <7 is 68.26% of the total data points. Detailed descriptions on the statistical analysis using the Gaussian distribution can be found in standard statistics reference books (11). 

No a priori assumptions about the distribution of data or class probability are required. 

Secondly, knowledge of the estimation variance E [P(2c)-P (2c)] falls short of providing the confidence Interval attached to the estimate p (3c). Assuming a normal distribution of error In the presence of an Initially heavily skewed distribution of data with strong spatial correlation Is not a viable answer. In the absence of a distribution of error, the estimation or "krlglng variance o (3c) provides but a relative assessment of error the error at location x Is likely to be greater than that at location 2 " if o (2c)>o (2c ). Iso-varlance maps such as that of Figure 1 tend to only mimic data-posltlon maps with bull s-eyes around data locations. 

The theoretical work that exploited the advantages of the multidimensional separation format appears to have been developed much later than the original experimental work. One of the earliest studies was conducted by Connors (1974), who assumed that the distribution of spots on a two-dimensional thin-layer chromatography (2DTLC) plate could be modeled using a Poisson distribution of data on each retention axis. He then constructed equations that related the number of chromatographic systems needed to resolve a specific number of compounds. One... 

Over time, statisticians have devised many tests for the distributions of data, including one that relies on visual inspection of a particular type of graph. Of course, this is no more than the direct visual inspection of the data or of the calibration residuals themselves. However, a statistical test is also available, this is the x2 test for distributions, which we have previously described. This test could be applied to the question, but shares many of the disadvantages of the F-test and other tests. The main difficulty is the practical one this test is very insensitive and therefore requires a large number of samples and a large departure from linearity in order for this test to be able to detect it. Also, like the F-test it is not specific for nonlinearity, false positive indication can also be triggered by other types of defects in the data. 

Table 67-1 presents the results of computing the linearity evaluation results for the curves shown in Figure 67-1, for the case of a uniform distribution of data along the... 

The normal distribution describes the way measurement results are commonly distributed. This type of distribution of data is also known as a Gaussian distribution. Most measurement results, when repeated a number of times, will follow a normal distribution. In a normal distribution, most of the results are clustered around a central value with fewer results at a greater distance from the centre. The distribution has an infinite range, so values may turn up at great distances from the centre of the distribution although the probability of this occurring is very small. 

In a normal distribution of data, 68.3% of the values lie within 1 standard deviation of the mean value while 95.4% of the values lie within 2 standard... 

Two of the major points to be made throughout this chapter are (1) the use of the appropriate statistical tests, and (2) the effects of small sample sizes (as is often the case in toxicology) on our selection of statistical techniques. Frequently, simple examination of the nature and distribution of data collected from a study can also suggest patterns and results which were unanticipated and for which the use of additional or alternative statistical methodology is warranted. It was these three points which caused the author to consider a section on scattergrams and their use essential for toxicologists. 

The first is to normalize the data, making them suitable for analysis by our most common parametric techniques such as analysis of variance ANOYA. A simple test of whether a selected transformation will yield a distribution of data which satisfies the underlying assumptions for ANOYA is to plot the cumulative distribution of samples on probability paper (that is a commercially available paper which has the probability function scale as one axis). One can then alter the scale of the second axis (that is, the axis other than the one which is on a probability scale) from linear to any other (logarithmic, reciprocal, square root, etc.) and see if a previously curved line indicating a skewed distribution becomes linear to indicate normality. The slope of the transformed line gives us an estimate of the standard deviation. If... 

The distribution of data plays an important role in statistics chemometrics is not so happy with this concept because the number of available data is often small or the... 

By plotting the highly precise Mg isotope data collected by MC-ICPMS in terms of 8 Mg vs. 8 Mg (Table 2) it is possible to constrain the values for p from the best-fit slopes defined by the data and therefore the nature of the fractionation processes that lead to the distribution of data. This method is applied in sections below. 

Substitution of the typical variances and covariance into Equation (19) suggests that the for tho MC-ICPMS measurements of Mg in solutions is on the order of +0.010%o. This is regarded as an internal precision for an individual solution measurement. We note, however, that the reported measurements represent averages of several replicate analyses of the same solution and so more realistic assessments of the internal precision for A Mg data presented here would be obtained from the imcertainties in the means (standard errors). For example, four analyses of the same solution yields a standard error for A Mg of +0.005%o (this is stiU regarded as an internal precision because the effects of column chemistry and sample dissolution are not included). No attempt has been made here to review all of the raw data sets to calculate standard errors for each datum in Table 1. However, the distribution of data indicates that +0.010%o Icj is an overestimate of the internal precision of A Mg values and that a more realistic imcertainty is closer to a typical standard error, which in most cases will be < +0.005%o (since the number of replicates is usually >4, e.g., Galy et al. 2001). 

Construction of an Approximate Confidence Interval. An approxi-mate confidence interval can be constructed for an assumed class of distributions, if one is willing to neglect the bias introduced by the spline approximation. This is accomplished by estimation of the standard deviation in the transformed domain of y-values from the replicates. The degrees of freedom for this procedure is then diminished by one accounting for the empirical search for the proper transformation. If one accepts that the distribution of data can be approximated by a normal distribution the Student t-distribution gives... 

Further analysis of linearity data typically involves inspection of residuals for fit in the linear regression form and to verify that the distribution of data points around the line is random. Random distribution of residuals is ideal however, non-random patterns may exist. Depending on the distribution of the pattern seen in a plot of residuals, the results may uncover non-ideal conditions within the separation that may then help define the range of the method or indicate areas in which further development is required. An example of residual plot is shown in Figure 36. There was no apparent trend across injection linearity range. 

Tab. 3.1 S ummed distribution of data from Figure 3.2 across the ca. 222k pharmacophore GaP space...

All residue levels greater than 1,0 were coded in the analysis file with an extra 0 between the decimal point and the first unit s place. For example 2.46 was recorded as 20.46. The limited number of such levels did not significantly affect previously computed univariate statistics and these artificial outliers remained undetected. Figure 5. presents a plot of the PRINCOMP output after the analysis file was corrected. This plot shows a more uniform distribution of data points for specimens collected in each of the three years. 

The probability that 99.7 % of the correct measurement results are expected to lie within the action limits means that only 3 out of 1000 correct measurement results will lie outside this range. Correct means that they belong to the same statistical distribution of data. Therefore the probability is very high that a result that is outside the action range is incorrect. 

Odd sample values occurring some way from the cluster of values around the mean are known as outliers. The problem of whether or not they are acceptable, especially with skewed distributions of data, is considered in AMC (2001), which is also found at ... 

Prerequisite for the t-test is a normal distribution of data, i.e., the frequencies of data with the same deviation from mean forms a bell-shaped curve. In case of a large number of experimentally obtained data, mostly a Gaussian distribution is given. 

An extreme, large positive, value may sometimes be a manifestation of an underlying distribution of data that is heavily skewed. Transforming the data to be more symmetric may then be something to consider. 

This equation describes the linear distribution of data points on a plot of 8D versus 8lsO that is commonly referred to as the global meteoric-water line (GMWL). The zero intercept for this line, defined as the deuterium excess... 

From the data listed in Tables I-V, we conclude that most authors would probably accept that there is evidence for the existence of a compensation relation when ae < O.le in measurements extending over AE 100 and when isokinetic temperature / , would appear to be the most useful criterion for assessing the excellence of fit of Arrhenius values to Eq. (2). The value of oL, a measure of the scatter of data about the line, must always be considered with reference to the distribution of data about that line and the range AE. As the scatter of results is reduced and the range AE is extended, the values of a dimin i, and for the most satisfactory examples of compensation behavior that we have found ae < 0.03e. There remains, however, the basic requirement for the advancement of the subject that a more rigorous method of statistical analysis must be developed for treatment of kinetic data. In addition, uniform and accepted criteria are required to judge quantitatively the accuracy of obedience of results to Eq. (2) or, indeed, any other relationship. 

The values of the normed Laplace s-function are in Table J (appendix). When determining the given values zg=zmin is replaced by -< >, and zfe-zrn lx by +°°. When the calculated value of the Pirson criterion is below the tabular value the null hypothesis on normal distribution of data is accepted ... 

One has to keep in mind that groups of objects found by any clustering procedure are not statistical samples from a certain distribution of data. Nevertheless the groups or clusters are sometimes analyzed for their distinctness using statistical methods, e.g. by multivariate analysis of variance and discriminant analysis, see Section 5.6. As a result one could then discuss only those clusters which are statistically different from others. 

---