Data distribution kurtosis

Another problem which is frequently encountered in the distribution of data is the presence of outliers. Consider the data shown in Table 3.1 where calculated values of electrophilic superdelocalizability (ESDLIO) are given for a set of analogues of antimycin Al, compounds which kill human parasitic worms, Dipetalonema vitae. The mean and standard deviation of this variable give no clues as to how well it is distributed and the skewness and kurtosis values of —3.15 and 10.65 respectively might not suggest that it deviates too seriously from normal. A frequency distribution for this variable, however, reveals the presence of a single extreme value (compound 14) as shown in Fig. 3.5. This data was analysed by multiple linear regression (discussed further in Chapter 6), which is a parametric method based on the normal distribution. The presence of this outlier had quite profound effects on the analysis, which could have been avoided if the data distribution had been checked at the outset (particularly by the present author). Outliers can be very informative and should not simply be discarded as so frequently happens. If an outlier is found in one of the descriptor variables (physicochemical data), then it may show... 

To facilitate interpretation of the outputs, the authors also created two simulation data sets with identical distributional properties (number of indicators, number of levels, indicator intercorrelations, skew and kurtosis) one taxonic set and one dimensional set. The taxonic data set was created to have a base rate of. 23, which corresponds to the proportion of cases falling at or above a BDI threshold of 10 in the undergraduate data set. Ruscio and Ruscio tried to ensure that indicator validities and nuisance correlations matched the estimated parameters of the real indicators, but they did not indicate how successful this was. 

One major task of statistics is to describe the distribution of a set of data. The most important characteristics of a distribution are the location, the dispersion, the skewness and the kurtosis. These are discussed in the following slides. 

There is a fourth parameter to describe the characteristics of a distribution, which is the kurtosis. This is the peakedness of the data set. If the data set has a flat peak in the distribution curve it is called platykurtic. If the peak is very sharp it is called leptokurtic. Distributions with peaks in-between are called mesokurtic. 

Asa rule, accurate estimation of lower distribution moments will require fewer data than accurate estimation of higher moments (e.g., fewer data are needed for a decent estimate of the mean than for a decent estimate of kurtosis). 

Besides the calculation of average molecular weights, several other means of characterizing the distribution were produced. These include tables of percentile fractions vs. molecular weights, standard deviation, skewness, and kurtosis. The data for the tables were obtained on punched cards as well as printed output. The punched cards were used as input to a CAL COMP plotter to obtain a curve as shown in Figure 3. This plot is normalized with respect to area. No corrections were made for axial dispersion. 

Fig.l and Fig.2 contain histograms showing the distribution of the oxides and the ions in their various structural positions. Most of the Si02, A1203, MgO, Na20, and K2 O have a normal type distribution. The interlayer cations have a log-normal distribution and the tetrahedral and octahedral cations have a normal-type distribution. Calculated skewness and kurtosis values are listed in Tables III and VI. The data are too limited to draw any significant conclusions. Ahrens (1954) and others have shown... 

The standardized coefficients, both skewness and kurtosis, indicate significant deviations from the normal distribution. The data depart significantly from a normal distribution when the standardized coefficients are outside the range — 2.0 to + 2.0. 

Low values of the kurtosis are obtained when the data points (i.e. the atom projections) assume opposite values (-/ and t) with respect to the centre of the scores. When an increasing number of data values are within the extreme values t along a principal axis, the kurtosis value increases (i.e., k = 1.8 for a uniform distribution of points, K = 3.0 for a normal distribution). When the kurtosis value tends to infinity the corresponding T] value tends to zero. 

The coefficient is approximately zero for the Gaussian distribution. The sign of a nonzero coefficient indicates the type of kurtosis present in the data (Figure 16-5, C and D). 

These experiments show that correlation coefficients between individual descriptors and the ASDs are valuable for indicating similarity and diversity of data sets and compounds. Since the effects of symmetry of distribution may be significant, it is recommended that correlation coefficients be handled together with skewness or kurto-sis. The skewness and kurtosis of the descriptors both show the same trends, although the kurtosis — in particular, the Harness of distribution — is generally more sensitive... 

The terms similarity and diversity can have quite different meanings in chemical investigations. Describing the diversity of a data collection with a general valid measure is almost impossible. Descriptor flexibility allows the characterization of similarity by means of statistics for different tasks. The statistical evaluation of descriptors shows that it is recommended to interpret correlation coefficients together with the symmetry of distribution. In contrast to correlation coefficients, skewness and kurtosis are sensitive indicators to constitutional and conformational changes in a molecule. This feature allows a more precise evaluation of structural similarity or diversity of molecular data sets. 

The mean and standard deviation of correlation coefficients seems to be a reliable diversity measure. However, as mentioned in the theoretical section, the reliability of the correlation coefficient itself depends on the symmetry of distribution within a descriptor skewness or kurtosis should be regarded if a data set has to be classified as similar or diverse. 

What is more important for a diversity evaluation is that with increasing diversity of a descriptor collection, the mean deviation in skewness should increase. In fact, the skewness of the two data sets investigated is similar and does not clearly indicate a difference between the sets (Table 6.1), whereas the deviation in skewness of the high-diversity data set is about twice the one in the low-diversity data set. The distribution of the kurtosis of the data sets leads to a similar result. 

Whereas the mean correlation coefficient is significantly lower in the arbitrary data set, the mean skewness and mean kurtosis are similar. Though the latter values do not indicate clearly a difference between the data sets — they just indicate a similar symmetry and flatness of distribution — the deviations from the average behavior describes properly the diversity of the data set The average deviations in skewness and kurtosis are about twice as high in the arbitrary data set as those of the benzene derivatives. The ASD and the combination of deviations in correlation coefficients, skewness, and kurtosis provide the most reliable measure for similarity and diversity of data sets. 

At times, particularly if Y is negative, such as may be the case when modeling percent change from baseline as the dependent variable, it may necessary to add a constant c to the data. Berry (1987) presents a method that may be used to define the optimal value of c while still ensuring a symmetrical distribution. The method makes use of skewness and kurtosis, which characterize the degree of asymmetry of a distribution. These statistics will be used to identify a transformation constant c such that the new data set will have better normality properties than the original scores. Let 7, and y2 be the skewness and kurtosis of the distribution as defined by Eqs. (4.36) and (4.37), respectively, substituting Ln(Y + c) for the variable e in Eq. (4.38). If g is defined as... 

Testing for nonnality. One method that has been suggested for testing whether the disfribution underlying a sample is normal is to refer the statistic L = n skewness /6 + (kurtosis-3) /24 to the chi-squared distribution with 2 degrees of freedom. Using the data in Exercise 1, earry out the test. 

With respect to the normality condition of the composite T-scores in Table 1, skewness and kurtosis values confirm that the distributions are close to a normal curve so the data may be analyzed through parametric analyses such as f-test and Pearson correlation. Mean comparisons are performed to investigate if gender-based differences exist. 

Analysis of data obtained in experiments usually starts with the estimation of statistical measures that characterize the range, the mean value, the variance of the data, and their confidence intervals. Sometimes, when the experiment concerns the identification of changes in the distribution of the dependent factor, such as fibre length or fibre diameter distribution, the analysis continues with the estimation of the skewness and kurtosis, which are measures of the distribution symmetry and sharpness, respectively. Table 1.3 summarizes equations for the calculation of statistical measures. In this table Xi,X2,. ..,x . ..,x are individual measurements or observations for a sample of n measurements. 

When the fiuctuations show current or voltage transients, data analysis may be performed in the time domain by investigating the shape, size and occurrence rate of the random events. It can be also performed by measuring the moments of the potential or current fiuctuations (standard deviation, skewness, kurtosis) [12]. However, this approach is extremely limited for data interpretation. In the absence of current or voltage transients, the values of the moments are most likely close to zero, as for signals with a Gaussian distribution. Any deviation from zero indicates the existence of transients. 

The experimental grain size distribution is shown as a histogram in Figure 6.13. The main parameters are s 0.42, sk 0.23, and K 0.26. Thus, experimental data are somewhere in between Di Nunzio and Rayleigh the standard deviation almost coincides with Di Nunzio, skewness lies quite in the middle between Di Nunzio and Rayleigh, and kurtosis seems to be too sensitive (Table 6.1). 

The fourth mechanism for freak wave generation requires additional knowledge before predictions can be made. Relationship between kurtosis and freak wave events has been studied using laboratory data, numerical results, and field data. North Sea data shows that the maximum crest, normalized by its wave heights, correlates with skewness, while the wave heights, normalized by significant wave height, correlate with the kurtosis. This is because the kurtosis is the fourth moment of the probability density function hence, it measures the relevance of tails in a distribution. 

Figure 6.6 shows the probability of occurrence of freak waves i/max/ rms > 8 for short time records, as a function of kurtosis. We assumed 100 waves per record and counted the number of freak waves. In the experimental data, the occurrence probability for freak waves shows a clear dependence on the kurtosis (this effect of course is not described by the Rayleigh distribution). But the experimental data appear to depend quadratically on the kmtosis, while the nonhnear theory Eq. (6.31) predicts a linear dependence. This is probably because the nonlinear theory includes only the lowest-order correction for nonlinearity it excludes high-order cumulants. 

---