Log-transformation

The best known approach to measurements with positive skewness is transformation. In environmental data analysis, the measurements are often transformed to their logarithms. In this paper, we consider power transformations with a shift, a set of transformations that includes the log transformation and no transformation at all ( ). These transformations are given by... 

Each of these data sets is skewed, yet each can be transformed to normality. With no transformation applied, the probability plot correlation coefficients for the Co, Fe, and Sc data sets are 0.855, 0.857, and 0.987, respectively. For Co and Fe, the hypothesis of normality is rejected at the 0.5 percent level (12). On the other hand, the maximum probability plot correlation coefficients are 0.993, 0.990, and 0.993 for Co, Fe, and Sc, respectively. The maxima occur at (X,t) (0,0.0048), (0,0.42), and (0.457,0), respectively. These maxima are so high that they provide no evidence that the range of transformations is inadequate. Note that the (, x) values at which the maxima occur correspond to log transformations with a shift for the Co and Fe and nearly a square-root transformation for the Sc. 

Scaling is a very important operation in multivariate data analysis and we will treat the issues of scaling and normalisation in much more detail in Chapter 31. It should be noted that scaling has no impact (except when the log transform is used) on the correlation coefficient and that the Mahalanobis distance is also scale-invariant because the C matrix contains covariance (related to correlation) and variances (related to standard deviation). 

Also, if conversion of drug to active metabolite shows significant departure from linear pharmacokinetics, it is possible that small differences in the rate of absorption of the parent drug (even within the 80-125% range for log transformed data) could result in clinically significant differences in the concentration/ time profiles for the active metabolite. When reliable data indicate that this situation may exist, a requirement of quantification of active metabolites in a bioequivalency study would seem to be fully justified. 

Figure 2 Typical log-transformed decays of DFR for estimating half-lives of methio-carb (o) and thiophanate-methyl (+).

The DFR values were followed over a period of 4 weeks from the high-volume application. The decrease of DFR in the two zones was monitored in six greenhouses (three times after application of thiophanate-methyl and three times after application of methiocarb). Figure 2 shows typical log-transformed decays of DFR (average of two samples for each zone) for methiocarb and thiophanate-methyl. Assuming a first-order decay, half-lives were calculated using Equations (3) and (4) and were found to be 29 8.5 days and 11 3.4 days for thiophanate-methyl and methiocarb, respectively. 

Principal component analysis (PCA) of the soil physico-chemical or the antibiotic resistance data set was performed with the SPSS software. Before PCA, the row MPN values were log-ratio transformed (ter Braak and Smilauer 1998) each MPN was logio -transformed, then, divided by sum of the 16 log-transformed values. Simple linear regression analysis between scores on PCs based on the antibiotic resistance profiles and the soil physico-chemical characteristics was also performed using the SPSS software. To find the PCs that significantly explain variation of SFI or SEF value, multiple regression analysis between SFI or SEF values and PC scores was also performed using the SPSS software. The stepwise method at the default criteria (p=0.05 for inclusion and 0.10 for removal) was chosen. 

Two data sets of LC50 values for different time periods of exposure were analyzed using a linear regression analysis of the log-log transformation of a plot of C vs t to derive values of n for monomethylhydrazine. 

To determine whether the skew was responsible for the taxonic findings, Gleaves et al. transformed the data using a square root or log transformation and were successful at reducing the skew of all but one indicator to less than 1.0. This is a fairly conservative test of the taxonic Conjecture, because data transformation not only reduces indicator skew, but it can also reduce indicator validities, and hence produce a nontaxonic result. Yet, this did not happen in this study. All but one plot originally rated as taxonic were still rated as taxonic after the transformation. MAMBAC base rate estimates were. 19 (SD =. 18) for transformed empirical indicators, and. 24 (SD =. 06) for transformed theoretical indicators. Nevertheless, these estimates are probably not as reliable as the original estimates because of the possible reduction in validity, which is likely to lower the precision of the estimates. 

In all these cases, A and B are constant while p is a log transform. These curves are illustrated in Figure 22.7. 

If the data distribution is extremely skewed it is advisable to transform the data to approach more symmetry. The visual impression of skewed data is dominated by extreme values which often make it impossible to inspect the main part of the data. Also the estimation of statistical parameters like mean or standard deviation can become unreliable for extremely skewed data. Depending on the form of skewness (left skewed or right skewed), a log-transformation or power transformation (square root, square, etc.) can be helpful in symmetrizing the distribution. 

As already noted in Section 1.6.1, many statistical estimators rely on symmetry of the data distribution. For example, the standard deviation can be severely increased if the data distribution is much skewed. It is thus often highly recommended to first transform the data to approach a better symmetry. Unfortunately, this has to be done for each variable separately, because it is not sure if one and the same transformation will be useful for symmetrizing different variables. For right-skewed data, the log transformation is often useful (that means taking the logarithm of the data values). More flexible is the power transformation which uses a power p to transform values x into xp. The value of p has to be optimized for each variable any real number is reasonable for p, except p 0 where a log-transformation has to be taken. A slightly modified version of the power transformation is the Box Cox transformation, defined as... 

FIGURE 3.9 PCA for skewed autoscaled data In the left plot PCI explains 79% of the total variance but fails in explaining the data structure. In the right plot x2 was log-transformed and then autoscaled PCI now explains 95% of the total variance and well follows the data structure. 

By looking at the data one can observe right-skewed distributions for some of the variables. Thus an appropriate data transformation (e.g., the log-transformation) can improve the quality of the cluster results. However, it turned out that the results changed only marginally for the transformed data, and thus they will not be presented in the following. 

The semi-log transformation again makes the rise and fall of the graph linear. Note that this time there is no recirculation hump. As the fall on the initial plot was exponential, so the curve is transformed to a linear fall by plotting it as a semi-log. The AUC is still used in the calculations of cardiac output. 

The reference sampling rate (/ s,ref) as well as the exposure-specific effect jSj are divided out. For practical applications, it therefore suffices to know how the compound-specific effect depends on the properties of the analytes. Observing that the experimental sampling rates have a similar dependence on log ATow, but show a varying offset for the different studies, the log-transformed sampling rates observed in 19 calibration experiments in 9 studies were fitted as a third order polynomial in log Kq -... 

Ordinary least squares regression requires constant variance across the range of data. This has typically not been satisfied with chromatographic data ( 4,9,10 ). Some have adjusted data to constant variance by a weighted least squares method ( ) The other general adjustment method has been by transformation of data. The log-log transformation is commonly used ( 9,10 ). One author compares the robustness of nonweighted, weighted linear, and maximum likelihood estimation methods ( ). Another has... 

Another solution to the problem of non-constant variance is to transform the response data. A common way of transforming data has been by taking the logarithms of both the response and amount variables ( 8-10 ). However, for all the data we looked at, the log transformation has been too strong. See Tables I and V. 

Table V shows that in some data sets the log transformation is acceptable, but this is usually as a result of less perfect data and not an inherent quality.

To demonstrate the accuracy of three data treatment methods we show the results of treating the same data in three ways no transformation, log-log transformation, and the selected transformation as determined by this work. 

---