Correlation Between Two Sets of Data

In science, one is often interested in whether one type of data is connected with another type, i.e. whether the data points from one set can be used to predict the other. We will denote such two data sets x and y, and ask whether there is a function /(x) that can model the y data. When the function /(x) is defined or known a priori, the question is how well the function can reproduce the y data. 

Two quantities are commonly used for qualifying the goodness of fit , the Root Mean Square (RMS) deviation and the Mean Absolute Deviation (MAD), which for a set of N data points are defined in eq. (17.4). 

When the functional form/(x) is unknown, correlation analysis may be used to seek an approximate function connecting the two sets of data. The simplest case corresponds to a linear correlation. 

We want to determine the a (slope) and b (intersection) parameters to give the best possible fit, i.e. in a plot of y against x, we seek the best straight line. 

The familiar least squares linear fit arises by defining the best line as the one that has the smallest deviation between the actual yrpoints and those derived from eq. (17.5), and taking the error to be the deviation squared. The equations defining a and b can be derived by minimizing (setting the first derivations to zero) an error function. 

---

It is worth noting that ISO/IEC and ASTM test methods for many properties, in principle, are similar and the differences between the specific test methods are rather minimal. Even so, differences in measured data between the two methods, depending on the type of material, are expected, particularly with thickness-dependent properties such as impact strength, DTUL, and flexural properties stemming from the variance in test specimen dimensions and its preparation. Thus, any correlation between two sets of data is dependent on the material type, implying that one should not assume that the property values generated by the ISO test methods would always be equivalent to the values obtained by using the ASTM method with simple conversion to appropriate units. 

Hypothesis testing is used in the experimental research if the aim of the experiment is to determine whether the difference between two characteristics, such as two means or two standard deviations, is caused by controlled changes in independent variables or to examine the significance of correlation between two sets of data. In statistical terms, hypothesis is a statement about the relationship between two statistical parameters. It includes a null hypothesis, usually stating that two parameters are equal, which is tested against an alternative hypothesis that they are not. Table 1.4 summarizes the equations and the rules for making the dedsion in favour of one of the hypotheses. 

Kendall s rank correlation, represented by r(tau), should be used to evaluate the degree of association between two sets of data when the nature of the data is such that the relationship may not be linear. Most commonly, this is when the data are not continuous and/or normally distributed. An example of such a case is when we are trying to determine if there is a relationship between the length of hydra and their survival time in a test medium in hours. Both of our variables here are discontinuous, yet we suspect a relationship exists. Another common use is in comparing the subjective scoring done by two different observers. 

The correlation coefficient (y) is a measure of linear relationship between two sets of data. It can attain a value which may vary between 0 and 1. A value of+1 (or-1, when the slope is negative) indicates the maximum possible linearity on the other hand, a zero y indicates there is absolutely no link between the data. In environmental analysis, especially in spectrophotometric methods, y is calculated to determine the linearity of the standard calibration curve, y may be calculated from one of the following equations. 

Correlation analysis only asks whether there is a relationship between two sets of data. Regression goes a step further and asks how are they related More specifically it derives a mathematical equation that will allow us to predict one of the parameters if we know the value of the other. 

Correlation—The relationship between two sets of data such that when one changes, the other is likely to make a corresponding change. If the changes are in the same direction, there is positive correlation. When changes tend to occur in opposite directions, there is negative correlation. When there is a little correspondence or random changes, there is no correlation. 

Figure 2.15(a) shows the relationship between and Cp for the component characteristics analysed. Note, there are six points at q = 9, Cp = 0. The correlation coefficient, r, between two sets of variables is a measure of the degree of (linear) association. A correlation coefficient of 1 indicates that the association is deterministic. A negative value indicates an inverse relationship. The data points have a correlation coefficient, r = —0.984. It is evident that the component manufacturing variability risks analysis is satisfactorily modelling the occurrence of manufacturing variability for the components tested. 

Here, the notation (, I C, X2) stands for the squared multiple correlation coefficient (or coefficient of determination) of the multiple regression of y, on Xj and X2. The improvement is quite modest, suggesting once more that there is only a weak (linear) relation between the two sets of data. 

In order to obtain an in vitro-in vivo relationship two sets of data are needed. The first set is the in vivo data, usually entire blood/plasma concentration profiles or a pharmacokinetic metric derived from plasma concentration profile (e.g., cmax, tmax, AUC, % absorbed). The second data set is the in vitro data (e.g., drug release using an appropriate dissolution test). A mathematical model describing the relationship between these data sets is then developed. Fairly obvious, the in vivo data are fixed. However, the in vitro drug-release profile is often adjusted by changing the dissolution testing conditions to determine which match the computed in vivo-release profiles the best, i.e., results in the highest correlation coefficient. 

This measure is equivalent to the correlation coefficient between two sets of mean-centered data—corresponding here to the vector components of xA and xB. It is frequently used for the comparison of spectra in IR and MS. 

Specific to our research, the multidimensional techniques such as MBS, factor analysis, canonical correlation and regression analysis have been used not only to analyze sensory and analytical data, but also to perform correlation between the two sets of data. 

The quadrant sum test is recommended as a quick check on whether a correlation exists between two sets of measurements. It is a good method, although it relies entirely on the extreme values of the variables to detect a correlation. Every once in a while, one runs across a situation where the main bulk of the data appears to be correlated, but one or two of the extreme values fall out of line. In these situations, the quadrant sum test fails to detect the relationship. 

Table I. This analysis reveals that the primary effect of substituents in both the meta and para positions, as indicated by the magnitudes of these p values, is the inductive effect. Resonance effects are small. The situation therefore is analogous to that found in the treatment of acidities. However, the correlation method does not provide a clear distinction between two sets of resonance parameters, gr(BA) and aR. The degree of fit, presented in the form of a ratio of the standard deviation (SD) to the root mean square (RMS) of the data, is similar for both resonance parameters. Perhaps this limitation reflects an early transition state in which resonance effects play a small role.

Canonical correlation analysis identifies and quantifies the associations between two sets of variables [126]. Canonical correlation analysis is conducted by using canonical variates. Consider n observations of two random vectors X and y of dimensions p and q forming data sets Xpxn and Y xn with... 

In order to obtain the time lag between two sets of reflected light signals of known distance apart, points a and b, for particles in random movement, the cross-correlation function method is generally used in treating the signals. The cross-correlation function of two sets of random signals a t) and b(t) express the independence of the two sets of sampled data, i.e.,... 

Although extractive CO concentrations were found to be lower than the TDL-based measurements, analysis revealed a close correlation between these two sets of data. The analysis also displayed a time lag between the two measurements, which is to be expected since the extractive measurements were done 100 m downstream of the TDL measurements. 

Suppose we have two sets of data values, x and y, and we wish to determine what correlation (if any) exists between them. For example, imagine that we are performing a simulation of a fluid in a capillary, and that we wish to determine the correlation between the absolute velocity of an atom and its distance from the wall of the tube. One way to do this would be to plot the sets of values as a graph. A correlation function (also known as a correlation coefficient) provides a numerical value that encapsulates the data and quantifies the strength of the correlation. A series of simulations with different capillary diameters could then be compared by examining the correlation coefficients. A variety of correlation functions can be defined, a commonly used one being ... 

If there is the possibility of some correlation (i.e., a relationship between the two sets of data), Ishikawa recommends using a simple sign test to see if the correlation is significant. This test is illustrated in Figure 5.14. 

The sensory data obtained from a panel are calculated by determining the overall mean scores for intensity or quality total score points divided by the number of panelists for each sensory session. In some panel procedures, the scores are discarded if the means differ by more than two units from the average score. This procedure requires at least 10 or 12 final judgments for statistical analyses. The significance of the overall mean scores is calculated statistically by two-way analysis of variance (ANOVA). Sensory scores can also be correlated with the results of other tests of Upid oxidation by regression analyses. If an objective test correlates well with sensory analyses, it is usually interpreted as giving similar information regarding the level of oxidation. However, correlation data must be interpreted with care because they can only be used to show trends between two sets of analyses, and caimot be used to obtain cause and effect relationships. 

---