Complex data sets

Handling of complex data sets Visual data mining methods especially show huge advantages over classical approaches if only Httle information about the data is known or if the expected patterns and relationships are not clearly defined. Furthermore, very inhomogeneous data sets or data with a high noise level can still be analyzed by these methods. 

Microarray experiments generate large and complex data sets that constitute e.g. lists of spot intensities and intensity ratios. Basically, the data obtained from microarray experiments provide information on the relative expression of genes corresponding to the mRNA sample of interest. Computational and statistical tools are required to analyze the large amount of data to address biological questions. To this end, a variety of analytical platforms are available, either free on the Web or via purchase of a commercially available product. 

Mankind s impact on the global climate and whether pollution from modern energy use is indeed warming the Earth have become important issues for national and international policy makers. Political pressure and public sentiment are based on complex data sets that, alone, cannot tell the whole story. The ultimate question is whether our climate is becoming warmer because of the slow build-up in atmospheric greenhouse gas concentrations (1). The answer is not clear, because much of what we know about global climate change is inferred from historical evidence of uncertain quality. 

For reproducible expression analysis and protein quantification MS methods based on isotopic labeling are available. They were designed in conjunction with two or more dimensional chromatographic peptide separation coupled online to MS and require advanced bioinformatics input to analyze the complex data sets in a reasonable time frame. This is also true for the alternative fluorescence-based technology of differential gel electrophoresis (DIGE Fig. 10.6) with tailor-made software which allows statistical validation of multiple data sets. 

Over the past twenty years, an entirely new approach has been developed to get the most information out of the increasingly larger and more complex data sets that scientists are faced with. This approach involves the use of a very diverse set of fairly simple techniques which comprise exploratory data analysis (EDA). As expounded by Tukey (1977), there are four major ingredients to EDA. 

Pattern recognition studies on complex data from capillary gas chromatographic analyses were conducted with a series of microcomputer programs based on principal components (SIMCA-3B). Principal components sample score plots provide a means to assess sample similarity. The behavior of analytes in samples can be evaluated from variable loading plots derived from principal components calculations. A complex data set was derived from isomer specific polychlorinated biphenyl (PCBS) analyses of samples from laboratory and field studies. 

By using statistical methods for the interpretation, several conclusions were drawn from this complex data set ... 

The potential for multi-fluorochrome analysis has increased further with the availability of dyes excited by shorter wavelengths (such as Cascade blue) and longer wavelengths (such as Allophycocyamn and Texas red). With multiple laser flow cytometers, five cell-surface antigens (1) or more may be analyzed simultaneously. In principle, these methods are a natural extension of the methods outlined here, but require more sophisticated flow cytometers, and much more powerful computer facilities to explore the exponentially more complex data sets arising from such multiparameter analyses. 

To extract a maximum of chemical and analytical information from complex data sets. 

Principal component analysis and Kohonen self-organizing maps allow multivariate data to be displayed as a graph for direct viewing, thereby extending the ability of human pattern recognition to uncover obscure relationships in complex data sets. This enables the scientist or engineer to play an even more interactive role in the data analysis. Clearly, these two techniques can be very useful when an investigator believes that distinct class differences exist in a collection of samples but is not sure about the nature of the classes. 

There is a transition away from nsing a deterministic approach in which high end or upper bound point estimates and defanlt valnes are nsed towards nsing a probabilistic approach in distribntional models which incorporate complex data sets to build realistic estimates of exposure. While probabilistic dietary exposure assessments can now be carried out routinely for many pesticides, available occupational and residential exposure data sets are typically insufficiently robust. Work on developing newer exposure databases (e.g. ARTF, ORETF, AHETF and EUROPOEM II) and distributional use pattern data would facilitate this transition. The topic of probabilistic exposure assessment is covered in Chapter 8. 

Figure 17.11 Traditional representation of impedance data obtained for the AZ91 alloy at the corrosion potential after different immersion times in 0.1 M NaCI a) complex-impedance-plane or Nyquist representation (the lines represent the measurement model fit to the complex data sets) b) Bode representation of the magnitude of the impedance as a function of frequency and c) Bode representation of the phase angle as a function of frequency. (Taken from Orazem et al. ° and reproduced with permission of The Electrochemical Society.)...

A common method of data analysis is to examine changes in one variable at a time (e.g., solubility) and relate these to the structure of the compound. The large volumes of data that are now produced from high throughput methods, as well as their complexity and correlation, suggest that there are more productive ways to derive information from the data. One useful approach for analyzing large and complex data sets is multivariate analysis [27,28]. With this approach an entire data set is simultaneously analyzed to derive increased information. 

After our brief excursion into the classical arena to inspect some of the earliest entries in the similarity cavalcade, we now leapfrog the centuries to return foursquare to the present time. For the concept of similarity to be applicable in a modern scientific setting, it is necessary that it have considerable flexibility. It must be sufficiently broadly defined that it can cope, among other things, with the cla fication of diverse information into well-defined categories, the expression of relationships in a wide variety of conceptual spaces, the discernment of patterns in complex data sets, and the formulation of scientific hypotheses and laws. The first step involves envisaging some part of reality that has been isolated for separate study such a part is usually referred to as a domain of knowledge... 

---