Classifying data

The application described here shows how a classified data set of reactions producing pyrazole derivatives can be used to predict the correct regioisomer in a pyrazole synthesis before it is carried out practically in the laboratory. 

The mean can be evaluated from the classified data of the histogram it measures the center of the distribution. The mean (whose symbol is an overbar) is defined as... 

Note that a statistical study could be done on an electron micrograph like that shown in Fig. 1.1. The dimensions of the blobs could be converted to volumes and then to masses with a knowledge of the density of the deposited polymer. This approach could be organized into a table of classified data from which any of these averages could be calculated. 

Until information can be compared with similarly classified data its use must be limited. In order to plan ahead, a business will prepare a strategy or budget for the next trading year and probably several years thereafter, with that for the next year broken down into the business s scheme of accounting periods. 

Rubin, J., Friedman, H. P., A Cluster Analysis and Taxonomy System for Grouping and Classifying Data, IBM Corporation, Scientific Center, New York, 1967. 

The traditional method of analyzing multiple, cross-classified data has been to collapse the NxR contingency table over all but two of the variables, and to follow... 

Herein lies the value of these different averages the divergence between the averages calculated by different methods offers a clue as to the breadth of the distribution of particle sizes. Remember, the average, however evaluated, is only one measure of the distribution of sizes. A fuller description requires some measure of the width of the distribution as well. For classified data, the standard deviation (see Appendix C) is routinely used for this purpose. For characterizations based on macroscopic experiments such as we have been discussing it is quantities such as ds/d or dv/ds that quantify this spread. (The averages ds and dv are defined below and are also discussed in Appendix C.)... 

Two-way analysis of variance (and higher classifications) leads to the presence of interactions. If, for example, an additive A is added to a lube oil stock to improve its resistance to oxidation and another additive, B, is added to inhibit corrosion by the stock under load or stress, it is entirely possible that the performance of the lube oil in a standard ball-and-socket wear test will be different from that expected if only one additive has present. In other words, the presence of one additive may adversely or helpfully affect the action of the other additive in modifying the properties of the lube oil. The same phenomenon is clearly evident in a composite rocket propellant where the catalyst effect on burning rate of the propellant drastically depends on the influence of fine oxidizer particles. These are termed antagonistic and synergistic effects, respectively. It is important to consider the presence of such interactions in any treatment of multiply classified data. To do this, the two-way analysis of variance table is set up as shown in Table 1.24. 

The major objective of this chapter is to examine and synthesize the published literature with respect to sources and production of terrestrially derived DOC, its relationship with dissolved organic nitrogen (DON), and the mechanistic controls on their export from terrestrial ecosystems to surface waters. With the exception of wet precipitation (which is ranked by continental landmass), we have classified data for throughfall and soil solution under biome type. Where possible, we have shown mean and standard deviations of some biomes to illustrate the amount of variance within and between biomes. Relationships between DOC and DON are illustrated using only those studies that report both DOC and DON concentration. Because most research on DOC and DON has been accomplished in relatively undisturbed areas, particularly forests, this chapter concentrates on the aspect of diffuse-source allochthonous inputs to surface waters and not point-source inputs from urban and agricultural areas. Recent work by Westerhoff and Anning (2000), however, indicates that more research on effluent or point-source DOC as a contributor to riverine allochthonous inputs may be... 

Generation or use of existing rule-based screens Refinement of known rule-based screens Use of hierarchical models, discriminant functions or decision trees to classify data Generation of QSPkR models (replacing complex or 3D parameters with more rapidly calculable 1D and 2D parameters wherever possible)... 

There are several advantages in using SIMCA to classify data. First, an unknown sample is only assigned to the class for which it has a high probability. If the sample s residual variance exceeds the upper limit for every class in the training set, the sample would not be assigned to any of these classes because it is either an outlier... 

The essence of the differences between the operation of radial basis function networks and multilayer perceptrons can be seen in Figure 4.1, which shows data from the hypothetical classification example discussed in Chapter 3. Multilayer perceptrons classify data by the use of hyperplanes that divide the data space into discrete areas radial basis functions, on the other hand, cluster the data into a finite number of ellipsoid regions. Classification is then a matter of finding which ellipsoid is closest for a given test data point. 

Assumptions may be made or models adopted (often by implication) about a system being measured that are not consistent with reality. The selection of the method of data reduction may be partly on the basis of the model adopted and partly on the basis of features such as computation time and simplicity. Kelly classified data processing methods as direct, graphical, minmax, least squares, maximum likelihood, and bayesian. Each method has rules by which computations are made, and each produces an estimate (or numerical result) of reality. 

The test of the four lots is an example of a one-way ANOVA. The one-way comes from the fact that there is only one category (lot) into which the data is classified. Often, we have more than one category (class variable) in which we need to classify data. Although our interest may be to determine only whether a particular class variable has meaning, it is important to include other class variables that may influence the variability of the data. ANOVA involves a null hypothesis for each classification variable that proposes that the means at each different level of the class (category) are all equal. If we reject the null hypothesis we conclude in favor of the alternative hypothesis, that at least one mean in the class differs from at least one other mean in the class. This is also a conclusion that... 

Most of the accounting today is done by entering data into a computer and using software packages to record, accumulate, and classify data. The accounting... 

Today, analysis and prediction methods have mostly a statistical flavor. They are trained on classified data and make new classifications or predictions based on statistical models. In some sense, all Chapters of Volume 1 present such methods. For instance, the methods for homology-based protein structure prediction in Chapter 5 and 6 of volume 1 learn from a set of observed structures rules that predict alike structures. 

The development of QSARs for any endpoint requires biological effect data to model. Without these data, no modehng is possible. The hmitations of the data to be modeled (both biological and physicochemical) must be appreciated by both the model developer and user. Criteria have been estabhshed to classify data for QSAR analysis according to their quahty (Cronin, 2005 Cronin and Schultz, 2003). For some biological effects, large coherent databases have been specifically created for the development of products and possibly QSAR modeling. Unfortunately, this has not been the case in the area of skin permeabihty, and the enthusiastic QSAR modeler is left with historical literatore data unless the modeler has a personal source of in-house data to model. 

As with the supervised learning categorization networks, there are a few items that need to be discussed for the unsupervised case. Scaling or preprocessing of input data is still important. The number of output PEs is usually arbitrary unless you have reason to believe your data should fall into a certain number of categories. The issue of whether a network can be trained at the near-100% level is irrelevant because you do not know what the correct answers are. It is possible to use an unsupervised learning network to classify data whose correct classifications are known. In this case you can talk about percent correct again, see Chapter 10 of Ref. 19. 

In this classification numerical operations are performed that search for natural groupings of the spectral properties of pixels as examined in an image. The computer selects the mean class and covariance matrices to be used in classification. Once the data is classified, the classified data are assigned to some natural and spectral classes and the spectral classes are converted to information classes of interest. Some of the clusters are meaningless as they represent mixed classes of earth surface materials. The unsupervised classification attempts to cluster the Dn values of the scene into natural boundaries using numerical operations. 

The first two parts of this section describe supervised learning methods which may be used for the analysis of classified data. One technique, discriminant analysis, is related to regression while the other, SIMCA, has similarities with principal component analysis (PCA). The final part of this section discusses some of the conditions which data should meet when analysed by discriminant techniques. 

---