Classification and validation

Furey TS, Christianini N, Duffy N, Bednarski DW, Schummer M, Haussler D. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics 2000 16 906-14. 

Consider a two-structure model of L-cules and -cules obtained by any exact classification procedure as discussed in Chapter 6. The argument presented below is general, independent of the choice of classification, and valid for any solvent. 

Bioinformatics, 16, 906 (2000). Support Vector Machine Classification and Validation of Cancer Tissue Samples Using Microarray Expression Data. 

Identification and validation of biomarkers predictive of disease, particularly cancer, is a significant and expanding area in clinical research. Quality cancer biomarkers should facilitate early detection and diagnosis of the disease, with more specific markers used for classification (such as grade/stage) or subtype determination of the... 

Such an inter-type difference will not be utilized in this book, mainly because it complicates the classification and is not necessary as tbe focus is placed on the substrates and the products. The argument is also valid for enzymatic transformations [12d, 14], where one enzymatic system with one enzyme or different independent enzymatic systems with one or more enzymes may be used. In Nature, as well as in several artificial enzymatic domino reactions, a mixture of different enzymes catalyzing independent cycles is employed. 

The United Nations Globally Harmonized System of Classification and Labelling of Chemicals (GHS) includes an internationally standardized guidance procedure on Transformation/Dissolution Protocol (T/DP) for metals and sparingly soluble metal compounds (United Nations, 2007), recently validated by the OECD (Organization for Economic Cooperation and Development). To establish the acute aquatic hazard classification level of a metal-bearing substance under the GHS, data from the T/DP are compared with an acute ecotoxicity reference value (ERV) derived under conditions similar to those of the T/DP. 

Freshwater media based on the OECD 203 ecotoxicity testing medium for fish and daphnia have been used in all T/DP testing of metals, metal compounds and alloys in the pH range 6-8.5 to date. However, the composition of a marine medium is also given in the T/DP section of the GHS, and by implication, a method for marine T/D testing is open for development and validation. While not currently required for REACH dossiers, T/D data in marine media and attendant classification proposals may be required in the future for marine shipping. 

There are many results to be reviewed because there are multiple classes for which SIMCA models are constructed and validated. The order in which to examine the results is a matter of preference, and many approaches are equally appropriate. We will review one SIMCA model at a time, and examine the test set predictions for that one model against samples from all classes. Ideal performance of a SIMCA model means that it includes as part of the class those samples that truly belong to the class and excludes those samples that are from all of the other classes. In reality, a number of classification scenarios are possible. Table A. 18 lists the possibilities along with possible root causes for misclassified test samples. 

We apply our classification of study designs throughout the text to help the clinician interpret the quality of results from clinical trials. Further, we give the critical reader a perspective on the depth and validity of the available data. Most studies in our analyses of drug efficacy are class I or II, and if not, we discuss the studies accordingly. 

In addition to the somewhat empirical and difficult development of NIR applications, thorough documentation must be produced. NIR methods have to comply with the current good manufacturing practice (cGMP) requirements used in the pharmaceutical industry. Various regulatory aspects have to be carefully considered. For example, NIR applications in classification, identification, or quantification require extensive model development and validation, a study of the risk impact of possible errors, a definition of model variables and measurement parameters, and... 

The archiving section will include information about purpose, record classification, and archive location(s). It has to be defined which data generated during the validation of the spreadsheet will be classified as raw data or results (GXP records) the electronic data or a printout. The raw data and all data and documents that have been generated during validation must be archived. Figure 18.10 is an example. 

S. Lanteri, Full validation procedures for feature selection in classification and regression problems, Chemom. Intell. Lab. Syst., 15, 1992, 159-169. 

Fig. 8.11. The classifier development process. Clinical knowledge provides us with a set of classes for supervised classification (top, right). Large numbers of spectra from large sample numbers are reduced to a set of potentially useful features (top, left) or metrics. A modified Bayesian algorithm operates on the metrics to provide predictions that are compared to a gold standard. The end result of the training and validation process is an optimized algorithm, metric set, calibration and validation statistics, and sensitivity analysis of the data...

Fig. 8.12. A Conventional H E stained images and their classified counterpart (right) can be compared pixel by pixel for accuracy. B Selected regions showing correspondence between H E and class images. C Accuracy is best assessed by ROC curves shown for individual classes with colors as per legend in (A). D The number of spectral features and classification accuracy for training data (red) and validation data (blue) show that high accuracy is possible. The effects of over-training with a feature set larger than 45 can be seen...

As the uses of toxicological-based quantitative structure-activity relationships (QSARs) move into the arenas of priority setting, risk assessment, and chemical classification and labeling the demands for a better understanding of the foundations of these QSARs are increasing. Specifically, issues of quality, transparency, domain identification, and validation have been recognized as topics of particular interest (Schultz and Cronin, 2003). 

---