Misclassifications

Roles for an expert system can be learned by rule induction from a set of examples. This makes this method similar to the use of classifiers - it will solve problems of similar complexity and have similar disadvantages (e.g. possibility of unnoticed misclassifications). 

Neural network classifiers. The neural network or other statistical classifiers impose strong requirements on the data and the inspection, however, when these are fulfilled then good fully automatic classification systems can be developed within a short period of time. This is for example the case if the inspection is a part of a manufacturing process, where the inspected pieces and the possible defect mechanisms are well known and the whole NDT inspection is done in repeatable conditions. In such cases it is possible to collect (or manufacture) as set of defect pieces, which can be used to obtain a training set. There are some commercially available tools (like ICEPAK [Chan, et al., 1988]) which can construct classifiers without any a-priori information, based only on the training sets of data. One has, however, always to remember about the limitations of this technique, otherwise serious misclassifications may go unnoticed. 

Since functional outcome and risk of recurrent stroke are, in part, predictable based on the pathophysiologic subtype of stroke, the ability to accurately classify patients based on emergency clinical and imaging data would provide valuable predictive information. Unfortunately, misclassifications of stroke subtypes based on clinical data and a noncontrast CT scan are common. The final subtyping of stroke is made with all available clinical data, but is heavily influenced by neuroimaging studies that identify the location, size, and vascular distribution of the infarct, or that establish that the arteries supplying the region of stroke are stenotic or occluded. 

Friedman [12] introduced a Bayesian approach the Bayes equation is given in Chapter 16. In the present context, a Bayesian approach can be described as finding a classification rule that minimizes the risk of misclassification, given the prior probabilities of belonging to a given class. These prior probabilities are estimated from the fraction of each class in the pooled sample ... 

Performance requires a careful balance between generalization and memorization (Davis and Wang, 1995). On one hand, a cluster can be defined too broadly, leading to misclassification of data. On the other hand, clusters can be so tightly defined that similar patterns in the same pattern classes cannot be classified with confidence. The appropriate balance is dependent on the desired functionality. 

Next, Bayesian probabilities were computed and these produced a clear U-shaped pattern. The authors assigned cases with the probability of. 90 or above to the taxon, and found that 35 individuals were identified as taxon members by these rules, that is, the prevalence of this taxon was 3.3%. Note that the. 50 cutoff is generally associated with the lowest overall rate of misclassification, but the. 90 cutoff may be preferable under certain conditions. In epidemiological studies, however, accuracy—a low rate of misclassification—is the primary consideration. In fact, the actual prevalence of the taxon in the Waller and Ross study appears to be about 5% based on the non-Bayesian base rate estimates. Thus, it appears that the use of a conservative cutoff in this study may produce somewhat misleading findings. 

Finally, the base rate estimates were consistent across these four methods (two variants of Golden s procedure, SSMAXCOV and SQUABAC), ranging between. 44 and. 46. Unfortunately, no indexes of internal (within-method) consistency were reported, so it is difficult to determine the strength of each individual piece of evidence. Also, all methods consistently indicated that the hitmax in this sample, and hence the cutoff with the lowest overall rate of misclassification, was between the PCL-R total scores of 18 and 20, which is considerably lower than the conventional cutoff of 30. The implications of this finding remain to be explored. One concern regarding the accuracy of the cutoff is that it was derived with dichotomous PCL-R... 

Table 2. Confusion matrix for a test of the PLS-DA model for the Coso obsidian locality. The remaining 575 broadband LIBS spectra for Coso (not included in the original model) were assigned with nearly perfect locality classification (99.8%). Only Coso sample 189 from South Sugarloaf had some misclassification with Shoshone sample 173.

Private residences or other nonhospital facilities that are used to isolate confirmed or suspected smallpox patients should have nonshared ventilation, heating, and air-conditioning systems. Access to those facilities should be limited to recently vaccinated persons with a demonstrated immune response. If suspected smallpox patients are placed in the same isolation facility, they should be vaccinated to guard against accidental exposure caused by misclassification as someone with smallpox. 

This preliminary assessment will need to be updated as and when further information becomes available. It should favor sensitivity over specificity so that a borderline possible-probable case is classified as probable rather than possible to make certain that the case is not lost when at a later stage the probable cases are picked out as a signal. A full assessment when all the information is available can then rectify any misclassifications. 

The permeability of the drug substance can be determined by different approaches such as pharmacokinetic studies in humans (fraction absorbed or mass balance studies) or intestinal permeability studies (in vivo intestinal perfusion studies in humans or suitable animal models or in vitro permeation studies using excised intestinal tissue or epithelial cell culture monolayers like CaCo-2 cell line). In order to avoid misclassification of a drug subject to efflux transporters such as P-glycoprotein, functional expression of such proteins should be investigated. Low- and high-permeability model... 

FIGURE 5.3 An optimal discriminant mle is obtained in the left picture, because the group covariances are equal and an adjustment is made for different prior probabilities. The linear mle shown in the right picture is not optimal—in terms of a minimum probability of misclassification—because of the different covariance matrices. 

We come back to the problem of selecting the optimum dimensions a, ..., ak of the PCA models. This can be done with an appropriate evaluation technique like CV, and the goal is to minimize the total probability of misclassification. The latter can be obtained from the evaluation set, by computing the percentage of misclassified objects in each group, multiplied by the relative group size, and summarized over all groups. 

The above example demonstrates that the choice of k is crucial. As mentioned above, k should be selected such that the smallest misclassification rate for the test data occurs. If no test data are available, an appropriate resampling procedure (CV or bootstrap) has to be used. Figure 5.14 shows for the example with three overlapping groups, used above in Figure 5.12, how k can be selected. Since we have independent training and test data available, and since their group membership is known, we... 

FIGURE 5.14 k-NN classification for the training and test data used in Figure 5.12. The left plot shows the misclassification rate for the test data with varying value of k for k-NN classification, and the right plot presents the result for k = 25. The misclassified objects are shown by dark symbols. 

The Gini index and the cross entropy measure are differentiable which is of advantage for optimization. Moreover, the Gini index and the deviance are more sensitive to changes in the relative frequencies than the misclassification error. Which criterion is better depends on the data set, some authors prefer Gini which favors a split into a small pure region and a large impure one. 

---