Empirical Learning from Examples

There are many classes of concepts functions, relations, sequences, (stochastic) languages, and so on. The language of examples used for illustrating these concept classes, the nature of admissible presentations of these examples, and the kinds of hypotheses for describing elements of these concept classes vary accordingly. 

Example 3-6 The concept class aimed at by Definition 3-1 is the set of relations over any domain. Examples of a relation are elements of its graph, and any (possibly infinite) sequence of such examples constitutes an admissible presentation iff this sequence lists all and only the elements of the graph of the relation. Hypotheses could be expressed as algorithms, programs, Turing machines, automata, and so on. 

Example 3-7 In grammatical inference, the concept class aimed at is the set of languages over some alphabet. Examples of a language are sentences of that language. Hypotheses may be expressed as grammars, acceptors, regular expressions, and so on. 

Restrictions on admissible presentations may have to be imposed, such as computability (below some complexity threshold), or ordering according to some total order. A positive presentation lists all and only the positive examples, whereas a complete presentation lists all the positive and negative examples. A presentation by informant queries an oracle for classification of examples as positive or negative examples. A mixed presentation combines complete presentation and presentation by informant. Repetition is usually allowed in admissible presentations. 

Learning can be abstracted as the search through a state space, where states correspond to hypotheses, and operators correspond to rules of inductive inference. 

---

It should be noted that Definition 3-1 is not the only possible definition for specifications by examples. It is too much geared towards the synthesis of algorithms. So let s generalize it to the larger framework of empirical learning from examples. Specification approaches vary according to the following criteria ... 

In Section 3.2.1, we suggest a terminology for the components of an empirical learning system. Then, in Section 3.2.2, we define rules of inductive inference, and survey their usage in empirical learning from examples in Section 3.2.3. Finally, in... 

Section 3.2.4, we define the niche of algorithm synthesis from examples within empirical learning from examples, and give pointers to the literature in Section 3.2.5. 

The synthesis of algorithms from examples is a machine learning task as it falls into the category of empirical learning from examples. Indeed, let s state the objectives of both fields ... 

But algorithm synthesis from examples is a highly specialized niche within empirical learning from examples. Table 3-1 summarizes the differences between the concerns of algorithm synthesis (as we view it) and the mainstream concerns (so far) of empirical learning. 

In Section 3.2.4, we have defined algorithm synthesis from examples as a niche of empirical learning from examples. As a reminder, for algorithm synthesis, we are here only interested in the setting with human specifiers who know (even if only informally) the intended relation, and who are assumed to choose only examples that are consistent with the intended relation. Moreover, the intended relation is assumed to have a recursive algorithm. There is a general consensus that a synthesizer from examples would be a useful component of any larger synthesis system. So we now draw some conclusions about the approaches surveyed in the previous two sections. 

Such applications of NN as a predictive method make the artificial neural networks another technique of data treatment, comparable to parametric empirical modeling by, for example, numerical regression methods [e.g., 10,11] briefly mentioned in section 16.1. The main advantage of NN is that the network needs not be programmed because it learns from sets of experimental data, which results in the possibility of representing even the most complex implicit functions, and also in better modeling without prescribing a functional form of the actual relationship. Another field of... 

The knowledge to be used by the system is formulated in terms of fuzzy inference rules. There are two distinct ways in which these fuzzy rules can be determined (a) by using the experience of human operators and (b) by obtaining them from empirical data found through suitable learning, for example, using neural networks. As stated before, the canonical form of the fuzzy inference rules is... 

NN applications, perhaps more important, is process control. Processes that are poorly understood or ill defined can hardly be simulated by empirical methods. The problem of particular importance for this review is the use of NN in chemical engineering to model nonlinear steady-state solvent extraction processes in extraction columns [112] or in batteries of counter-current mixer-settlers [113]. It has been shown on the example of zirconium/ hafnium separation that the knowledge acquired by the network in the learning process may be used for accurate prediction of the response of dependent process variables to a change of the independent variables in the extraction plant. If implemented in the real process, the NN would alert the operator to deviations from the nominal values and would predict the expected value if no corrective action was taken. As a processing time of a trained NN is short, less than a second, the NN can be used as a real-time sensor [113]. 

These systems provide a useful example because the calculations often work, but occasionally fail, either by distortion from planarity or by failure to locate a stable minimum for one of the tautomers. Thus, the students learn to consider their results critically with a healthy dose of skepticism, to analyze the success or failure of the calculation, to consider the influence of the choice of method (semi-empirical or ab initio), to consider the influence of the choice of basis set, and to determine the answer to the research question posed. 

In section 6.4, you learned several practical methods for determining empirical and molecular formulas of compounds. You may have noticed that these methods work because compounds react in predictable ways. For example, you learned that a compound containing carbon and hydrogen reacts with oxygen to produce water and carbon dioxide. From the mass of the products, you can determine the amount of carbon and hydrogen in the reactant. You also learned that a hydrate decomposes when it is heated to form water and an anhydrous compound. Again, the mass of one of the products of this reaction helps you identify the reactant. In Chapter 7, you will learn more about how to use the information from chemical reactions in order to do quantitative calculations. 

---