Growing neural network

Dopazo, J., and Carazo, J. M. (1997). Phylogenetic reconstruction using an unsupervised growing neural network that adopts the topology of a phylogenetic tree. J. Mol Evol, 44 226-233. 

Herrero, J., Valencia, A. and Dopazo, J. (2001). A hierarchical unsupervised growing neural network for clustering gene expression patterns. Bioinformatics, 17(2) 126-136. 

The structural unit of artificial neural networks is the neuron, an abstraction of the biological neuron a typical biological neuron is shown in Fig. 44.1. Biological neurons consist of a cell body from which many branches (dendrites and axon) grow in various directions. Impulses (external or from other neurons) are received through the dendrites. In the cell body, these signals are sifted and integrated. 

Overfitting arises when the network learns for too long. For most students, the longer they are trained the more they learn, but artificial neural networks are different. Since networks grow neither bored nor tired, it is a little odd that their performance can begin to degrade if training is excessive. To understand this apparent paradox, we need to consider how a neural network learns. 

A growing neural gas has an irregular structure. A running total is maintained of the local error at each unit, which is calculated as the absolute difference between the sample pattern and the unit weights when the unit wins the competition to match a sample pattern. Periodically, a new unit is added close to the one that has accumulated the greatest error, and the error at the neighbors to this node share their error with it. The aim is to generate a network in which the errors at all units are approximately equal. 

At present, it appears that the most productive types of constructive clustering in the physical and life sciences will be the growing neural gas and the GCS methods in this chapter we focus on the latter. Although this method has notable advantages over the SOM, scientific applications of the GCS have only recently started to appear. There is a little more to the method than a SOM because of the need to grow the network as well as train it, but lack of familiarity with the technique rather than a lack of power explains the present paucity of applications in science because GCSs have nearly all the advantages of the SOM with few of the drawbacks. 

Includes an introduction to artificial intelligence, artificial neural networks, self-organizing maps, and growing cell structures... 

Just as there are several varieties of evolutionary algorithm, so the neural network is available in several flavors. We shall consider feedforward networks and, briefly, Kohonen networks and growing cell structures, but Hop-field networks, which we shall not cover in this chapter, also find some application in science.31... 

Fritzke, B. (1994) Growing cell structures-a self-organizing network for unsupervised and supervised learning. Neural Networks 7 1441-1460... 

The field of artificial neural networks is a new and rapidly growing field and, as such, is susceptible to problems with naming conventions. In this book, a perceptron is defined as a two-layer network of simple artificial neurons of the type described in Chapter 2. The term perceptron is sometimes used in the literature to refer to the artificial neurons themselves. Perceptrons have been around for decades (McCulloch Pitts, 1943) and were the basis of much theoretical and practical work, especially in the 1960s. Rosenblatt coined the term perceptron (Rosenblatt, 1958). Unfortunately little work was done with perceptrons for quite some time after it was realized that they could be used for only a restricted range of linearly separable problems (Minsky Papert, 1969). 

We could grow bitek neural networks which you could inhabit. You would be able to receive the full range of human senses. After that they could be placed in artificial bodies, rather like a cosmonik."... 

---