Spurious Attractors

While we have shown that the Hebb rule (equation 10.19) yields the desired dynamical attractors at tlu local minima of the energy function (equation 10.9), we have not shown that the.,se attractors arc in fact the only ones possible in this system. In fact, they are not, and spurious attractors are also possible. 

It is obvious, for example, that a rev( rsal all of the neuronal values for a given state, Si —Si, leaves the energy unchanged and therefore also represents a local minimum. There are also stable mixed states that are not equal to any of the stored patterns but are linear combinations of an odd number of those patterns [amit85a). For example, it is easy to show that any symmetric combination of three stored patterns is a stable state  

Finally, if there are a large number of stored patterns, there may enough mutual interference to give rise to local minima, or inetastable states - sometimes also called spin glass states to emphasize the similarity between the formalisms of Hopfield nets and spin glasses - that are not correlated with any subset of the set of stored patterns [amitSSb]. 

---

The basic backpiopagation algorithm described above is, in practice, often very slow to converge. Moreover, just as Hopfield nets can sometimes get stuck in undesired spurious attractor states (i.e. local minima see section 10.6.5), so to can multilayer perceptrons get trapped in some undesired local minimum state. This is an unfortunate artifact that plagues all energy (or cost-function) minimization schemes. 

---