Minsky

M. Minsky, SemanticInformation Processing, MIT Press, Gambridge, Mass., 1968. 

A universal Turing machine uses an arbitrarily long tape as a potentially infinite memory storage device. Instead, for his proof, Conway used Minsky s idea that a potentially infinite memory can also be obtained by storing arbitrarily large numbers in memory registers. The idea is sketched in figure 3.85. 

To say that Minsky and Papert s stinging, but not wholly undeserved, criticism of the capabilities of simple perceptrons was taken hard by perceptron researchers, would be an understatement. They were completely correct in their assessment of the limited abilities of simple perceptrons and they were correct in pointing out that XOR-like problems require perceptrons with more than one decision layer. Where Minsky and Papert erred - and erred strongly - was in their conclusion that since no learning rule for multi-layered nets was then known and will never be found, perceptrons represent a dead end field of research. ... 

While, as mentioned at the close of the last section, it took more than 15 years following Minsky and Papert s criticism of simple perceptrons for a bona-fide multilayered variant to finally emerge (see Multi-layeved Perceptrons below), the man most responsible for bringing respectability back to neural net research was the physicist John J, Hopfield, with the publication of his landmark 1982 paper entitled Neural networks and physical systems with emergent collective computational abilities [hopf82]. To set the stage for our discussion of Hopfield nets, we first pause to introduce the notion of associative memory. 

In a contribution to the first conference on physics and computation held at MIT in 1982 [land82b], Minsky considered whether a universe satisfying the following two conditions could exist [minsky82] (1) each volume of space contains a finite amount of information, and (2) over a suitable range of size and speed, the mechanics of the universe are approximately classical. 

Envisioning space-time as a four-dimensional CA lattice, wherein sites take on one of a finite number of values and interact via a local dynamics, Minsky explored various elementary properties of this universe particle (or packet ) size and speed, time contraction, symmetry, and how the notion of field might be made palatable within such a framework. 

The idea that localized partic le-like propagating structures can be defined on a lattice Wcus nothing new. For example, Minsky was well aware of the existence of gliders in Conway s Life rule. Minsky s own pedagogical example was effectively a four-state one-dimensional CA with states a e 0,1,a,/ and rules 4> (cri i,CTi,cri+i) —cr given by ... 

Notice that in this example, the speed of the packet is inversely proportional to the packet s spatial size. While there is certainly nothing unique about this particular representation, it is interesting to speculate, along with Minsky, whether it may be true that, just as the simultaneous information about position and momentum is fundamentally constrained by Heisenberg s uncertainty relation in the physical universe, so too, in a discrete CA universe, there might be a fundamental constraint between the volume of a given packet and the amount of information that can be encoded within it. 

How might the interaction between two discrete particles be described by a finite-information based physics Unlike classical mechanics, in which a collision redistributes the particles momentum, or quantum mechanics, which effectively distributes their probability amplitudes, finite physics presumably distributes the two particles information content. How can we make sense of the process A scatters J5, if B s momentum information is dispersed halfway across the galaxy [minsky82]. Minsky s answer is that the universe must do some careful bookkeeping, ... 

If a particle A must know B s total information content before colliding, the collision process must be delayed until A has full access to that information. However, such a delay is consistent neither with classical nor quantum mechanics, Minsky instead suggests that the collision proceeds immediately, but with the particles both working with less than all the information that is classically required i.e, the incoming particles momenta are estimated. Outgoing momenta are determined via conventional classical rules, but, because of the estimation errors, each scattered particle leaves behind a receipt recording how much momentum was really taken away in the process. Receipts not only mark prospective event-locations at which future collisions might take place, but harbor information that can be used to estimate new real momenta. 

We will not go any further into the interesting speculations Minsky offers in his 1982 paper [minsky82]. The point of these speculations was not to propose a serious alternative model of fundamental physics, per se, but to stimulate thinking along the. lines of What if physics were, fundamentally, discrete How would we describe the processes we now think we iinderstaiul with our continuous models Two questions that we will repeatedly come back to in this concluding chapter. 

IZuse does not consider the complicated question of how the integrity of such clusters can be maintained by a rule, or set of rules, operating on a random lattice. Minsky [minsky82] also considers using a random lattice but wonders how to build particles into such a universe that would necessMily have to be insensitive to local cell-connection fluctuations. 

We have already seen hints of the kinds of theories that are possible using the first axiom see Kantor s Information Mechanics (section 12.4.5), Stonier s Information Physics (section 12.4.6) and Frieden s Extreme Physics Information Principle (section 12.4.7). Minsky, Zuse and Fredkiii, among many others, have speculated on a CA-like dynamics of primitive information (see section 12.6). 

Eairweather-Tait, S. J., Porhvood, D. E., Symss, L. L., Eagles, J., and Minski, M. J. (1989). Iron and zinc absorption in human subjects from a mixed meal of extruded and nonextruded wheat bran and flour. Am. ]. Clin. Nutr. 49, 151-155. 

---