Kolmogorov’s theorem

Kolmogorov s Theorem Any real-valued continuous function f defined on an N-dimensional cube can be represented as... 

Kolmogorov s Theorem (Reformulated by Hecht-Nielson) Any real-valued continuous function f defined on an N-dimensional cube can be implemented by a three layered neural network consisting of 2N -)-1 neurons in the hidden layer with transfer functions from the input to the hidden layer and (f> from all of... 

Kolmogorov s theorem thus effectively states that a three-layer net with N 2N -)-1) neurons using continuously increasing nonlinear transfer functions can compute any continuous function of N variables. Unfortunately, the theorem tells us nothing about how to select the required transfer functions or set the weights in our net. 

A detailed discussion of the implications of Kolmogorov s theorem to multilayered nets is given by Kurkova [kurk92]. 

The theorem of Littlewood complements the Kolmogorov s theorem, since it gives information on orbits lying in an open subset of the phase space, but only over a large time. Here is a formal statement. 

Benettin, G., Galgani, L., Giorgilli, A. and Strelcyn, J. M. (1984). A proof of Kolmogorov s theorem on invariant tori using canonical transformations defined by the Lie method. R Nuovo Cimento, 79 201-216. 

Giorgilli, A. and Locatelli, U. (1999). A classical self-contained proof of Kolmogorov s theorem on invariant tori. In Simo, C., editor. Hamiltonian systems with three or more degrees of freedom, NATO ASI series C, 533. Kluwer Academic Publishers, Dordrecht-Boston-London. 

Khrkov, V. (1992). Kolmogorov s theorem and multilayer neural networks. Neural Netw., 5,501-506. 

While Kolmogorov s original theorem has subsequently been improved upon a number of times, the most interesting formulation for our purposes is due to Hecht-Nielson, who rephrased the theorem in the language of neural nets [hecht87]. 

Kolmogorov, A. N., 114,139,159 Konigs thorem applied to Bernoulli method, 81 Koopman, B., 307 Roster, G.F., 727,768 Kraft theorem, 201 Kronig-Penney problem, 726 antiferromagnetic, 747 Krylov-Bogoliubov method, 359 Krylov method, 73 Krylov, N., 322 Kuhn, W. H., 289,292,304 Kuratowski s theorem, 257... 

According to dynamical systems theory, the escape rate is given by the difference (92) between the sum of positive Lyapunov exponents and the Kolmogorov-Sinai entropy. Since the dynamics is Hamiltonian and satisfies Liouville s theorem, the sum of positive Lyapunov exponents is equal to minus the sum of negative ones ... 

A first application of the normal form method is the proof of the theorem of Kolmogorov on the persistence of invariant tori. We outline here the basic scheme proposed in Kolmogorov s original paper. 

R. Hecht-Nielsen, in Proceedings of the IEEE First International Conference on Neural Networks, Vol. Ill, IEEE San Diego, CA, 1987, pp. 11-14. Kolmogorov s Mapping Neural Network Existence Theorem. 

---