Pontryagin

Variational calculus, Dreyfus (1962), may be employed to obtain a set of differential equations with certain boundary condition properties, known as the Euler-Lagrange equations. The maximum principle of Pontryagin (1962) can also be applied to provide the same boundary conditions by using a Hamiltonian function. 

Pontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V. and Mishchenko, E.F. (1962) The Mathematical Theory of Optimal Processes, John Wiley Sons, New York. 

The above formulation is a well posed problem in optimal control theory and its solution can be obtained by the application of Pontryagin s Minimum Principle (Sage and White (1977)). 

The first approach to obtain exact time characteristics of Markov processes with nonlinear drift coefficients was proposed in 1933 by Pontryagin, Andronov, and Vitt [19]. This approach allows one to obtain exact values of moments of the first passage time for arbitrary time constant potentials and arbitrary noise intensity moreover, the diffusion coefficient may be nonlinear function of coordinate. The only disadvantage of this method is that it requires an artificial introducing of absorbing boundaries, which change the process of diffusion in real smooth potentials. 

We will not present here how to derive the first Pontryagin s equation for the probability Q(t, x0) or P(f,x0). The interested reader can see it in Ref. 19 or in Refs. 15 and 18. We only mention that the first Pontryagin s equation may be obtained either via transformation of the backward Kolmogorov equation (2.7) or by simple decomposition of the probability P(t, xq) into Taylor expansion in the vicinity of xo at different moments t and t + t, some transformations and limiting transition to r — 0 [18]. 

One can obtain an exact analytic solution to the first Pontryagin equation only in a few simple cases. That is why in practice one is restricted by the calculation of moments of the first passage time of absorbing boundaries, and, in particular, by the mean and the variance of the first passage time. 

Equation (4.15) was first obtained by Pontryagin and is called the second Pontryagin equation. 

Pontryagin LS (1962) The mathematical theory of optimal processes. Interscience, New York... 

Pontryagin s method can be applied to a number of different cases, but only the form needed for comparison will be given here. Consider an n-dimensional state space vector x = (jc. .. "), with r control variables u = (w1,... ur), related by... 

Pontryagin s adjoint variables i/j are clearly the partial derivatives of Bellman s F and the continuity of the adjoint variables (as solutions of adjoint differential equations) implies the smoothness of the surface that was lacking in the first paper. 

Studies in optimization-VII The application of Pontryagin s methods to the control of batch and tubular reactors (with C.D. Siebenthal). Chem. Eng. ScL 19,747-761 (1964). 

Here the control set U consists of functions (control signals) able to move the system from the CA to the SC. The Pontryagin Hamiltonian (34) and the corresponding equations of motion take the form... 

However, using a method proposed [60,62,95,112] for experimental analysis of the Hamiltonian flow in an extended phase space of the fluctuating system, we can exploit the analogy between the Wentzel-Freidlin and Pontryagin Hamiltonians arising in the analysis of fluctuations, and the energy-optimal control problem in a nonlinear oscillator. To see how this can be done, let us consider the fluctuational dynamics of the nonlinear oscillator (35). 

In principle, it is possible to find the optimal path by direct solution of the Pontryagin Hamiltonian (37), with appropriate boundary conditions. We must stress that even for this relatively simple system, the solution is a formidable, and almost impossible, task. First of all, in general one has no insight into the appropriate boundary conditions, in particular into those at the starting time (which belong to the strange attractor). But even if the boundaries were known, in practice the determination of the optimal path is impossible the functional R of Eq. (36) has so many local minima, that it proved impractical to attempt a (general) search for the optimal path. 

L.S. Pontryagin. Obyknovennye differentsial nye uravneniya- Moskwa Nauka, 1982. 

Depending on the numerical techniques available for solving optimal control or optimisation problems the model reformulation or development of simplified version of the original model was always the first step. In the Sixties and Seventies simplified models represented by a set of Ordinary Differential Equations (ODEs) were developed. The explicit Euler or Runge-Kutta methods (Huckaba and Danly, 1960 Domenech and Enjalbert, 1981) were used to integrate the model equations and the Pontryagin s Maximum Principle was used to obtain optimal operation policies (Coward, 1967 Robinson, 1969, 1970 etc.). 

The Maximum Principle of Pontryagin (Pontryagin et al., 1964) may be stated briefly as follows. For a given set of ODEs ... 

---