State and costate equations

The improvement in control functions causes the objective functional value to get closer to the optimum. Therefore, iterative application of the above steps leads to the optimum, i. e., the optimal functional value, and the corresponding optimal control functions. While the state and costate equations are satisfied in each iteration, the variational derivatives are reduced successively. They vanish when the optimum is attained. 

In the gradient method, the state and costate equations are solved using initial guesses for the control u and the final time tf. The guessed u and tf are then improved using, respectively,... 

With the state and costate equations satisfied at the end of Step 3 above, the variation of the augmented objective functional becomes [see Equation (6.9), p. 156]... 

These improvements are done in Step 4 of Section 7.1.2 (p. 186) where the state and costate equations are satisfied so that... 

Some times the stationarity condition = 0 of an optimal control problem can be solved to obtain an explicit expression for u in terms of the state y and the costate A. That expression, when substituted into state and costate equations, couples them. Thus, state equations become dependent on A and must be integrated simultaneously with the costate equations. The simultaneous integration constitutes a two point boundary value problem in which... 

The shooting Newton-Raphson method enables the solution of this problem. With a guessed initial costate, both state and costate equations are integrated forward or shot to the final time. The discrepancy between the final costate obtained in this way and that specified is improved using the Newton-Raphson... 

Suppose that it is possible to solve Hu = 0 and obtain an explicit expression, u = u y, A). Utilizing this expression, the state and costate equations for the minimum turn into the two point boundary value problem... 

Differentiating the state and costate equations with respect to Aq, we get d fdy dg d dh... 

The last two equations arise from differentiating with respect to Aq the initial conditions for the state and costate equations, which are, respectively,... 

The shooting Newton Raphson method will use the derivative A o at t[ to improve the guess A(0) = Aq, thereby zeroing out A(tf). We differentiate with respect to Aq the latest state and costate equations as well as the initial boundary conditions... 

The solution of a optimal periodic control problem requires the integration of state and costate equations, both subject to periodicity conditions. Other than this integration aspect, the solution methods for optimal periodic control problems are similar to those for non-periodic problems. Therefore, we will focus on the methods to integrate state and costate equations under periodicity conditions. 

A periodicity condition implies that the initial and final values of a state (or costate) variable are equal to a single value. Thus, in a optimal periodic control problem, the set of state as well as costate equations poses a two point boundary value problem. Either successive substitution or the shooting Newton-Raphson method may be used to integrate the periodic state and costate equations. 

In order to integrate the state and costate equations satisfying the periodicity conditions, we need the respective derivative differential equations for the shooting Newton-Raphson method. 

The integration of the derivative state and costate equations provides the derivative values at the endpoint Newton-Raphson method [see Equations (8.1) and (8.2)]. 

At this point, we introduce the Hamiltonian description of the system and corresponding form of the optimum theorem. This is done firstly because the Hamiltonian is a concise way to express the state and costate equations. But, more than this conciseness, it turns out that the Hamilton density itself has an interesting and useful property in the optimum system. 

Then we have the classical result that the state and costate equations can be obtained from... 

This is a simple but slow method in which a set of state (or costate) equations is integrated assuming the initial conditions. The final conditions obtained from integration are then substituted for the initial conditions in the next round of integration. This procedure is repeated until the initial and final... 

If the adjoint function satisfies these equations and boundary conditions, Lis a stationary expression, insensitive to small errors in the density, whose numerical value will yield C. Inspection shows that the Lagrangian has a certain symmetry such that, if N satisfies its equation and boundary conditions, then the Lagrangian is stationary to errors in iV (stationary, in fact, to large errors, since /o and M are not functions of the costate variable). In practice, both equations are perturbed by a change in the control variable and simultaneous errors are made in both functions. For small control perturbations, we anticipate small perturbations in the state and costate variables and that the resulting expression is in error in the cost function only through terms involving the product of small errors. We write... 

Some work on distributed reactors and their optimum control has been undertaken by means of an expansion in orthogonal modes. If the expansion is terminated and a finite number of terms used, the model reduces essentially to a system of ordinary differential equations with an increased number of elements in the state and costate vectors. However, we shall give a more general account. At the same time, this general account can meaningfully be reduced to the steady state for certain problems of interest. Another motive for this section is to demonstrate a connection between Pontryagin s work and some well-established results of reactor theory. 

Integrate the costate equations backward to t = 0 using the final conditions at the final time, the control functions, and the state determined in the previous step. 

Integrate costate equations backward using the final conditions, the control function values, and the saved values of the state variables. Save the values of costate variables at each grid point. 

Integrate costate equations backward from <7 = 1 to 0 using the final conditions, the controls n s, and the state variables y s. Save the values... 

In the previous discussion, it will perhaps have become apparent that the generalized Lagrange multiplier or adjoint function plays a significant role in the theory of optimal processes. Furthermore, it becomes as necessary to solve the adjoint or costate equations as the state equations if we are to analyze or synthesize optimal systems. We have also noted that the adjoint functions appear in the Lagrangian as a weighting given to the source density 5. In this section, we shall take up this idea to develop a physical interpretation of the adjoint function which should help us understand its role and perhaps find the adjoint equations, boundary conditions, and even solutions more easily. This physical interpretation as an importance function follows closely the interpretation given to the adjoint function in reactor theory 54). 

Therefore, we solve the adjoint equations (equation (7.7.2)) starting at tf and going to toi times. Each solution starts with an assumed final costate condition of which has a zero for all elements except the element, which is unity. Each f unity element corresponds to a known state boundary condition in X . 

---