Lagrange Multiplier Theorem

In Section 3.2.1 (p. 59), we had asserted the Lagrange Multiplier Rule that the optimum of the augmented J is equivalent to the constrained optimum of I. This rule is based on the Lagrange Multiplier Theorem, which provides the necessary conditions for the constrained optimum. We will first prove this theorem and then apply it to optimal control problems subject to different types of constraints. 

The theorem states that if a functional I y) subject to the constraint 

There exists a constant multiplier A such that 

The provision that SK y 6z) 0 along at least one variation is called the normality condition or the constraint qualification. 

Both / and if in a general case are functionals of a function y depending on an independent variable. 

---

Figure 4.1 Outline of the proof for the Lagrange Multiplier Theorem...

This rule is based on the Lagrange Multiplier Theorem. Consider the augmented functional... 

According to the Lagrange Multiplier Theorem, Equations (4.2) and (4.3) are the necessary conditions for the optimum of I subject to the following preconditions ... 

Before considering the applications of the Lagrange Multiplier Theorem, it is worthwhile to generalize the theorem to handle several equahty constraints and functions. 

Lagrange Multiplier Theorem for Several Equality Constraints... 

Step 3 The augmented objective functional is M = J + pK where is a Lagrange multiplier. From the Lagrange Multiplier Theorem, assuming that... 

In this case, the Lagrange Multiplier Theorem yields 6M = 0, which is the necessary condition for the minimum. 

Erom the previous two theorems, any stationary point of. /(p) yields the maximum of. /(p). Such a stationary point can often be found by using Lagrange multipliers or by using the symmetry of the channel. In many cases, a numerical evaluation of capacity is more convenient in these cases, convexity is even more useful, since it guarantees that any reasonable numerical procedure that varies p to increase. /(p) must converge to capacity. 

Lagrange multipliers 255-256 Lagrange s moan-value theorem 30-32 Lagperre polynomials 140, 360 Lambert s law 11 Langevin function 61n Laplace transforms 279—286 convolution 283-284 delta function 285 derivative of a function 281-282 differential equation solutions 282-283... 

Remark 3 If the primal problem (P) has an optimal solution and it is stable, then using the theorem of existence of optimal multipliers (see section 4.1.4), we have an alternative interpretation of the optimal solution (A, p) of the dual problem (D) that (A, p) are the optimal Lagrange multipliers of the primal problem (P). 

This is a statement of Brillouin s theorem [37], that (a H i) = 0, i < N < a is a necessary condition for (4> 7/ d>) to be stationary. The normalization of occupied variables must also be varied in order to determine the Lagrange multipliers c,. Definition of the effective Hamiltonian H requires diagonal matrix elements determined by SE/niScj) for unconstrained variations 8(p,. 

Hurley proposed a simple, sufficient condition for the applicability of the Hellmann-Feynman theorem. " If within a variational framework, the family of trial functions is invariant to changes in parameter a, then the Hellmann-Feynman theorem is satisfied by the optimum trial function. In variational approaches involving Lagrange multipliers, for example, in the Hartree-Fock and multiconfigurational self-consistent field methods, Hurley s condition is fulfilled. ... 

In summary, condition 1 gives a set of n algebraic equations, and conditions 2 and 3 give a set of m constraint equations. The inequality constraints are converted to equalities using h slack variables. A total of M + m constraint equations are solved for n variables and m Lagrange multipliers that must satisfy the constraint qualification. Condition 4 determines the value of the h slack variables. This theorem gives an indirect problem in which a set of algebraic equations is solved for the optimum of a constrained optimization problem. 

Then the envelope of the spectrum is specified by the first My constraints and the additional, M — A/, constraints determine the finer structure seen only at resolution better than Ath. This is seen by coarse graining the spectrum (Eq. (62)) over At t. As discussed in Sec. II, the convolution theorem implies that the corresponding Fourier transform satisfies C(tr) = 0, r > My. This can also be seen directly from Eq. (67). Hence the Lagrange multipliers, Eq. (76) satisfy... 

Therefore we obtain, as the result of I-Shih Liu Theorem A.5.5, the five Lagrange multipliers A1, Af = 1 These are in our example... 

In this chapter, we introduce the concept of Lagrange multipliers. We show how the Lagrange Multiplier Rule and the John Multiplier Theorem help us handle the equality and inequality constraints in optimal control problems. 

In several plaees in this book, we need to use similar proofs with Lagrange multipliers. This is why we will demonstrate how to prove the same theorem using this technique (see Appendix N available at booksite.elsevier.com/978-0-444-59436-5 on p. el21). 

---