The Adjoint Method

This is popularly referred to as the adjoint of although it seems something of an abuse of mathematical language to refer to it in this way. For the flow map we know that the inverse map is precisely. -h, so = h, i-e. the flow map is in the normal sense self-adjoint, i.e. symmetric. However, such a property does not hold in the general case. In particular, consider Euler s method 

The method (2.18)-(2.19) is called the Symplectic Euler method. Its adjoint method has a similar structure  

Given a method with adjoint method it is possible to obtain the adjoint of the adjoint method but, as we might expect, the adjoint of the adjoint is the 

---

Results from this model were verified by Neupauer and Wilson [51] using the adjoint method. In this method, the forward governing equation, with concentration as the dependent variable, is replaced by the adjoint equation, with the adjoint state as the dependent variable. They showed that backward-in-time location and travel time probabilities are adjoint states of the forward-in-time resident concentration. In this and the follow-up paper, Neupauer and Wilson [51,52] presented the adjoint method as a formal framework for obtaining the backward-in-time probabilities for multidimensional problems and more complex domain geometries. 

Two widely used practical inverse approaches are presented in the next two sections ( Estimating absolute velocities and nutrient fluxes across sections and Estimating carbon export fluxes with the adjoint method ). These serve as examples to describe details of the mathematical methods and to list achievable results. The first method, the section inverse approach, infers nutrient, carbon, and tracer fluxes across sets of sections, based on hydro-graphic, tracer, and nutrient data along these sections. The second example describes an application of the adjoint method for the determination of ocean currents, biological productivity, and downward particle fluxes. This method is specifically adapted for the utilization of many different tracers and can handle problems with heterogeneous and sparse data coverage. 

Estimating Carbon Export Fiuxes with the Adjoint Method... 

Recall that the Stormer-Verlet method could be constructed by composing steps using Symplectic Euler and its adjoint method. Using more complicated methods it is possible to build higher order schemes. It seems natural that a similar procedure should be possible in the constrained setting. But what, precisely, is the adjoint method in the case of (4.20)-(4.24) ... 

Recall that, for a given method Qh the definition of the adjoint method Ql is given... 

Loulou, T., and Scott, E. P., 2000 2-D Thermal Dose Optimization in High Intensity Focused Ultrasound Treatments Using the Adjoint Method, ASME HTD—Vol. 368/BED—Vol. 47, Advances in Heat and Mass Transfer in Biotechnology—2(X)0, pp. 67—71. 

Response sensitivity computation can be performed using different methods, such as the forward/backward/central Finite Difference Method (FDM) (Kleiber et al. 1997, Conte et al. 2003, 2004), the Adjoint Method (AM) (Kleiber et al. 1997), the Perturbation Method (PM) (Kleiber Hien 1992), and the Direct Differentiation Method... 

I think the first step in this is sensitivity analysis, for which there has been a lot of work in past decade. The forward method of sensitivity analysis for differential equations has been thoroughly investigated, and software is available. The adjoint method is a much more powerful method of sensitivity analysis for some problems, but it is more difficult to implement, although good progress has been made. Next it moved to PDEs, where progress is being made for boundary conditions and adaptive meshes. 

When the number of unknowns is large, as for example in the case that all grid block property values are regarded as unknown parameters, then it is impractical to find the derivatives of the objective function. By treating the fluid flow model as a constraint, the adjoint method can be used. This is, conceptually, the same adjoint method as described in Section 4.5. The method exploits the fact that the dot product of the gradient with a small number of fixed vectors can be obtained at relatively low cost. However, this means that a lower order optimisation method, such as the conjugate gradient technique must be used. This is why for small numbers of unknowns the direct technique is best. 

The described method can generate a first-order backward or a first-order forward difference scheme depending whether 0 = 0 or 0 = 1 is used. For 9 = 0.5, the method yields a second order accurate central difference scheme, however, other considerations such as the stability of numerical calculations should be taken into account. Stability analysis for this class of time stepping methods can only be carried out for simple cases where the coefficient matrix in Equation (2.106) is symmetric and positive-definite (i.e. self-adjoint problems Zienkiewicz and Taylor, 1994). Obviously, this will not be the case in most types of engineering flow problems. In practice, therefore, selection of appropriate values of 6 and time increment At is usually based on trial and error. Factors such as the nature of non-linearity of physical parameters and the type of elements used in the spatial discretization usually influence the selection of the values of 0 and At in a problem. 

We thus have shown that under conditions (26) inequality (14) is sufficient for the stability of scheme (la) in the space Ha, that is, relation (29) occurs. Let us stress that the requirement of self-adjointness of the operator B is necessary here, while the energy method demands only the positivity of B and no more. 

It was reported that the convergence of the Krotov iteration method [81, 93] was four or five times faster than that of the gradient-type methods. The formulation of Rabitz and others, [44, 45, 92], designed to improve the convergence of the above algorithm, introduces a further nonlinear propagation step into the adjoint equation (i.e., the equation for the undetermined Lagrange multiplier % t)) and is expressed as... 

Gradient calculations for the x variables are obtained from implicit reformulations of the DAE model. Clearly the easiest, but least accurate, way is simply to re-solve the model for each perturbation of the parameters. Sargent and Sullivan (1977, 1979) derived these gradients using an adjoint formulation. In addition, they were able to accelerate the adjoint computations by retaining the information from the model solution (the forward step) for the adjoint solution in the backward step. This approach was later refined for variable stepsize methods by Morison (1984). The adjoint approach to parameterized optimal control was also used by Jang et al. (1987) and Goh and Teo (1988). 

Relation to the variational method. As we remarked in Introduction, we base our theory on the variational method in its generalized form.57) It will be convenient to give here a sketch of this relation. If FI is a self-adjoint operator for which we are to solve the eigenvalue problem, the cited variational method consists in making the quantity... 

On the other hand, the trajectory sensitivity equations method requires simultaneous integration of a greater number of equations than the adjoint system approach. However, it is more stable than the adjoint system approach due to the requirement of forward integration only. It is usually preferred in the area of parameter estimation and sensitivity (Kalogerakis and Luus, 1983 Caracotsios and... 

While Huang and Ozisik solved the spacewise variation of wall heat flux for laminar forced convection problem, Silva Neto and Ozisik [57] used the conjugate gradient method and the adjoint equation simultaneously to solve for the timewise-varying strength of a two plane heat source. 

In the section of this chapter describing the Kirchhoff inversion method and general non-linear inversion techniques, we have demonstrated that the calculation of the Kirchhoff adjoint operator (15.142) and the adjoint Frechet derivative operator... 

We have used sensitivity equation methods (Leis and Kramer, 1985) for gradient evaluation as these are simple and efficient for problems with few parameters and constraints. In general, the balance in efficiency between sensitivity and adjoint methods depends on the type of problem being addressed. Adjoint methods are particularly advantageous for optimal control problems in which the inputs are represented as a large number of piecewise constant input values and few interior point constraints exist. Sensitivity methods are preferable for problems with few parameters and many constraints. 

There are several types of matrix operations that are used in the MCSCF method. The transpose of a matrix A is denoted A and is defined by (A )ij = Xji. The identity (AB) = B A is sometimes useful where AB implies the usual definition of the product of matrices. A vector, specifically a column vector unless otherwise noted, is a special case of a matrix. A matrix-vector product, as in Eq. (5), is a special case of a matrix product. The conjugate of a matrix is written A and is defined by (A )jj = (A,j). The adjoint, written as A is defined by A = (A ) . The inverse of a square matrix, written as A , satisfies the relation A(A = 1 where = du is called the identity or unit matrix. The inverse of a matrix product satisfies the relation (AB) =B" A" . A particular type of matrix is a diagonal matrix D, where D,y = y, and is sometimes written D = diag(dj, d2> ) or as D = diag(d). The unit matrix is an example of a diagonal matrix. 

The notation u = (u u ), v = (v v ) has been used. These simplifications are easy to introduce in any method based upon the utilization of cofactors. This is true whether the cofactors are computed directly, or indirectly as elements of the adjoint matrix of the overlap —provided only that the block diagonal form of is fully exploited. 

---