Automatic differentiation

More recently, automatic differentiation (AD) techniques have been introduced for the calculation of concentration sensitivities. In general these methods have been more commonly applied in atmospheric and air quality models, possibly because they are often linked to data assimilation methods (Zhang et al. 1998 He et al. 2000 Djouad et al. 2003). The methods provide advantages over the finite-difference-based bmte force method since they can calculate the required derivatives up to machine precision. The methods rely on the fact that any function that is calculated on a computer is basically a sequence of simple operations such as 

Another example of the use of AD is in the calculation of the Jacobian used by the decoupled direct method. AD can provide a more automatic approach compared to using an analytic or symbolic expression for the definition of the Jacobian based on the RHS of the kinetic system of differential equations, but one which is more accurate than defining the Jacobian numerically using finite-difference methods. This approach is implemented in the freely available KPP package for atmospheric chemical simulations (Damian et al. 2002 Sandu et al. 2003 Daescu et al. 2003 KPP). 

---

The Z-scan technique, first introduced in 1989 [64, 65], is a sensitive single-beam technique to determine the nonlinear absorption and nonlinear refraction of materials independently from their fluorescence properties. The simplicity of separating the real and imaginary parts of the nonlinearity, corresponding to nonlinear refraction and absorption processes, makes the Z-scan the most widely used technique to measure these nonlinear properties however, it does not automatically differentiate the physical processes leading to the nonlinear responses. 

For optimization tools linked to procedural models, note that reliable and efficient automatic differentiation tools are available that link to models written, say, in FORTRAN and C, and calculate exact first (and often second) derivatives. Examples of these include ADIFOR, ADOL-C, GRESS, Odyssee, and PADRE. When used with care, these can be applied to existing procedural models and, when linked to modern NLP and MINLP algorithms, can lead to powerful optimization capabilities. More information on these and other automatic differentiation tools can be found at http //www-unix.mcs.anl.gov/ autodiff/ADTools/. 

Currently, the most widely used AD codes are ADIFOR (automatic differentiation of FORTRAN) and ADIC (automatic differentiation of C). These are available... 

Griewank, A., On automatic differentiation, in Mathematical Programming Recent Developments and Applications. (M. Iri and K. Tanebe, eds.). Kluwer Academic Publishers, 1989, 83-108. 

Next we show differentiability. Consider Figure B.l. By construction, the function is surjective. So given an arbitrary element c e S[/(V)/ there is an element A SU(V) such that ttiIA) = c. By Theorem B.3, we know that TTi is a local diffeomorphism. Hence there is a neighborhood TV of A such that TTi I// has a differentiable inverse. The inclusion function is automatically differentiable. Finally, from Theorem B.3 we know that 712 is a differentiable function. Hence the function... 

In solving the underlying model problem, the Jacobian matrix is an iteration matrix used in a modified Newton iteration. Thus it usually doesn t need to be computed too accurately or updated frequently. The Jacobian s role in sensitivity analysis is quite different. Here it is a coefficient in the definition of the sensitivity equations, as is 3f/9a matrix. Thus accurate computation of the sensitivity coefficients depends on accurate evaluation of these coefficient matrices. In general, for chemically reacting flow problems, it is usually difficult and often impractical to derive and program analytic expressions for the derivative matrices. However, advances in automatic-differentiation software are proving valuable for this task [36]. 

A second way consists in calculating the derivatives (d/dXi)E p(p, p ) of the approximated energy Efp(p,p ). This second approach can be subdivided into three methods (d/d i)E s>(p, p ) can be computed (i) by finite differences, (ii) by deriving analytically the discrete equations used for the calculation of E p, p ), (iii) by automatic differentiation [24]. Although (ii) and (iii) are theoretically equivalent, they are not in practice they correspond to two dramatically different implementations of a single mathematical formalism. 

G. Corliss, C. Faure, A. Griewank, L. Hascoet and U. Naumann, (eds), Automatic Differentiation of Algorithms, from Simulation to Optimization, Springer, Heidelberg, (2001). 

Hwang D, Byun DW, Odman MT (1997) An automatic differentiation technique for sensitivity analysis of numerical advection schemes in air quality models. Atmospheric Environment, 31(6) 879-888. 

Isukapalli S, Roy Z, Georgopoulos PG (2000) Efficient sensitivity/uncertainty analysis using the combined stochastic response surface method (SRSM) and automatic differentiation for FORTRAN code (ADIFOR) Application to environmental and biological systems. Risk Analysis, 20 591-602. 

L. B. Rail, Automatic Differentiation—Techniques and Applications, Lecture Notes in Computer Science 120, Springer-Verlag, Berlin/New York, 1981. 

L. C. W. Dixon, SIAM J. Opt., 1,475 (1991). On the Impact of Automatic Differentiation on the Relative Performance of Parallel Truncated Newton and Variable Metric Algorithms. 

The NONLIN module is responsible for intializing the concentration vector, C(t), for l i NRCT. Here NRCT is the number of reactants. If there are no equilibrium reactions, then C i) is set to IC i), the initial concentration vector, for 1 < f < NRCT. If equilibrium reactions do exist, then the type (2) equations (with derivatives set to zero) and the Type (1) and Type (3) equations are all solved simultaneously for the equilibrium concentrations of all reactants. Because the equilibrium equations are generally nonlinear, the Newton-Raphson iteration method is used to solve these equations. Also, since there is no symbol manipulation capability in the current version of CRAMS, numerical differentiation is used to calculate the required partial derivatives. That is, the rate expressions cannot at this time be automatically differentiated by analytical methods. A three point differentiation formula is used 27) ... 

The first term in the product is the previous J derived from the transport model. The second term has previously been tedious to calculate in an optimization routine since it requires a numerical coding of the derivative of the process model. The emergence of automatic differentiation tools (e.g., Giering, 2000) will greatly facilitate the approach. 

The dimension of S no longer appears in the optimization problem. So, provided the procedure has access to J at high resolution, it is possible to avoid some of the problems of aggregating fluxes into large regions. Using automatic differentiation techniques, Kaminski et al. (1999) calculated J for a full transport-model grid (8° X 10°) and a network of a few dozen observation sites. At this resolution, over 20 years, we would need to solve for approximately 200,000 flux components, while the parameter approach may use only hundreds of unknowns. 

The reactivity and selectivity of frans-2-phenyldimethylsilylvinyl-9-BBN is almost comparable with 2-trimethylsilylvinyl-9-BBN [23]. The results are summarized in Scheme 30.2 [22]. The use of this reagent as a dihydroxyethylene equivalent is indirect, and it has the advantage as the latent hydroxyl groups are automatically differentiated, and this differentiation maybe used for further stereochemical elaboration (Chart 30.3). Another advantage of using this re-... 

Averick, B. M., More, J. J., Bischof, C. H., Carle, A., Griewank, A., 1994. Computing large sparse Jacobian matrices using automatic differentiation. SIAM J. Sci. Comput. 

Certain interval methods make use of interval gradients which require derivatives. Automatic Differentiation (AD) is a method for simultaneously computing partial derivatives using a multicomponent object, called gradient, whose algebraic properties incorporate the chain rule of differentiation. The rules, together with the gradient type, form an extended type that we call AD. This type can be used with intervals. 

An object-oriented language for modelling general dynamic process was successfully developed and its usage has proved efficiency in code reusability. The development of model libraries of models for thermodynamics, process engineering and other application areas is one of the future tasks. The DAE index reduction method allows EMSO to directly solve high-index DAE systems without user interaction. This fact combined with the symbolic and automatic differentiation systems and the CAPE-OPEN interfaces leads to a software with several enhancements. 

Luca, L. De and Musmanno, R., 1997, A Parallel Automatic Differentiation Algorithm for Simulation Models, Simulation Practice and Theory, 5,235-252. 

Newton s method or its simplified versions require the knowledge of the Jacobian F (x) at some points. Unfortunately, today s multibody formalisms and programs don t provide this extra information, though at least by applying tools for automatic differentiation this would be possible in principle [Gri89]. Normally, the Jacobian is approximated by finite differences with being the unit... 

Gri89] Griewank A. (1989) On automatic differentiation. In Mathematical Programming Recent Development and Applications pages 83-108. Kluwer. 

---