Optimization method

Numerical optimization is perhaps one of the most dazzling issues in all the sciences and engineering. Essentially all modem computational codes. 

In spite of the broad use of optimization, from fundamental sciences to practical engineering design, there is no single numerical method that can guarantee a solution. There are a variety of methods available, each one capable of solving a class of problems, and the pursuit of new methods is still an active area of research. The interested reader is referred to textbooks, such as Refs. [6,7], to get a comprehensive view of the methods available and learn the theory behind the different techniques. A brief overview of some of these methods, in the context of parameter estimation in chemical kinetics, is presented in Ref. [8]. 

A brief description of optimizations methods will be given (also see refs. 41-44). In contrast to other fields, in computational chemistry great effort is given to reduce the number of function evaluations since that part of the calculation is so much more time consuming. Since first derivatives are now available for almost all ab initio methods, the discussion will focus on methods where first derivatives are available. The most efficient methods, called variable metric or quasi-Newton methods, require an approximate matrix of second derivatives that can be updated with new information during the course of the optimization. Some of the more common methods have different equations for updating the second derivative matrix (also called the Hessian matrix). 

The PES, a function of 3N cartesian coordinates or 3N - 6 internal coordinates (3N - 5 for linear molecule) can be expanded about an arbitrary point such that the energy at a displaced point x is given by the Taylor series 

If the series is truncated after the quadric term, in vector notation, the series becomes 

At a stationary point dE/dx = 0 and x = -Fq i-go or x = -Hp-go where Hq is the approximate inverse Hessian matrix Fo In subsequent steps Hq is updated by an updating formula to yield a new Hq. 

Eigenvalue Following (Augmented Hessian or Restricted Step) ° 

Note You cannot use the Extended Hiickel method or any one of the SCFmethods with the Cl option being turned on for geometry optimizations, molecular dynamics simulations or vibrational calculations, in the current version of HyperChem. 

Note You can not use the Extended Hiickel method, nor any of the other SCFmethods with the Cl option turned on, for geometry optimization or molecular dynamics simulations. 

You can use any ab initio SCF calculation and all the semi-empiri-cal methods, except Extended Hiickel, for molecular dynamics simulations. The procedures and considerations are similar for simulations using molecular mechanics methods (see Molecular Dynamics on page 69). 

In order to conserve the total energy in molecular dynamics calculations using semi-empirical methods, the gradient needs to be very accurate. Although the gradient is calculated analytically, it is a function of wavefunction, so its accuracy depends on that of the wavefunction. Tests for CH4 show that the convergence limit needs to be at most le-6 for CNDO and INDO and le-7 for MINDO/3, MNDO, AMI, and PM3 for accurate energy conservation. ZINDO/S is not suitable for molecular dynamics calculations. 

A vibrations calculation is the first step of a vibrational analysis. It involves the time consuming step of evaluating the Hessian matrix (the second derivatives of the energy with respect to atomic Cartesian coordinates) and diagonalizing it to determine normal modes and harmonic frequencies. For the SCFmethods the Hessian matrix is evaluated by finite difference of analytic gradients, so the time required quickly grows with system size. 

---

In practice, it is nontrivial to manually select the parameter values to obtain successful results. Moreover, it is not obvious how to measure the quality of the results. Therefore, a well defined performance measure and an efficient parameter optimization method are desired. 

In this section, we will discuss general optimization methods. Our example is the geometry optimization problem, i.e., the minimization of (q). However, the results apply to electronic optimization as well. There are a number of usefiil monographs on the minimization of continuous, differentiable fimctions m many variables [6, 7]. 

The simplest smooth fiuictioii which has a local miiiimum is a quadratic. Such a function has only one, easily detemiinable stationary point. It is thus not surprising that most optimization methods try to model the unknown fiuictioii with a local quadratic approximation, in the fomi of equation (B3.5.1). 

Most gradient optimization methods rely on a quadratic model of the potential surface. The minimum condition for the... 

A very pedagogical, highly readable introduction to quasi-Newton optimization methods. It includes a modular system of algoritlnns in pseudo-code which should be easy to translate to popular progrannning languages like C or Fortran. 

Concomitantly with the increase in hardware capabilities, better software techniques will have to be developed. It will pay us to continue to learn how nature tackles problems. Artificial neural networks are a far cry away from the capabilities of the human brain. There is a lot of room left from the information processing of the human brain in order to develop more powerful artificial neural networks. Nature has developed over millions of years efficient optimization methods for adapting to changes in the environment. The development of evolutionary and genetic algorithms will continue. 

Unconstrained optimization methods [W. II. Press, et. ah, Numerical Recipes The An of Scieniific Compulime.. Cambridge University Press, 1 9H6. Chapter 101 can use values of only the objective function, or of first derivatives of the objective function. second derivatives of the objective function, etc. llyperChem uses first derivative information and, in the Block Diagonal Newton-Raphson case, second derivatives for one atom at a time. TlyperChem does not use optimizers that compute the full set of second derivatives (th e Hessian ) because it is im practical to store the Hessian for mac-romoleciiles with thousands of atoms. A future release may make explicit-Hessian meth oils available for smaller molecules but at this release only methods that store the first derivative information, or the second derivatives of a single atom, are used. 

Schlick T 1992. Optimization Methods in Computational Chemistry. In Lipkowitz K B and D B Boyd (Editors) Reviews in Computational Chemistry Volume 3. New York. VCH Publishers, pp. 1-71. 

Numerous optimization methods aim at approximating the Hessian (or its inverse) in various ways. 

An efficient optimization method that allows several factors to be optimized at the same time. 

In many cases an optimized method may produce excellent results in the laboratory developing the method, but poor results in other laboratories. This is not surprising since a method is often optimized by a single analyst under an ideal set of conditions, in which the sources of reagents, equipment, and instrumentation remain the same for each trial. The procedure might also be influenced by environmental factors, such as the temperature or relative humidity in the laboratory, whose levels are not specified in the procedure and which may differ between laboratories. Finally, when optimizing a method the analyst usually takes particular care to perform the analysis in exactly the same way during every trial. 

The following texts and articles provide an excellent discussion of optimization methods based on searching algorithms and mathematical modeling, including a discussion of the relevant calculations. 

Optimization Methods for Chemists. VCH Publishers Deerfield Beach, FL, 1986. 

Reklaitis, G. V, A. Ravindran, and K. M. RagsdeU. Engineering Optimization Methods and Applications. Wiley, New York (1983). 

FIG. 3-54 Compl ex method, a pattern search optimization method. 

The development of micellar liquid chromatography and accumulation of numerous experimental data have given rise to the theory of chromatographic retention and optimization methods of mobile phase composition. This task has had some problems because the presence of micelles in mobile phase and its modification by organic solvent provides a great variety of solutes interactions. 

Alternative algorithms employ global optimization methods such as simulated annealing that can explore the set of all possible reaction pathways [35]. In the MaxFlux method it is helpful to vary the value of [3 (temperamre) that appears in the differential cost function from an initially low [3 (high temperature), where the effective surface is smooth, to a high [3 (the reaction temperature of interest), where the reaction surface is more rugged. 

Finding the minimum of the hybrid energy function is very complex. Similar to the protein folding problem, the number of degrees of freedom is far too large to allow a complete systematic search in all variables. Systematic search methods need to reduce the problem to a few degrees of freedom (see, e.g.. Ref. 30). Conformations of the molecule that satisfy the experimental bounds are therefore usually calculated with metric matrix distance geometry methods followed by optimization or by optimization methods alone. 

---