Iterative optimization methods —

The number of experiments can considerably be decreased by iterative optimization methods which starts at an area that can be selected by experience, supposition or randomly. This start area is moved step by... 

Therefore, an iterative optimization method requires that both the direction, pj, and the value of the step, Ui, along p be selected. 

The transformation in Eq. 7-13 leads to the fraction incomplete response method of determining first-order models discussed in the next section. However, for step responses of higher-order models, such as Eq. 5-48, the transformation approach is not feasible. For these calculations, we must use an iterative optimization method to find the least-squares estimates of the time constants (Edgar et al., 2001). 

There are several mathematical methods for producing new values of the variables in this iterative optimization process. The relation between a simulation and an optimization is depicted in Eigure 6. Mathematical methods that provide continual improvement of the objective function in the iterative... 

Once the path has been optimized with NEB, it is used as the initial guess for the parallel path optimizer method. In this second step the path is again iteratively optimized with the parallel path optimizer method for the core set, followed by the optimization of the environment set. In this part of the calculation no restraints are imposed on the environment set during the optimization. The iterations are continued until all the convergence criteria are met and the final optimized MEP is obtained. 

In order to perform FEP calculations on optimized paths with a small number of images, extra images need to be added on the path between the previously optimized points. Once these extra images have been added, an optimization has to be performed to minimize them to the MEP. Here we have developed a modification to our NEB QM/MM implementation [27], This modification allows for the optimization of only selected images on the path while maintaining the points previously optimized with the parallel iterative path method or the combined procedure fixed. 

In our simulations of histone modifying enzymes, the computational approaches centered on the pseudobond ab initio quantum mechanical/molecular mechanical (QM/MM) approach. This approach consists of three major components [20,26-29] a pseudobond method for the treatment of the QM/MM boundary across covalent bonds, an efficient iterative optimization procedure which allows for the use of the ab initio QM/MM method to determine the reaction paths with a realistic enzyme environment, and a free energy perturbation method to take account... 

Basically, two types of approaches are developed here iterative (optimization-based) approaches like the one by Sippl et al. [101] and direct approaches like the one by Kabsch [102, 103], based on Lagrange multipliers. Unfortunately, the much expedient direct methods may fail to produce a sufficiently accurate solution on some degenerate cases. Redington [104] suggested a hybrid method with an improved version of the iterative approach, which requires the computation of only two 3x3 matrix multiplications in the inner loop of the optimization. 

The MOs in eq 5 are typically optimized using a reorthogonalization technique that has been described by Gianinetti et al.,(30) though they can also be obtained using a Jacobi rotation method that sequentially and iteratively optimizes each individual orbital.(28,37)... 

The choice of optimization scheme in practical applications is usually made by considering the convergence rate versus the time needed for one iteration. It seems today that the best convergence is achieved using a properly implemented Newton-Raphson procedure, at least towards the end of the calculation. One full iteration is, on the other hand, more time-consuming in second order methods, than it is in more approximative schemes. It is therefore not easy to make the appropriate choice of optimization method, and different research groups have different opinions on the optimal choice. We shall discuss some of the more commonly implemented methods later. 

Several attempts have been made to devise simpler optimization methods than the lull second order Newton-Raphson approach. Some are approximations of the full method, like the unfolded two-step procedure, mentioned in the preceding section. Others avoid the construction of the Hessian in every iteration by means of update procedures. An entirely different strategy is used in the so called Super - Cl method. Here the approach is to reach the optimal MCSCF wave function by annihilating the singly excited configurations (the Brillouin states) in an iterative procedure. This method will be described below and its relation to the Newton-Raphson method will be illuminated. The method will first be described in the unfolded two-step form. The extension to a folded one-step procedure will be indicated, but not carried out in detail. We therefore assume that every MCSCF iteration starts by solving the secular problem (4 39) with the consequence that the MC reference state does not... 

Only the gradient vector is calculated exactly in approximate optimization methods like the super-CI approach. This information about the exact gradients can be used to improve the convergence of the calculation via a procedure, that updates the approximate Hessian, which is implicitly used in the calculation. Suppose that we know the gradient at two consecutive points in a sequence of iterations, p(n+1) and p(n). Let us expand the gradient around the point p(n+1) ... 

Step 5 in Table 8-6 involves the computation of the optimum point. Quite a few techniques exist to obtain the optimal solution for a problem. We describe several classes of methods below. In general, the solution of most optimization problems involves the use of a digital computer to obtain numerical answers. Over the past 15 years, substantial progress has been made in developing efficient and robust computational methods for optimization. Much is known about which methods are most successful. Virtually all numerical optimization methods involve iteration, and the effectiveness of a given technique can depend on a good first guess for the values of the variables at the optimal solution. After the optimum is computed, a sensitivity analysis for the objective function value should be performed to determine the effects of errors or uncertainty in the objective function, mathematical model, or other constraints. 

Newton s method and quasi-Newton techniques make use of second-order derivative information. Newton s method is computationally expensive because it requires analytical first-and second-order derivative information, as well as matrix inversion. Quasi-Newton methods rely on approximate second-order derivative information (Hessian) or an approximate Hessian inverse. There are a number of variants of these techniques from various researchers most quasi-Newton techniques attempt to find a Hessian matrix that is positive definite and well-conditioned at each iteration. Quasi-Newton methods are recognized as the most powerful unconstrained optimization methods currently available. 

For such applications of classical optimization theory, the data on energy and gradients are so computationally expensive that only the most efficient optimization methods can be considered, no matter how elaborate. The number of quantum chemical wave function calculations must absolutely be minimized for overall efficiency. The computational cost of an update algorithm is always negligible in this context. Data from successive iterative steps should be saved, then used to reduce the total number of steps. Any algorithm dependent on line searches in the parameter hyperspace should be avoided. 

---