Large-scale minimization algorithms

X. Zou, I. M. Navon, F. X. Le Dimet, A. Nouailler, and T. Schlick, SIAM J. Opt., in press. A Comparison of Efficient Large-Scale Minimization Algorithms for Optimal Control Applications in Meteorology. 

T. Schlick and A. Fogelson. TNPACK — A truncated Newton minimization package for large-scale problems I. Algorithm and usage. ACM Trans. Math. Softw., 14 46-70, 1992. 

Unfortunately, the publication by Williams and coworkers is one of the only reports of a scale-up problem involving liquids or semisolids in the pharmaceutical literature. A number of papers that purport to deal with scale-up issues and even go so far as to compare the properties of small versus large batches failed to apply techniques, such as dimensional analysis, that could have provided the basis for a far more substantial assessment or analysis of the scale-up problem for their system. Worse yet, there is no indication of how scale-up was achieved or what scale-up algorithm(s), if any, were used. Consequently, their usefulness, from a pedagogical point of view, is minimal. In the end, effective scale-up requires the complete characterization of the materials and processes involved and a critical evaluation of all laboratory and production data that may have some bearing on the scalability of the process. 

Another introductory note may provide further incentive for novice optimizers to read on. Despite extensive developments in optimization methods in the last decade, large-scale nonlinear optimization still remains an art that requires considerable computing experience, algorithm familiarity, and intuition. In general, black box minimization implementations, even those using state-of-the-art algorithms, are only partially successful. 

The multiple-minimum problem is a severe handicap of many large-scale optimization applications. The state of the art today is such that for reasonable small problems (30 variables or less) suitable algorithms exist for finding all local minima for linear and nonlinear functions. For larger problems, however, many trials are generally required to find local minima, and finding the global minimum cannot be ensured. These features have prompted research in conformational-search techniques independent of, or in combination with, minimization.26... 

The differential equation has been resolved with an algorithm which makes use of an explicit Runge-Kutta(4,5) formula, the so-called Dormand-Prince pair. The least square minimization was implemented by a large-scale algorithm based on the interior-reflective Newton method. All the computations have been achieved by MATLAB software. 

---