Mathematics of Optimization

Let us contrast two approaches to optimization. One approach is searching, and the other one is mathematical optimization. 

Our next guess for the solution of the equation is Xj +. The approximation process is repeated until the function is close enough to zero to satisfy us. 

This mathematical optimization procedure is a rational process because the slope (or derivative) enables us to know which way to go and how far to go. In contrast, in the search procedure, we just arbitrarily choose some values of x at which to evaluate the function. Those arbitrary choices are much like what people do in most design situations. They are simply searching in a rather crude mannerfotThe solution to the problem, and they will not achieve the solution precisely. With mathematical optimization, our hope is both to speed up that process and to get a more precisely optimum solution. 

The mathematical procedure for a single merit function optimization for many design variables involves derivatives of the merit function with respect to each of the design variables (as a generalization to multiple 

---

In this volume, several key concepts used by practicing computational chemists are brought into focus. The first chapter by Tamar Schlick is dedicated to the mathematics of optimization. After some mathematical preliminaries, approaches to large-scale optimization are described. Basic decent structure of local methods is highlighted, and then nonderivative, gradient, and Newton methods are explained. 

Having filled in some of the mathematical foundations of optimization procedures, we shall return to the practical calculation of quantities of everyday use to the chemist. 

The following texts and articles provide an excellent discussion of optimization methods based on searching algorithms and mathematical modeling, including a discussion of the relevant calculations. 

Westerterp et al. (1984 see Case Study 4, preceding) conclude, Thanks to mathematical techniques and computing aids now available, any optimization problem can be solved, provided it is reahstic and properly stated. The difficulties of optimization lie mainly in providing the pertinent data and in an adequate construc tion of the objective function. ... 

Essential Features of Optimization Problems The solution of optimization problems involves the use of various tools of mathematics. Consequently, the formulation of an optimization problem requires the use of mathematical expressions. From a practical viewpoint, it is important to mesh properly the problem statement with the anticipated solution technique. Every optimization problem contains three essential categories ... 

No single method or algorithm of optimization exists that can be apphed efficiently to all problems. The method chosen for any particular case will depend primarily on (I) the character of the objective function, (2) the nature of the constraints, and (3) the number of independent and dependent variables. Table 8-6 summarizes the six general steps for the analysis and solution of optimization problems (Edgar and Himmelblau, Optimization of Chemical Processes, McGraw-HiU, New York, 1988). You do not have to follow the cited order exac tly, but vou should cover all of the steps eventually. Shortcuts in the procedure are allowable, and the easy steps can be performed first. Steps I, 2, and 3 deal with the mathematical definition of the problem ideutificatiou of variables and specification of the objective function and statement of the constraints. If the process to be optimized is very complex, it may be necessaiy to reformulate the problem so that it can be solved with reasonable effort. Later in this section, we discuss the development of mathematical models for the process and the objec tive function (the economic model). 

Formulation of the Objective Function The formulation of objective functions is one of the crucial steps in the application of optimization to a practical problem. You must be able to translate the desired objective into mathematical terms. In the chemical process industries, the obective function often is expressed in units of currency (e.g., U.S. dollars) because the normal industrial goal is to minimize costs or maximize profits subject to a variety of constraints. 

Pontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V. and Mishchenko, E.F. (1962) The Mathematical Theory of Optimal Processes, John Wiley Sons, New York. 

In this step, theoretical optimum conditions for the entire catalyst bed involving a number of pertinent parameters, such as temperature, pressure, and composition, are determined using mathematical methods of optimization [7,8]. The optimum conditions are found by attainment of a maximum or minimum of some desired objective. The best quality to be formed may be conversion, product distribution, temperature, or temperature program. 

How many organizations do what we could really call direct rational solution of the composite structure design problem — very, very few. Perhaps only in some very restricted design areas do people feel that they can use a mathematically oriented optimization approach. That situation is unfortunate, but changing. 

Goldfard, D., "Factorized Variable Metric Methods for Unconstrained Optimization", Mathematics of Computation, 30 (136) 796-811 (1976). 

The more variables one has in a given system, the more complicated becomes the job of optimization. But regardless of the number of variables, there will be a relationship between a given response and the independent variables. Once this relationship is known for a given response, it defines a response surface, such as that represented in Fig. 1. It is this surface that must be evaluated to find the values of the independent variables, Xi and X2, which give the most desirable level of the response, Y. Any number of independent variables can be considered representing more than two becomes graphically impossible, but mathematically only more complicated. 

Pontryagin LS (1962) The mathematical theory of optimal processes. Interscience, New York... 

A stochastic program is a mathematical program (optimization model) in which some of the problem data is uncertain. More precisely, it is assumed that the uncertain data can be described by a random variable (probability distribution) with sufficient accuracy. Here, it is further assumed that the random variable has a countable number of realizations that is modeled by a discrete set of scenarios co = 1,..., 2. 

The mathematical model of two-stage stochastic mixed-integer linear optimization problems was discussed as well as state-of-the-art solution algorithms. A new hybrid evolutionary algorithm for solving this class of optimization problems was presented. The new algorithm exploits the specific problem structure by stage decomposition. 

Constraints in optimization arise because a process must describe the physical bounds on the variables, empirical relations, and physical laws that apply to a specific problem, as mentioned in Section 1.4. How to develop models that take into account these constraints is the main focus of this chapter. Mathematical models are employed in all areas of science, engineering, and business to solve problems, design equipment, interpret data, and communicate information. Eykhoff (1974) defined a mathematical model as a representation of the essential aspects of an existing system (or a system to be constructed) which presents knowledge of that system in a usable form. For the purpose of optimization, we shall be concerned with developing quantitative expressions that will enable us to use mathematics and computer calculations to extract useful information. To optimize a process models may need to be developed for the objective function/, equality constraints g, and inequality constraints h. 

In this chapter we will discuss several factors that need to be considered when constructing a process model. In addition, we will examine the use of optimization in estimating the values of unknown coefficients in models to yield a compact and reasonable representation of process data. Additional information can be found in textbooks specializing in mathematical modeling. To illustrate the need to develop models for optimization, consider the following example. 

The formulation of objective functions is one of the crucial steps in the application of optimization to a practical problem. As discussed in Chapter 1, you must be able to translate a verbal statement or concept of the desired objective into mathematical terms. In the chemical industries, the objective function often is expressed in units of currency (e.g., U.S. dollars) because the goal of the enterprise is to minimize costs or maximize profits subject to a variety of constraints. In other cases the problem to be solved is the maximization of the yield of a component in a reactor, or minimization of the use of utilities in a heat exchanger network, or minimization of the volume of a packed column, or minimizing the differences between a model and some data, and so on. Keep in mind that when formulating the mathematical statement of the objective, functions that are more complex or more nonlinear are more difficult to solve in optimization. Fortunately, modem optimization software has improved to the point that problems involving many highly nonlinear functions can be solved. 

Part I comprises three chapters that motivate the study of optimization by giving examples of different types of problems that may be encountered in chemical engineering. After discussing the three components in the previous list, we describe six steps that must be used in solving an optimization problem. A potential user of optimization must be able to translate a verbal description of the problem into the appropriate mathematical description. He or she should also understand how the problem formulation influences its solvability. We show how problem simplification, sensitivity analysis, and estimating the unknown parameters in models are important steps in model building. Chapter 3 discusses how the objective function should be developed. We focus on economic factors in this chapter and present several alternative methods of evaluating profitability. 

An understanding of optimization techniques does not require complex mathematics. We require as background only basic tools from multivariable calculus and linear algebra to explain the theory and computational techniques and provide you with an understanding of how optimization techniques work (or, in some cases, fail to work). 

Olieman N (2004b) Optimal robust product design. Presentation at the 29th Conference on the Mathematics of Operations Research, Lunteren, The Netherlands... 

Accumulation of water inside the DLs and CLs may cause serious failure modes that can significantly deteriorate the performance and lifetime of a fuel cell. To ensure appropriate water removal, it is vital to understand the water transport mechanism inside a fuel cell, especially in the DLs. Because CFP and CC contain complex structures and porosities, many researchers have developed methods that could facilitate the characterization and design of optimal diffusion layers with proper water removal capabilities. A lot of work has also been performed on mathematical models that attempt to analyze the water flooding and transport inside DLs. A comprehensive review describing these models can be found in Sinha, Mukherjee, and Wang [222]. This section will discuss only examples of the experimental techniques. 

---