Direct methods random search

Once a direction is estabflshed for the next poiat ia the space of the variables of optimization (whether by random search, by systematic evaluation of gradients, or by any other methods of making perturbations), it is possible to take a jump ia the directioa of the improvement much greater than the size of the perturbations. This could speed up the process of finding the optimum and reduce computer time. If such a leap is successful, the next iteration may take a bigger leap and so on, until the improvement stops. Then one could reverse the direction and decrease the size of the step until the optimum is found. 

One of the most reliable direct search methods is the LJ optimization procedure (Luus and Jaakola, 1973). This procedure uses random search points and systematic contraction of the search region. The method is easy to program and handles the problem of multiple optima with high reliability (Wang and Luus, 1977, 1978). A important advantage of the method is its ability to handle multiple nonlinear constraints. 

Deterministic methods. Deterministic methods follow a predetermined search pattern and do not involve any guessed or random steps. Deterministic methods can be further classified into direct and indirect search methods. Direct search methods do not require derivatives (gradients) of the function. Indirect methods use derivatives, even though the derivatives might be obtained numerically rather than analytically. 

Banga et al. [in State of the Art in Global Optimization, C. Floudas and P. Pardalos (eds.), Kluwer, Dordrecht, p. 563 (1996)]. All these methods require only objective function values for unconstrained minimization. Associated with these methods are numerous studies on a wide range of process problems. Moreover, many of these methods include heuristics that prevent premature termination (e.g., directional flexibility in the complex search as well as random restarts and direction generation). To illustrate these methods, Fig. 3-58 illustrates the performance of a pattern search method as well as a random search method on an unconstrained problem. 

Random Search / 6.1.2 Grid Search / 6.1.3 Univariate Search / 6.1.4 Simplex Search Method / 6.1.5 Conjugate Search Directions / 6.1.6 Summary... 

Nonderivative methods include random search, grid search, simplex search, and conjugate directions (or Powell s method). The nonderivative methods use various patterns for generating new test points for decision variables, and then a comparison of the new objective function value against previous values. A subsequent test point is then generated, either based on the immediate comparison or using the previous history of test points. 

Clark et al. - also compared the GA with distance geometry, directed tweak, and random search methods on the problem of finding good phar-macaphoric matches in databases df thousands of molecules. Their final conclusion is that GA and directed tweak outperform all of the methods, and that the directed tweak may in fact be preferable to GA, as mentioned earlier in this chapter. 

As an alternative to RSM, simulation responses can be used directly to explore the sample space of control variables. To do so, a lot of combinatorial optimization approaches were adapted for simulation optimization. In general, there are four main classes of methods that have shown a particular applicability in (multi-objective) simulation optimization Meta-heuristics, gradient-based procedures, random search, and sample path optimization. Of particular interest are meta-heuristics as they have shown a good performance for a wide range of combinatorial optimization approaches. Therefore, commercial simulation software primarily uses these techniques to incorporate simulation optimization routines. Among meta-heuristics, tabu search, scatter search, and genetic algorithms are most widely used. Table 4.13 provides an overview on aU aforementioned techniques. 

An exception of that is PATSEE [77], which uses one or two known fragments of the structure, whose orientation is found by a real-space Patterson rotation search and its translation in the cell by direct methods (Section 15.2.2.2). starting from random positions. Figures of merit related to those... 

Methods that combine a search technique with other optimization approaches include distance geometry, random search, and genetic algorithms. A comparison of these methods with derivative-based optimization concluded that all were inferior in performance to directed tweak, with the most favorable of the three being the genetic algorithm. ... 

A more sophisticated method uses a random walk or simplex optimization search pattern, which was developed and is used to find downed aircraft or ships lost at sea. Variable limits are set, then three conditions within these limits are selected at random, injections are made, and chromatograms are run. The resolution sums for the injections are measured and calculated, the lowest value is discarded, and a new variable setting is selected directly opposite the discarded value and equidistant from the reject on a line connecting the two remaining values from the original triad (Fig. 14.3). 

Search problems can be divided into two groups, depending on whether or not random experimental error is associated with each measurement. There are, indeed, significant problems that have no experimental error as when the function in question is given as an exact mathematical expression, but one too complicated to be optimized directly by calculus or by known methods of mathematical programming. Design problems are often of this latter nature. We shall discuss mainly the no-error case, since its principles are simple and can be used even in the presence of experimental error. 

---