Simulated annealing , subset selection

As might be expected, established optimisation techniques such as simulated annealing and genetic algorithms have been used to tackle the subset selection problem. These methods... 

Linear Models. Variable selection approaches can be applied in combination with both linear and nonlinear optimization algorithms. Exhaustive analysis of all possible combinations of descriptor subsets to find a specific subset of variables that affords the best correlation with the target property is practically impossible because of the combinatorial nature of this problem. Thus, stochastic sampling approaches such as genetic or evolutionary algorithms (GA or EA) or simulated annealing (SA) are employed. To illustrate one such application we shall consider the GA-PLS method, which was implemented as follows (136). 

Models can be generated using stepwise addition multiple linear regression as the descriptor selection criterion. Leaps-and-bounds regression [10] and simulated annealing (ANNUN) can be used to find a subset of descriptors that yield a statistically sound model. The best descriptor subset found with multiple linear regression can also be used to build a computational neural network model. The root mean square (rms) errors and the predictive power of the neural network model are usually improved due to the higher number of adjustable parameters and nonlinear behavior of the computational neural network model. 

Figure 5. Calculated vs. observed -log( 5o) values using a computational neural network model with the descriptor subset selected by generalized simulated annealing (anndes).

Agrafiotis [60] has also developed a simulated annealing method for maximising diversity. The method employs a user-defined objective function and can therefore be tailored to encode different selection criteria. The results of subset selection can be visualized using Sammon s nonlinear mapping algorithm [61]. 

Sutter and co-workers reported a method for automated descriptor selection for quantitative structure-activity relationships using generalized simulated annealing (36,132). The cost function used to evaluate the effectiveness of the deseriptors was based on a neural network. The result is an automated descriptor selection algorithm that is an optimization inside of an optimization. Application of the method to QSAR shows that effective descriptor subsets are found, and they support models that are as good or better than those obtained using traditional linear regression methods. 

---