Least squares optimization technique

The computer programs FITEQL (12) and MICROQL (13) were used to model chemical speciation in this study. FITEQL uses a non-linear, least squares optimization technique to calculate equilibrium constants from chemical data. The program was used here to select... 

The most notable advance in computational crystallography was the availability of methods for rehning protein structures by least-squares optimization. This developed in a number of laboratories and was made feasible by the implementation of fast Fourier transform techniques [32]. The most widely used system was PROLSQ from the Flendrickson lab [33]. 

Historically, treatment of measurement noise has been addressed through two distinct avenues. For steady-state data and processes, Kuehn and Davidson (1961) presented the seminal paper describing the data reconciliation problem based on least squares optimization. For dynamic data and processes, Kalman filtering (Gelb, 1974) has been successfully used to recursively smooth measurement data and estimate parameters. Both techniques were developed for linear systems and weighted least squares objective functions. 

The mathematical techniques are part of multivariate statistics. They are closely related and often exchangeable. Two main approaches can be distinguished Least Squares Optimization (LSO), and Factor Analysis (FA). 

Esteban, M., Anno, C., Dfaz-Cruz, J.M., Dfaz-Cruz, M.S., and Tauler, R., Multivariate curve resolution with alternating least squares optimization a soft-modeling approach to metal complexation studies by voltammetric techniques, Trends Anal. Chem., 19, 49-61, 2000. 

Partial Least Squares (PLS) regression (Section 35.7) is one of the more recent advances in QSAR which has led to the now widely accepted method of Comparative Molecular Field Analysis (CoMFA). This method makes use of local physicochemical properties such as charge, potential and steric fields that can be determined on a three-dimensional grid that is laid over the chemical stmctures. The determination of steric conformation, by means of X-ray crystallography or NMR spectroscopy, and the quantum mechanical calculation of charge and potential fields are now performed routinely on medium-sized molecules [10]. Modem optimization and prediction techniques such as neural networks (Chapter 44) also have found their way into QSAR. 

Most of the force fields described in the literature and of interest for us involve potential constants derived more or less by trial-and-error techniques. Starting values for the constants were taken from various sources vibrational spectra, structural data of strain-free compounds (for reference parameters), microwave spectra (32) (rotational barriers), thermodynamic measurements (rotational barriers (33), nonbonded interactions (1)). As a consequence of the incomplete adjustment of force field parameters by trial-and-error methods, a multitude of force fields has emerged whose virtues and shortcomings are difficult to assess, and which depend on the demands of the various authors. In view of this, we shall not discuss numerical values of potential constants derived by trial-and-error methods but rather describe in some detail a least-squares procedure for the systematic optimisation of potential constants which has been developed by Lifson and Warshel some time ago (7 7). Other authors (34, 35) have used least-squares techniques for the optimisation of the parameters of nonbonded interactions from crystal data. Overend and Scherer had previously applied procedures of this kind for determining optimal force constants from vibrational spectroscopic data (36). 

Always based on the use of IR spectrophotometry, a novel attenuated total reflection-Fourier-transform infrared (ATR-FTIR) sensor [42] was proposed for the on-line monitoring of a dechlorination process. Organohalogenated compounds such as trichloroethylene (TCE), tetrachloroethylene (PCE) and carbon tetrachloride (CT) were detected with a limit of a few milligrams per litre, after extraction on the ATR internal-reflection element coated with a hydro-phobic polymer. As for all IR techniques, partial least squares (PLS) calibration models are needed. As previously, this system is promising for bioprocess control and optimization. 

The multivariate techniques which reveal underlying factors such as principal component factor analysis (PCA), soft Independent modeling of class analogy (SIMCA), partial least squares (PLS), and cluster analysis work optimally If each measurement or parameter Is normally distributed In the measurement space. Frequency histograms should be calculated to check the normality of the data to be analyzed. Skewed distributions are often observed In atmospheric studies due to the process of mixing of plumes with ambient air. 

The optimal number of components from the prediction point of view can be determined by cross-validation (10). This method compares the predictive power of several models and chooses the optimal one. In our case, the models differ in the number of components. The predictive power is calculated by a leave-one-out technique, so that each sample gets predicted once from a model in the calculation of which it did not participate. This technique can also be used to determine the number of underlying factors in the predictor matrix, although if the factors are highly correlated, their number will be underestimated. In contrast to the least squares solution, PLS can estimate the regression coefficients also for underdetermined systems. In this case, it introduces some bias in trade for the (infinite) variance of the least squares solution. 

Fitting model predictions to experimental observations can be performed in the Laplace, Fourier or time domains with optimal parameter choices often being made using weighted residuals techniques. James et al. [71] review and compare least squares, stochastic and hill-climbing methods for evaluating parameters and Froment and Bischoff [16] summarise some of the more common methods and warn that ordinary moments matching-techniques appear to be less reliable than alternative procedures. References 72 and 73 are studies of the errors associated with a selection of parameter extraction routines. 

In a strict sense parameter estimation is the procedure of computing the estimates by localizing the extremum point of an objective function. A further advantage of the least squares method is that this step is well supported by efficient numerical techniques. Its use is particularly simple if the response function (3.1) is linear in the parameters, since then the estimates are found by linear regression without the inherent iteration in nonlinear optimization problems. 

A least squares technique was used to optimize the virial coefficient and the k values (34), using the light-scattering measurements at Mw = 1,285,000 and 413,000. The k value could not be resolved statistically since variations in k gave only slight variations in the least squares deviations from the experimental values. For this and the above-mentioned reason, k was considered to be equal to 0.25. With this value, the virial coefficients were optimized. 

SimSim performs a pressure match of measured and calculated reservoir or compartment pressures with an automatic, non-linear optimization technique, called the Nelder-Mead simplex algorithm3. During pressure matching SimSim s parameters (e.g. hydrocarbons in place, aquifer size and eigentime, etc.) are varied in a systematic manner according to the simplex algorithm to achieve pressure match. In mathematical terms the residuals sum of squares (least squares) between measured and calculated pressures is minimized. The parameters to be optimized can be freely selected by the user. 

This comparison is performed on the basis of an optimality criterion, which allows one to adapt the model to the data by changing the values of the adjustable parameters. Thus, the optimality criteria and the objective functions of maximum likelihood and of weighted least squares are derived from the concept of conditioned probability. Then, optimization techniques are discussed in the cases of both linear and nonlinear explicit models and of nonlinear implicit models, which are very often encountered in chemical kinetics. Finally, a short account of the methods of statistical analysis of the results is given. 

For models in which the dependent variables are linear functions of the parameters, the solution to the above-mentioned optimization problems can be obtained in closed form when the least squares objective functions (3.22) and (3.24) are considered. However, in chemical kinetics, linear problems are encountered only in very simple cases, so that optimization techniques for nonlinear models must be considered. 

This problem contains 31 variables and 29 equality constraints (or governing equations) including the objective function. This gives rise to 2 variables as independent (or decision) variables. For a practical reason, the saturation pressure for steam, P, and the fraction of steam generated in the evaporator, which is reused for heating, a.., are selected as the independent variables. A random search technique (26) is adopted to locate the optimal point for each given e. The results are tabulated in Table I, and the trade-off curve is plotted in Figure 3. The relationship between these two objectives is obtained by the least square method as... 

---