Non-linearity parameter

Marquadt, D. W., An algorithm for least-squares estimation of non-linear parameters, J. Soc. Indust. Appl. Math., 11, 431, 1963. 

Instead of using repeated solution of a suitable eigenvalue equation to optimize the orbitals, as in conventional forms of SCF theory, we have found it more convenient to optimize by a gradient method based on direct evaluation of the ener functional (4), ortho normalization being restored after every parameter variation. Although many iterations are required, the energy evaluation is extremely rapid, the process is very stable, and any constraints on the parameters (e.g. due to spatial symmetry or choice of some type of localization) are very easily imposed. It is also a simple matter to optimize with respect to non-linear parameters such as orbital exponents. 

Basically two search procedures for non-linear parameter estimation applications apply. (Nash and Walker-Smith, 1987). The first of these is derived from Newton s gradient method and numerous improvements on this method have been developed. The second method uses direct search techniques, one of which, the Nelder-Mead search algorithm, is derived from a simplex-like approach. Many of these methods are part of important mathematical computer-based program packages (e.g., IMSL, BMDP, MATLAB) or are available through other important mathematical program packages (e.g., IMSL). 

The application of optimisation techniques for parameter estimation requires a useful statistical criterion (e.g., least-squares). A very important criterion in non-linear parameter estimation is the likelihood or probability density function. This can be combined with an error model which allows the errors to be a function of the measured value. A simple but flexible and useful error model is used in SIMUSOLV (Steiner et al., 1986 Burt, 1989). 

Non-linear parameter estimation is far from a trivial task, even though it is greatly simplified by the availability of user-friendly program packages such as a) SIMUSOLV (Steiner et al., 1986), b) ESL, c) a set of BASIC programs (supplied with the book of Nash and Walker-Smith, 1987) or d) by mathematical software (MATLAB). ISIM itself does not supply these advanced features, but ISIM programs can easily be translated into other more powerful languages. 

The virial ratio is, as we noted above, 1.3366 for the separate-atom AO basis MO calculation, i.e. not 1.0. Now within the confines of the linear variation method (the usual LCAO approach) there is no remaining degree of freedom to use in order to constrain the virial ratio to its formally correct value (or indeed to impose any other constraint). Thus imposing the correct virial ratio on the linear variation method is, in this case, not possible without simultaneously destroying the symmetry of the wave function. Only by optimising the non-linear parameters can we improve the virial ratio as the above results show. Even at this most elementary level, the imposition of various formally correct constraints on the wave function is seem to generate contradictions. 

The conclusion above that optimisation of the non-linear parameters in the AO basis leads to a basis with correct spatial symmetry properties cannot be true for all intemuclear separations. At R = 0 the orbital basis must pass over into the double-zeta basis for helium i.e. two different 1 s orbital exponents. It would be astonishing if this transition were discontinuous at R = 0. While considering the variation of basis with intemuclear distance it is worth remembering that the closed-shell spin-eigenfunction MO method does not describe the molecule at all well for large values of R the spin-eigenfunction constraint of two electrons per spatial orbital is completely unrealistic at large intemuclear separation. With these facts in mind we have therefore computed the optimum orbital exponents as a function of R for three wave functions ... 

The most difficult problem we face in deciding to use a basis of hybrids which reflects the molecular symmetry is how do we choose such a basis in view of the enormous numerical difficulties involved in optimising the non-linear parameters in molecular calculations The real question is are there any rules for this choice, can the optimisation be done (at least approximately) once and for all The chemical evidence is for us — it is the most basic concept of the theory of valence that particular electronic sub-structures tend to be largely environment-independent. How can we select our basis to reflect this chemical fact ... 

In any book, there are relevant issues that are not covered. The most obvious in this book is probably a lack of in-depth statistical analysis of the results of model-based and model-free analyses. Data fitting does produce standard deviations for the fitted parameters, but translation into confidence limits is much more difficult for reasonably complex models. Also, the effects of the separation of linear and non-linear parameters are, to our knowledge, not well investigated. Very little is known about errors and confidence limits in the area of model-free analysis. 

More careful examination of this shape reveals two important facts, (a) Plots of ssq as a function of k at fixed Io are not parabolas, while plots of ssq vs. Io at fixed k are parabolas. This indicates that Io is a linear parameter and k is not. (b) Close to the minimum, the landscape becomes almost parabolic, see Figure 4-6. We will see later in Chapter 4.3, Non-Linear Regression, that the fitting of non-linear parameters involves linearisation. The almost parabolic landscape close to the minimum indicates that the linearisation is a good approximation. 

It is probably more realistic to assume that we know neither the rate constants nor the absorption spectra for the above example. All we have is the measurement Y and the task is to determine the best set of parameters which include the rate constants ki and /cj and the molar absorptivities, the whole matrix A. This looks like a formidable task as there are many parameters to be fitted, the two rate constants as well as all elements of A. In Multivariate Data, Separation of the Linear and Non-Linear Parameters (p.162), we start tackling this problem. 

Multivariate Data, Separation of the Linear and Non-Linear Parameters... 

The secret is to realise that there are linear and non-linear parameters and that they can be separated, essentially reducing the number of parameters to be fitted iteratively, to the number of non-linear ones. The vector p defines the matrix C of concentrations and C, in turn, allows the computation of the best matrix A as a linear least-squares fit, A=C+Y, recall equation (4.61). Thus R can be computed as... 

A straightforward way to organise J is as a 3-dimensional array The derivative of R with respect to one particular parameter pi is a matrix of the same dimensions as R itself. The collection of all these nsx nl derivatives with respect to all the np non-linear parameters (e.g. rate constants) can be arranged in a 3-D array of dimensions nsxnlxnp, all individual matrices dR/dpi written slice-wise behind each other. This is illustrated in Figure 4-41. 

Remember, p only comprises the non-linear parameters, i.e. in case of our consecutive reaction the rate constants ki and fe. The linear parameters, the elements of the matrix A containing the molar absorptivities, have effectively been eliminated. 

The important point is that the above manipulations do not affect the matrix C which is directly related to the model and to the non-linear parameters. Only the matrices Y and A are reduced to Yred and Ared. 

As outlined in Multivariate Data, Separation of the Linear and Non-Linear Parameters, (p.162), it is crucial to eliminate the linear parameters by calculating the matrix A of molar absorptivities as a function of C and thus the rate constants. In fact, the function SsqCalc ABC is almost identical to Rcalc ABC (p.167). The only difference concerns the sum of squares, ssq, which is now returned instead of the residuals. 

In the next example, we re-analyse the consecutive reaction, Data ABC, m [p. 143)and [p.165], This time however, we use fewer data in order to keep the Excel spreadsheet reasonably compact. The important concept of treating linear and non-linear parameters separately can be implemented in Excel as well. 

The matrix C is defined by the non-linear parameters (rate constants). It is possible to minimise Ru, i.e. the corresponding ssq, as a function of these parameters in a normal Newton-Gauss algorithm. The chain of equations goes as follows... 

One requirement for second-order non-linearity in optical molecules is that they exhibit non-centrosymmetric symmetry, i.e. they must be dipolar in nature and all point in the same direction. Hence, materials suitable for electro-optical uses should have high figures for the multiplier where p is the dipole moment and P the molecular second order optical non-linearity parameter. ... 

---