Levenberg-Marquardt modification

The learning rate tj is used to adjust the stabihty and convergence rate, especially when the optimum is very flat. The Levenberg-Marquardt modification of the Gauss-Newton method introduces an additional factor /u >=0, such that ... 

Modifications of Levenberg-Marquardt Method Fletcher s Modification... 

A number of modifications to eliminate some less favorable aspects of the Levenberg-Marquardt method were considered by Fletcher. For instance, the arbitrary initial choice of the adjustable parameter A, if poor, can cause an excessive number of evaluations of squared error, before a realistic value is obtained. This is especially noticeable if v, i.e., J R x), is chosen to be small, i.e., v = 2. Another disadvantage of the method is that the reduction of A to v at the start of each iteration may also cause excessive evaluations, especially when V is chosen to be large, i.e., = 10. The... 

In general there are two remedies to this problem (i) use a pseudoinverse and/or (ii) use Levenberg-Marquardt s modification. 

A more interesting interpretation of Levenberg-Marquardt s modification can be obtained by examining the eigenvalues of the modified matrix (A+y2 ). If we consider the eigenvalue decomposition of A, V1 AV we have,... 

The binary interaction parameter, k j, is initially assumed to be zero, and a modification of the Levenberg-Marquardt algorithm (MINPACK) is applied to minimize the sum of the squares given by Equation (1). This calculation was applied to the following systems at the indicated temperatures ... 

One valid alternative to the Levenberg-Marquardt method is the dogleg method, also known as Powell s hybrid method (Rabinowitz, 1970). Once again, this couples the Newton and gradient methods. The original version of Powell s method was close to the tmst region concept. Powell proposed a strategy for the modification of parameter d subject to both the successes and failures of the procedure. 

---