Gradient vector estimation

A more sophisticated version of the sequential univariate search, the Fletcher-Powell, is actually a derivative method where elements of the gradient vector g and the Hessian matrix H are estimated numerically. 

The quasi-Newton methods estimate the matrix = H-1 by updating a previous guess of C in each iteration using only the gradient vector. These methods are very close to the quasi-Newton methods of solving a system of nonlinear equations. The order of convergence is between 1 and 2, and the minimum of a positive definite quadratic function is found in a finite number of steps. 

The basic idea is that the gradient vector of the objective function, VI/(0) = [91//36>i dU/d0Nf], represents the direction of faster increase of the function. Hence, the estimate at step s + 1 can be computed via the recursive law... 

On solving these equations for the coefficients A j, the solution of minimum norm is the interpolated gradient vector P, such that pi = P 2 0, at the interpolated coordinate vector Q. The Lagrange multiplier /x in this method provides an estimate of the residual error. 

The interpolated gradient vector is p = p GDIIS algorithm, a currently updated estimate of the inverse Hessian is used to estimate a coordinate step based on these interpolated vectors. This gives... 

Any reasonably efiScient method for optimizing the expected value function g(x), say by using its sample average approximations, is based on estimation of its first (and maybe second) order derivatives. This has an independent interest and is called sensitivity or perturbation analysis. We will discuss that in Section 3.3. Recall that Vg(x) = dgix)/dXi,. . . , dg(x)/dXj) is called the gradient vector of g(0 at x. 

Note that using Eq. (10) does not eliminate the gradient vector which provides the direction of the required displacement along the energy surface toward the minimum. An improved estimate of the configuration of the system is then... 

Figure 21 shows the upper bounds of the maximum drift of the base-isolation story compared for various methods (URP methods with second-order Taylor series approximation/with RSM and the Monte Carlo Simulation (MCS)). As explained before, the difference between the URP method with second-order Taylor series approximation and that with RSM is how to estimate the variation of the objective function. In the former one, the numerical sensitivities, i.e., the gradient vector and the Hessian matrix, of the objective function are needed. On the other hand, in the latter one, a kind of RSM is applied where appropriate response samplings are made and the gradient vector and the Hessian matrix are evaluated from the constmcted approximate function. 

Fig. 2. Gradient estimates for a horizontal reflector with locally varying amplitude. The black lines correspond to reflection amplitude countours and the arrows to the gradient vectors.

Now, we would like to attach a variance to the estimates of a and P that make c2 minimum. Given the complex and non-Unear nature of the gradient equations (5.4.67) and (5.4.68), we assume for simplicity that a and / are normally distributed and resort to linear propagation in order to retrieve an estimate for the covariance matrix of a and p. The covariance matrix Saj, of the vector (aT, ft) is approximated by... 

We can now qualitatively describe the conjugate-gradient method for minimizing a general function, E(x), where x is an /V-dimensional vector. We begin with an initial estimate, xo. Our first iterate is chosen to lie along the direction defined by do = V (xo), so xi = xo a0do- Unlike the simple example... 

For linear functions of the first order (without interaction terms) the vector of the coefficients forms the gradient of the estimated model. Following the steps indicated by the coefficients one will reach the optimum in the steepest ascent mode. 

For a given sequence, Bloch equations give the relationship between the explanatory variables, x, and the true response, i]. The / -dimensional vector, 0, corresponds to the unknown parameters that have to be estimated x stands for the m-dimensional vector of experimental factors, i.e., the sequence parameters, that have an effect on the response. These factors may be scalar (m — 1), as previously described in the TVmapping protocol, or vector (m > 1) e.g., the direction of diffusion gradients in a diffusion tensor experiment.2 The model >](x 0) is generally non-linear and depends on the considered sequence. Non-linearity is due to the dependence of at least one first derivative 5 (x 0)/50, on the value of at least one parameter, 6t. The model integrates intrinsic parameters of the tissue (e.g., relaxation times, apparent diffusion coefficient), and also experimental nuclear magnetic resonance (NMR) factors which are not sufficiently controlled and so are unknown. 

Here Pij gives the value of the parameter having the number i for the iteration with the number j. The parameter m of the relation (3.238) can be estimated using a variation of the Gauss-Newton gradient technique. The old procedure for the estimation of m starts from the acceptance of the vector of parameters being limited between a minimal and maximal a priori accepted value Pmin N P -< Pmax- Here we can introduce a vector of dimensionless parameters Pnd = (P Pmin)/(Pmax Pmin)> which is ranged between zero and one for the minimal and the maximal values, respectively. With these limit values, we can compute the values of the dimensionless function for P d = 0,0.5, las (0),(D(0. 5) and (1) and then they can be used for the estimation of mp... 

---