Ill-conditioned equation

The solution of a differential equation consists of a special trend univocally determined by the differential equation itself and the initial conditions. Figure 2.2 shows an ill-conditioned equation formulation small perturbations in the initial conditions or small deviations from the solution lead to completely different trends. 

This set is said to be ill-conditioned because the second equation is almost an exact multiple of the first. The matrix of coefficients is almost singular. 

If a matrix is ill-conditioned, its inverse may be inaccurate or the solution vector for its set of equations may be inaccurate. Two of the many ways to recognize possible ill-conditioning are... 

In practice, the solution of Equation 3.16 for the estimation of the parameters is not done by computing the inverse of matrix A. Instead, any good linear equation solver should be employed. Our preference is to perform first an eigenvalue decomposition of the real symmetric matrix A which provides significant additional information about potential ill-conditioning of the parameter estimation problem (see Chapter 8). 

Thus, the error in the solution vector is expected to be large for an ill-conditioned problem and small for a well-conditioned one. In parameter estimation, vector b is comprised of a linear combination of the response variables (measurements) which contain the error terms. Matrix A does not depend explicitly on the response variables, it depends only on the parameter sensitivity coefficients which depend only on the independent variables (assumed to be known precisely) and on the estimated parameter vector k which incorporates the uncertainty in the data. As a result, we expect most of the uncertainty in Equation 8.29 to be present in Ab. 

If matrix A is well-conditioned, the above equation should be used. If however, A is ill-conditioned, we have the option without any additional computation effort, to use instead the pseudoinverse of A. Essentially, instead of A 1 in Equation 8.31, we use the pseudoinverse of A, A. ... 

At this point we should always try and see whether there is anything else that could be done to reduce the ill-conditioning of the problem. Upon reexamination of the structure of the model given by Equation 16.4 we can readily notice that it can be rewritten as... 

Algorithmic Details for NLP Methods All the above NLP methods incorporate concepts from the Newton-Raphson method for equation solving. Essential features of these methods are that they rovide (1) accurate derivative information to solve for the KKT con-itions, (2) stabilization strategies to promote convergence of the Newton-like method from poor starting points, and (3) regularization of the Jacobian matrix in Newton s method (the so-called KKT matrix) if it becomes singular or ill-conditioned. 

This system of 2 M nonlinear equations is ill-conditioned for large M, but can be efficiently solved using the product-difference (PD) algorithm introduced by McGraw (1997). Thus, given the set of 2 M moments on the left-hand side of Eq. (107), the PD algorithm returns wm and lm for m — 1., M. The closed microscopic transport equation for the moments can then be written for k — 0,..., 2 M— 1 as... 

Thus, it would be natural to attempt to extend the QMOM approach to handle a bivariate NDF. Unfortunately, the PD algorithm needed to solve the weights and abscissas given the moments cannot be extended to more than one variable. Other methods for inverting Eq. (125) such as nonlinear equation solvers can be used (Wright et al., 2001 Rosner and Pykkonen, 2002), but in practice are computationally expensive and can suffer from problems due to ill-conditioning. 

These equations become harder and harder to solve numerically as V2/ becomes more ill-conditioned. When its condition number exceeds 1014, there will be few if any correct digits in the computed solution using double precision arithmetic (see Luenberger, 1984). 

Both problems cannot be solved in a straightforward manner. Mathematical treatments can be ill-conditioned because of the large number of independent variables and noise. Approximations were often made in order to simplify the equation of motion. Examples of strategies of reconstruction are given in Ref. 28 and are briefly described below, in a decreasing order of approximations ... 

In the present case, the system has more equations (11) than unknowns (7) and may be conveniently solved for x = Al3...Ah2. Png, and Pn by the least-square solution alluded to above. The system is, in general, ill-conditioned and extended precision should be used for the inversion. 

P. Deuflhard. A Modified Method for the Solution of Ill-Conditioned Systems of Nonlinear Equations with Application to Multiple Shooting. Numer. Math., 32 289-315,1974. 

An alternate approach is to impose continuity constraints prior to estimation, thus utilizing the sine waves that would be eliminated aposteriori due to ill-conditioning. For example, a linear model for each frequency trajectory can be shown to lead to a generalization of Equation (9.77). Such an approach may lead to more robust separation with the presence of closely spaced frequencies. 

An expanded formulation of the steady-state permeation model has been presented. Two numerical problems - stiffness and an ill-conditioned boundary value problem - are encountered in solving the system equations. These problems can be circumvented by matching forward and reverse integrations at a point near the inlet (n = 0) but outside the combustion zone. The model predicts a... 

From a computational point of view it is important to ensure that the problem is not ill-conditioned so as to maintain numerical stability. Therefore, the use of rational functions as proposed by Bergmann et a/.81 is combined with the established practice of fitting polynomials to given data using orthogonal polynomial bases.83 A system of linear equations is solved at each iteration.81 Hence, the condition number83 of this system may be used to monitor numerical stability. 

---