System of normal equations

If the explicit solution cannot be used or appears impractical, we have to return to the general formulation of the problem, given at the beginning of the last section, and search for a solution without any simplifying assumptions. The system of normal equations (34) can be solved numerically in the following simple way (164). Let us choose an arbitrary value x(= T ) and search for the optimum ordinate of the point of intersection y(= log k) and optimum values of slopes bj to give the least residual sum of squares Sx (i.e., the least possible with a fixed value of x). From the first and third equations of the set eq. (34), we get... 

Values a , bj and m are obtained from this system of normal equations ... 

Deviation sum of squares is obtained by squaring the difference of expected values (Table 2.56) and experimentally obtained results (Table 2.54). By partial differentiation of ai bj and m and bringing it down to zero, we get the following system of normal equations ... 

The system above contains N equations and consequently it will produce a single real solution for Pq, Pi,P (n unknowns). It is necessary to specify that the size of the statistical selection, here represented by Ne, must be appreciable. Moreover, whenever the regression coefficients have to be identified, Ne must be greater than n. This system (5.9) is frequently called system of normal equations [5.4, 5.12-5.14]. 

With the index o will be marked these coordinates or parameters which result from the equations (27) provided that the approximations Yo should be introduced into them. The coefficients of the design matrix A(Yo) are derived completely in Klumb (1990). Because of the miiumization of Q (31) we shall obtain a correction dY of the parameters by the system of normal equations... 

Because of the dependence on the axial parameters a and r this system of normal equations has a defect of the dimension two. By introduction of the linearized condition equations (19) and (20) this defect will be eliminated. The searched parameters Y result from a iterative Newton—process. Based on the start parameters Yo in a general i—th step of the iteration the actual solutions Yi = Yi-i + dYi will be calculated. After a new bnearization of A(Yi) a new least square solution will be determined. So the iteration of the unknowns reads as follows... 

In the rest of this chapter other approaches to the solution of this problem will be discussed. For immediate solution either the system of normal equations or the overdetermined system may be used, taking into account the possibilities for use of sparseness of the coefficient matrix from the point of view of effective use of modern computers. 

In addition to solution of the systems of equations (2) or (3) it is very often necessary to compute variances and covariances associated with the regression coefficients in order to estimate the accuracy of the results. Because of this values for some or all of the elements of the inverse,, of the matrix of coefficients of the system of normal equations have to be computed. 

The condition number of the matrix N is the square of the condition number of the matrix A and this brings into question the accuracy of the solution of the system of normal equations. 

When the computation is done with finite precision, using normal equations is not always recommended because for some matrices A, higher precision of computation is required in order to solve the system of normal equations correctly. For example, let us consider the problem of least-squares fitting by a straight line, Cq - - cix, to the set of data, shown in the accompanying tabulation. 

---