Elimination methods for solving linear systems

With these basic definitions in hand, we now begin to consider the solution of the linear system Ax = b, in which x, b and is an A x A/ real matrix. We consider here elimination methods in which we convert the linear system into an equivalent one that is easier to solve. These methods are straightforward to implement and work generally for any linear system that has a unique solution however, they can be quite costly (perhaps prohibitively so) for large systems. Later, we consider iterative methods that are more effective for certain classes of large systems. 

We wish to develop an algorithm for solving the set of iV linear equations 

The basic strategy is to define a sequence of operations that converts file original system into a simpler, but equivalent, one that may be solved easily. 

We first note that we can select any two equations, say j and k, and add them to obtain another one that is equally valid. 

If equation j is satisfied, and the equation obtained ty summing j and k is satisfied, it follows that equation k must be satisfied as well. We are thus fiee to replace in our system 

---

With Gaussian elimination and partial pivoting, we have a method for solving linear systems that either finds a solution or fails under conditions in which no unique solution exists. In this section, we consider at more depth the question of when a linear system possesses a real solution (existence) and if so, whether there is exactly one (uniqueness). These questions are vitally important, for linear algebra is the basis upon which we build algorithms for solving nonlinear equations, ordinary and partial differential equations, and many other tasks. 

The books by Gelfand (1967), Samarskii and Nikolaev (1989) cover in full details the general theory of linear difference equations. Sometimes the elimination method available for solving various systems of algebraic equations is referred to, in the foreign literature, as Thomas algorithm and this... 

As mentioned, Cramer s formula is only suitable for solving equation systems with two or three unknowns the advantage using this solution method is the clear systematics when keying on a pocket calculator. To solve linear equation systems with more unknowns than three, we can, for example, use Gauss elimination. 

In principle, the task of solving a linear algebraic systems seems trivial, as with Gauss elimination a solution method exists which allows one to solve a problem of dimension N (i.e. N equations with N unknowns) at a cost of O(N ) elementary operations [85]. Such solution methods which, apart from roundoff errors and machine accuracy, produce an exact solution of an equation system after a predetermined number of operations, are called direct solvers. However, for problems related to the solution of partial differential equations, direct solvers are usually very inefficient Methods such as Gauss elimination do not exploit a special feature of the coefficient matrices of the corresponding linear systems, namely that most of the entries are zero. Such sparse matrices are characteristic of problems originating from the discretization of partial or ordinary differential equations. As an example, consider the discretization of the one-dimensional Poisson equation... 

HS, S, HCCU, CO3, RR NH, RR NCOO", H+, OH- and H2O. Hence there are twenty-three unknowns (m and Yj for all species except water plus x ). To solve for trie unknowns there are twenty-three independent equations Seven chemical equilibria, three mass balances, electroneutrality, the use of Equation (6) for the eleven activity coefficients and the phase equilibrium for xw. The problem is one of solving a system of nonlinear algebraic equations. Brown s method (21, 22) was used for this purpose. It is an efficient procedure, based on a partial pivoting technique, and is analogous to Gaussian elimination in linear systems of equations. 

A system of two linear equations, such as 2x + 3y = 31 and 5x -y = 1 is usually solved by elimination or substitution. (Refer to Algebra For Dummies if you want a full explanation of each type of solution method.) For the problems in this chapter, I use the substitution method, to solve for a variable. This means that you change the format of one of the equations so that it expresses what one of the variables is equal to in terms of the other, and then you substitute into the other equation. For example, you solve for y in terms of x in the equation 3x + y = 11 if you subtract 3x from each side and write the equation as y = 11 - 3x. 

Special linear systems arise from the Poisson equation, d uldx + d uldy = f x, y) on a rectangle, 0 Laplace equation of Section II.A is a special case where fix, y) = 0.] If finite differences with N points per variable replace the partial derivatives, the resulting linear system has equations. Such systems can be solved in 0(N log N) flops with small overhead by special methods using fast Fourier transform (FFT) versus an order of AC flops, which would be required by Gaussian elimination for that special system. Storage space also decreases from 2N to units. Similar saving of time and space from O(N ) flops, 2N space units to 0(N log N) flops and space units is due to the application of FFT to the solution of Poisson equations on a three-dimensional box. 

A set of linear equations can be solved by a variety of procedures. In principle the method of determinants is applicable to any number of equations but for large systems other methods require much less numerical effort. The method of Gauss illustrated here eliminates one variable at a time, ends up with a single variable and finds all the roots by a reverse procedure. 

---