Geometrical interpretation of matrix products

The matrix-to-vector product can be interpreted geometrically as a projection of a pattern of points upon an axis. As we have seen in Section 29.4 on matrix products, if X is an nxp matrix and if v is a p vector then the product of X with v produces the n vector s  

The matrix X defines a pattern P of n points, e.g. x, in which are projected perpendicularly upon the axis v. The result, however, is a point s in the dual space S . This can be understood as follows. The matrix X is of dimension nxp and the vector V has dimensions p. The dimension of the product s is thus equal to n. This means that s can be represented as a point in S . The net result of the operation is that the axis v in 5 is imaged by the matrix X as a point s in the dual space 5 . For every axis v in 5 we will obtain an image s formed by X in the dual space. In this context, we use the word image when we refer to an operation by which a point or axis is transported into another space. The word projection is reserved for operations which map points or axes in the same space [11]. The imaging of v in S into s in S is represented geometrically in Fig. 29.9a. Note that the patterns of points P and P are represented schematically by elliptic envelopes. 

In a similar way, we can think of X as representing a pattern PP of p points, e.g. Xj, in S which can be projected upon an axis u and which results into a point I (not to be confounded with the sum vector 1) in the complementary (or dual) space S (Fig. 29.9b)  

Using the same argument as above, we can see that the product is of dimension p, since the matrix X has dimensions pxn and the vector u possesses dimension n. Hence, the vector can be imaged as a single point in the dual space S . We state that the vector u in S is imaged by the matrix X into the point 1 in the dual space S . 

Geometrical interpretation of multiple linear regression (MLR). The pattern of points in S representing a matrix X is projected upon a vector b, which is imaged in 5 by the point y. The orientation of the vector b is determined such that the distance between y and the given y is minimal. 

---