Moore-Penrose generalized inverse

The determination of output weights between hidden and output layers is to find the least-square solution to the given linear system. The minimum norm least-square solution to hnear system (1) is M Y, where M is the Moore-Penrose generalized inverse of matrix M. The minimum norm least-square solution is unique and has the smallest norm among the least-square solutions. 

Since C is not a square matrix (it is 4 X 3), the unknown X, Y, Z coordinates can be solved by using the Moore-Penrose generalized inverse, as follows ... 

To calculate the correction vector, Eq. 6.19 is solved via matrix inversion, e.g. by calculating the Moore-Penrose generalized matrix inverse ... 

It should be noted that in the case of a singular matrix A, the dimensions of V and A are pxr and rxr, respectively, where r is smaller than p. The expression in eq. (29.53) allows us to compute the generalized inverse, specifically the Moore-Penrose inverse, of a symmetric matrix A from the expression ... 

To be more precise, only the 1,3 generalized inverses solve the least squares problem, where 1 and 3 refer to the four Moore-Penrose conditions. 

In algebra, a number multiplied by its inverse results in a value of 1. In matrix algebra, the inverse of a square matrix (denoted by a superscript T) multiplied by itself results in the identity matrix. In other words, the inverse of X is the matrix X-1 such that XX-1 = X-1X = I. Two matrices are said to be orthogonal or independent if XYT = I. The inverse of an orthogonal matrix is its transpose. Not all matrices can be inverted. However, one condition for inversion is that the matrix must be square. Sometimes an inverse to a matrix cannot be found, particularly if the matrix has a number of linearly dependent column. In such a case, a generalized estimate of inverted matrix can be estimated using a Moore Penrose inverse (denoted as superscript e.g., X-). 

Alternatively, a weighted generalized inverse G may be used instead of the Moore-Penrose pseudo-inverse ... 

---