Orthogonal decomposition algorithm

The well known matrix decomposition theorem states that a positive square matrix P can be decomposed as 

Since the correlation matrix is symmetric and positive definite under the assumption that it is invertible, it can be expressed as 

To derive the least squares solution using the orthogonal decomposition result, we rewrite Equation (3.4) by inserting T T 

Taking advantage of the orthogonality of the matrix W, the least squares problem can be solved in terms of the auxiliary parameter vector g and the transformed data matrix W. That is, based on Equation (3.13), the vector g can be estimated from the least squares solution as 

Since Wd is a diagonal matrix, Equation (3.16) does not require matrix inversion and can be solved on an element-by-element basis. The least squares estimate of the original parameter vector 6 is then obtained using the relation 

---

A RECURSIVE PROPER ORTHOGONAL DECOMPOSITION ALGORITHM FOR FLOW CONTROL PROBLEMS... 

Sahoo, D., S. Park, D. Wee, A. M. Annaswamy, and A. F. Ghoniem. 2002. A recursive proper orthogonal decomposition algorithm for flow control problems. Adaptive Control Laboratory, MIT. Technical Report 0208. 

Chapter 3 presents the development of the PRESS statistic as a criterion for structure selection of dynamic process models which are linear-in-the-parameters. Computation of the PRESS statistic is based on the orthogonal decomposition algorithm proposed by Korenberg et al. (1988) and can be viewed as a by-product of their algorithm since very little additional computation is required. We also show how the PRESS statistic can be used as an efficient technique for noise model development directly fi-om time series data. 

This chapter consists of six sections. Section 3.2 introduces the orthogonal decomposition algorithm proposed by Korenberg et al. (1988). Section 3.3 describes the concept of the PRESS statistic. Section 3.4 shows how to compute the PRESS using the orthogonal decomposition algorithm. Section 3.5 applies the PRESS statistic to the problem of model structure selection for dynamic systems. Section 3.6 shows how the PRESS statistic... 

Computation of the true prediction errors e (A ), k = 1,2,is a tremendous task in dynamic system identification where we typically face a large amount of data (M) and possibly high dimensionality (n) of the parameter vector 6. It will be shown here that by using the orthogonal decomposition algorithm, the computation of the PRESS residuals is simplified to an extent that its calculation can be viewed as a byproduct of the algorithm. The following theorem presents the cornerstone for computation of the PRESS statistic. 

Although we could extend the least squares estimation results for SISO systems directly to the p-input, g-output multivariable case, we prefer to treat these systems as q multi-input, single-output (MISO) systems. This way, we can take full advantage of the orthogonal decomposition algorithm developed in Chapter 3 for parameter estimation and structure selection of the p subsystems associated with each of the q outputs. This will be illustrated using an industrial data set in Chapter 5. 

The power algorithm [21] is the simplest iterative method for the calculation of latent vectors and latent values from a square symmetric matrix. In contrast to NIPALS, which produces an orthogonal decomposition of a rectangular data table X, the power algorithm decomposes a square symmetric matrix of cross-products X which we denote by C. Note that Cp is called the column-variance-covariance matrix when the data in X are column-centered. 

This decomposition algorithm forms the first half of the discrete orthogonal wavelet transform. The reconstruction of coefficients 35+U from coefficients andy,- is performed with... 

Thus the decomposition algorithms is the same as in the case of discrete orthogonal wavelet transform... 

For the calculation of cofactors we use algorithms based on work by Lowdin [16], and Prosser and Hagstrom [25,26]. An overview of the theory of determinants, cofactors, adjugates and compound matrices can be found in a book by Aitken [17]. The symmetry and possible orthogonality in the orbital spaces give rise to a block-structure in the overlap matrices. This structure is exploited [22,27] to minimise the size of the matrices in the L-d-R decomposition, described below, an n3 process for each matrix. 

A little more expensive [n (m + I7nl3) flops and 2mn space versus n (m—nl3) and mn in the Householder transformation] but completely stable algorithm relies on computing the singular value decomposition (SVD) of A. Unlike Householder s transformation, that algorithm always computes the least-squares solution of the minimum 2-norm. The SVD of an m x n matrix A is the factorization A = ITEV, where U and V are two square orthogonal matrices (of sizes mxm and nxn, respectively), U U = Im, y V = In, and where the m x n matrix S... 

A similar 0(N ) method, presented by Angeles and Ma in [2], uses the concept of an orthogonal complement to construct the joint space inertia matrix. The Cholesky decomposition of this matrix is used in solving the appropriate linear system for the joint accelerations. The computational complexity of this algoithm is slightly better than that in [42], but the algorithm is still not the most efficient It, too, is restricted to configurations of simple revolute and prismatic joints. 

---