Least squares orthogonal

Kuo, J. E., Wang, H., and Pickup, S., Multidimensional Least-Squares Smoothing Using Orthogonal Polynomials, Anal. Chem. 63, 1991, 630-635. 

One of the earliest interpretations of latent vectors is that of lines of closest fit [9]. Indeed, if the inertia along v, is maximal, then the inertia from all other directions perpendicular to v, must be minimal. This is similar to the regression criterion in orthogonal least squares regression which minimizes the sum of squared deviations which are perpendicular to the regression line (Section 8.2.11). In ordinary least squares regression one minimizes the sum of squared deviations from the regression line in the direction of the dependent measurement, which assumes that the independent measurement is without error. Similarly, the plane formed by v, and Vj is a plane of closest fit, in the sense that the sum of squared deviations perpendicularly to the plane is minimal. Since latent vectors v, contribute... 

The purpose of Partial Least Squares (PLS) regression is to find a small number A of relevant factors that (i) are predictive for Y and (u) utilize X efficiently. The method effectively achieves a canonical decomposition of X in a set of orthogonal factors which are used for fitting Y. In this respect PLS is comparable with CCA, RRR and PCR, the difference being that the factors are chosen according to yet another criterion. 

Chen, S., Cowan, C. F. N., and Grant, P. M., Orthogonal least squares learning algorithm for radial basis function networks, IEEE Trans. Neur. Net. 2(2), 302-309 (1991). 

In this case, in which there are errors in both variables, y and x, the resulting error e2x+y has to be minimized (see Fig. 6.5) and orthogonal least squares fitting must be carried out as will be shown in Sect. 6.2.4. 

Danzer K, Wagner M, Fischbacher C (1995) Calibration by orthogonal and common least squares - theoretical and practical aspects. Fresenius J Anal Chem 352 407... 

Using the Q-R orthogonal factorization method described in Chapter 4, the constrained weighted least-squares estimation problem (5.4) is transformed into an unconstrained one. The following steps are required ... 

Figure 5.1 The least-square estimate y of the solution to equation (5.1.4) is the orthogonal projection of the observation vector y onto the column-space 1 a2, of matrix A.

The determination of the projections can be regarded as a linear least-squares fit only now we have an orthogonal set of vectors V =, as in Figure 5-28, rather than a general set of non-orthogonal vectors in F in the equivalent Figure 4-12. The projected test vector tproj is a linear combination of the vectors V. 

The normal residuals R are orthogonal to the space C, defined by the projection of the column vectors in US or Y into C. This is a straightforward linear least-squares calculation, equivalent to Figure 4-10. C(kc) is the closest the space C gets to the vectors US or Y. 

It is a direct result of the orthogonality property of linear least squares (Section III,A,1) that the crude sum of squares may be broken into two parts as follows ... 

A section has been added to Chapter 1 on the distinction between analytic vs. enumerative studies. A section on mixture designs has been added to Chapter 9. A new chapter on the application of linear models and matrix least squares to observational data has been added (Chapter 10). Chapter 13 attempts to give a geometric feel to concepts such as uncertainty, information, orthogonality, rotatability, extrapolation, and rigidity of the design. Finally, Chapter 14 expands on some aspects of factorial-based designs. 

A crystallographic example of optimization would be the minimization of a least-squares or a negative log-likelihood residual as the objective function, using fractional or orthogonal atomic coordinates as the variables. The values of the variables that optimize this objective function constitute the final crystallographic model. However, due to the... 

There are numerous ways in which to orthonormalize a basis. Here we choose to employ the symmetric orthonormalization procedure described by Lowdin (51), which has the benefit over other orthogonality procedures that the new basis is as close as possible, in a least squares sense, to the original basis (52)... 

---