Polynomial approximation and basis transformation

Suppose we have a set of n data points (X, yj), a calibration line, for example. The data are plotted in Fig. 6. 

We wish to describe y as a function of x. A straight line seems okay as a first approximation. In that case the model is a first-order polynomial  

If a second-order is not sufficient we try a third-order etc. If we use a polynomial of order n - 1, we are sure to perfectly describe the data. There would be no degrees of freedom left. Fig. 7 shows the first orders of approximation of the data of Fig. 6. 

In general, a perfect description is not what we aim for. As the responses have not been measured with infinite precision, a perfect description would go beyond describing the process we set out to observe. It would describe the measurement error as well. In a polynomial approximation, we would typically stop at an order well below the limiting n - 1. In other words, we suspect the higher-order terms to be representing noise. That is a general principle we will also encounter in Fourier. 

For the calculation of the coefficients in our polynomial model we use linear regression, i.e. a least squares projection of the data onto the model. This is very easy to write down in matrix notation. Our model becomes  

---