General formalisms for linear least-squares

The following description follows closely the treatment given by Hamilton (1964). Suppose we have n experimental observations 

If each observation f is subject to a random error r.h the observational equations may be written as 

The matrix A is called the design matrix. Its elements are the derivatives 

Given the n observations, our aim is to obtain the best estimates X of the m unknown parameters to be determined. Gauss proposed the minimization of the sum of the squares of the discrepancies, defining the error function S, which, after assignment of a weight w, to each of the observations, is given by 

Suppose the observations are correlated with the correlation coefficient ytj describing the correlation between the ith and jth observations. The variances and covariances of the observations will then be given by the variance-covariance matrix with elements otOflij, where tr, is the standard deviation of the ith observation. The error function S for a set of correlated observations is defined as 

---