The Linear Least Squares Objective Function

Given N measurements of the response variables (output vector), the parameters are obtained by minimizing the Linear Least Squares (LS) objective function which is given below as the weighted stun of squares of the residuals, namely, 

In this case we minimize the sum of squares of errors (SSE) without any weighting factor, i.e., we use Q,=I and Equation 3.6 reduces to 

This choice of Qi yields maximum likelihood estimates of the parameters if the error terms in each response variable and for each experiment (eu, i=l.N j=l.w) are all identically and independently distributed (i.i.d) normally with zero mean and variance, o . Namely, (e,) = 0 and COV(s,) = a I where I is the mxm identity matrix. 

In this case we minimize a weighted sum of squares of residuals with constant weights, i.e., the user-supplied weighting matrix is kept the same for all experiments, Q,=Q for all i=l.N and Equation 3.7 reduces to 

---