Linear Least Squares Estimation

Let us consider first the most general case of the multiresponse linear regression model represented by Equation 3.2. Namely, we assume that we have N measurements of the m-dimensional output vector (response variables), y , M.N. 

The computation of the parameter estimates is accomplished by minimizing the least squares (LS) objective function given by Equation 3.8 which is shown next 

Solution of the above linear equation yields the least squares estimates of the parameter vector, k,  

For the single response linear regression model (w=l), Equations (3.17a) and (3.17b) reduce to 

In practice, the solution of Equation 3.16 for the estimation of the parameters is not done by computing the inverse of matrix A. Instead, any good linear equation solver should be employed. Our preference is to perform first an eigenvalue decomposition of the real symmetric matrix A which provides significant additional information about potential ill-conditioning of the parameter estimation problem (see Chapter 8). 

---

The amounts of EMA and HEMA (derivatized as MEMA) are determined based upon external standard calibration. A nonweighted linear least-squares estimate of... 

The amount of NIPA is determined based upon external standard calibration. A non-weighted linear least-squares estimate of the calibration curve is used to calculate the amount of NIPA in the unknowns. The response of any given sample must not exceed the response of the most concentrated standard. If this occurs, dilution of the sample will be necessary. 

By constructing a plot of S(t,) versus Xvdt, we can visually identify distinct time periods during the culture where the specific uptake rate (qs) is "constant" and estimates of qs are to be determined. Thus, by using the linear least squares estimation capabilities of any spreadsheet calculation program, we can readily estimate the specific uptake rate over any user-specified time period. The estimated... 

The major disadvantage of the integral method is the difficulty in computing an estimate of the standard error in the estimation of the specific rates. Obviously, all linear least squares estimation routines provide automatically the standard error of estimate and other statistical information. However, the computed statistics are based on the assumption that there is no error present in the independent variable. 

A suitable transformation of the model equations can simplify the structure of the model considerably and thus, initial guess generation becomes a trivial task. The most interesting case which is also often encountered in engineering applications, is that of transformably linear models. These are nonlinear models that reduce to simple linear models after a suitable transformation is performed. These models have been extensively used in engineering particularly before the wide availability of computers so that the parameters could easily be obtained with linear least squares estimation. Even today such models are also used to reveal characteristics of the behavior of the model in a graphical form. 

Initial estimates for the parameters can be readily obtained using linear least squares estimation with the transformed model. 

In engineering we often encounter conditionally linear systems. These were defined in Chapter 2 and it was indicated that special algorithms can be used which exploit their conditional linearity (see Bates and Watts, 1988). In general, we need to provide initial guesses only for the nonlinear parameters since the conditionally linear parameters can be obtained through linear least squares estimation. 

Unweighted linear least-squares estimates of these parameters are shown in Table I. The negative parameter estimates of the single-site model are of no... 

The approximate 95 % confidence region for the non-linear least-squares estimates using these data is shown in Fig. 32. The surface represented in this figure is the contour of the sums of squares surface, which has the value Sc given by Eq. (56). 

P equals the number of components, and e is normal with standard deviation o. ) Then, Op can be a linear operator (on the y ), such as that associated with linear least squares estimation (or, for non-interfering peaks, 0-1-03) — and the... 

The rate constants (together with the model and initial concentrations) define the matrix C of concentration profiles. Earlier, we have shown how C can be computed for simple reactions schemes. For any particular matrix C we can calculate the best set of molar absorptivities A. Note that, during the fitting, this will not be the correct, final version of A, as it is only based on an intermediate matrix C, which itself is based on an intermediate set of rate constants (k). Note also that the calculation of A is a linear least-squares estimate its calculation is explicit, i.e., noniterative. 

Alapati S, Kabala ZJ (2000) Recovering the release history of a groundwater contaminant via the non-linear least-squares estimation. Hydrol Proc 14 1003-1016... 

Hartley, H. O.. and A. Booker, Non-linear least squares estimation, Ann. Math. Statist., 36, 638-650 (1965). 

R. I. Jennrich, Asymptotic properties of non-linear least squares estimators. Ann Math Stat 40 633-643 (1969). 

---