Linear estimates

Given a matrix Amxn of known coefficients with m n and a vector ym of observations or data, an unknown or model vector x of parameters is sought which fulfills the condition of the model 

This equation has in general no solution because the data vector y should represent a linear combination of the n column vectors aua2.a of the matrix A, for 

We therefore make the assumption that the sample data gathered in vector y are only our best estimates of the real (population) values which justifies the bar on the symbol as representing measured values. This notation contradicts the standard usage, but is consistent with the basic definitions of Chapter 4. Indeed, for an unbiased estimate, we can still write that 

The problem is therefore recast as a search for a population vector statistics j, of which the measurement y is an estimate, and which satisfy the model, i.e., 

None of the population parameters x and y can be found since their determination would require the whole range of attainable values to be measured. The least-square criterion provides estimates x and of x and y, respectively, which also satisfy the model, i.e., 

---

Cropley made general recommendations to develop kinetic models for compUcated rate expressions. His approach includes first formulating a hyperbolic non-linear model in dimensionless form by linear statistical methods. This way, essential terms are identified and others are rejected, to reduce the number of unknown parameters. Only toward the end when model is reduced to the essential parts is non-linear estimation of parameters involved. His ten steps are summarized below. Their basis is a set of rate data measured in a recycle reactor using a sixteen experiment fractional factorial experimental design at two levels in five variables, with additional three repeated centerpoints. To these are added two outlier... 

Substituting the expression for ko in the equation above Rule 5, the expression is ready for non-linear estimation of the coefficients. 

Rule 10. Adjust exponents to their nearest sensible value and run the non-linear estimation once more to get the best value for E, and K s. 

It is quite a simple matter to generalize the simple prediction problem just discussed to the situation where we want to obtain the best (in the sense of minimum mean square error) linear estimate of one random variable fa given the value of another random variable fa. The quantity to be minimized is thus... 

When the model equations are linear functions of the parameters the problem is called linear estimation. Nonlinear estimation refers to the more general and most frequently encountered situation where the model equations are nonlinear functions of the parameters. 

These problems refer to models that have more than one (w>l) response variables, (mx ) independent variables and p (= +l) unknown parameters. These problems cannot be solved with the readily available software that was used in the previous three examples. These problems can be solved by using Equation 3.18. We often use our nonlinear parameter estimation computer program. Obviously, since it is a linear estimation problem, convergence occurs in one iteration. 

When considering analytic description, asymptotically optimal estimates are of importance. Asymptotically optimal estimates assume infinite duration of the observation process for fjv —> oo. For these estimates an additional condition for amplitude of a leap is superimposed The amplitude is assumed to be equal to the difference between asymptotic and initial values of approximating function a = <2(0, xo) — <2(oc,Xq). The only moment of abrupt change of the function should be determined. In such an approach the required quantity may be obtained by the solution of a system of linear equations and represents a linear estimate of a parameter of the evolution of the process. 

In this section we develop the best linear estimate of x for the general linear measurement model... 

In the vicinity of the minimum, the H should be positive-definite. This may not be the case everywhere in which case there is a small but real danger of iterating towards a saddle instead of the minimum. It is therefore highly advisable, especially when the data scatter about the best-fit straight line, plane, or hyper-plane, to use the best possible initial estimate. Most commonly, one of the linear estimates (Section 5.1) will be good enough. 

Numerous examples of applications of nonlinear least squares to kinetic-data analysis have been presented (K7, K8, L3, L4, M7, P2) an exhaustive tabulation of references would, at this point, approach 100 entries. Typical results of a nonlinear estimation and comparison to linear estimates are shown in Table I and discussed in Section III,A,2. Many estimation problems exist, however, as typified in part by Fig. 7. This is the sum-of-squares surface obtained at fixed values of Ks and Ku in the rate equation used for the catalytic hydrogenation of mixed isooctenes (M7)... 

Decision levels and detection limits are relatively easy to define and evaluate for simple" (zero dimensional) measurements. The transition to higher dimensions and multiple components introduces a number of complications and added assumptions related to the number and identity of components, shapes and parameters of calibration functions and spectra, and distributional consequences of non-linear estimation. 

All of the methods described in this chapter are linear methods because each element of o(x) can be obtained by a linear combination of the elements of i(x). In the continuous regime, a linear estimate d(x) can always be expressed by the integral... 

Parameter estimation is rooted in several scientific areas with their own preferences and approaches. While linear estimation theory is a nice chapter of mathematical statistics (refs. 1-3), practical considerations are equally important in nonlinear parameter estimation. As emphasised by Bard (ref. 4), in spite of its statistical basis, nonlinear estimation is mainly a variety of computational algorithms which perform well on a class of problems but may fail on some others. In addition, most statistical tests and estimates of... 

D. W. Marquardt, Generalized inverses, ridge regression, biased linear estimation and nonlinear estimation. Technometrics, 12, 1970, 591-612. 

Clearly, the model cannot be estimated by ordinary least squares, since there is an autocorrelated disturbance and a lagged dependent variable. The parameters can be estimated consistently, but inefficiently by linear instrumental variables. The inefficiency arises from the fact that the parameters are overidentified. The linear estimator estimates seven functions of the five underlying parameters. One possibility is a GMM estimator. Let v, = g, -(y+< >)g,-i + (y< >)g, 2. Then, a GMM estimator can be defined in terms of, say, a set of moment equations of the fonn E[v,w,] = 0, where w, is current and lagged values of x and z. A minimum distance estimator could then be used for estimation. 

Each frequency track has a birth and death index and dt such that bt denotes the first DFT block at which f0i is present ( active ) and d, the last (each track is then continuously active between these indices). Frequencies are expressed on a log-freuency scale, as this leads to linear estimates of the pitch curve (see [Godsill, 1993] for comparison with a linear-frequency scale formulation). The model equation for the measured log-frequency tracks fa[n is then ... 

---