Time series modeling least squares

An alternative SPM framework for autocorrelated data is developed by monitoring variations in time series model parameters that are updated at each new measurement instant. Parameter change detection with recursive weighted least squares was used to detect changes in the parameters and the order of a time series model that describes stock prices in financial markets [263]. Here, the recursive least squares is extended with adaptive forgetting. 

In this section, we present an iterative algorithm in the spirit of the generalized least squares approach (Goodwin and Payne, 1977), for simultaneous estimation of an FSF process model and an autoregressive (AR) noise model. The unique features of our algorithm are the application of the PRESS statistic introduced in Chapter 3 for both process and noise model structure selection to ensure whiteness of the residuals, and the use of covariance matrix information to derive statistical confidence bounds for the final process step response estimates. An important assumption in this algorithm is that the noise term k) can be described by an AR time series model given by... 

The system identification step in the core-box modeling framework has two major sub-steps parameter estimation and model quality analysis. The parameter estimation step is usually solved as an optimization problem that minimizes a cost function that depends on the model s parameters. One choice of cost function is the sum of squares of the residuals, Si(t p) = yi(t) — yl(t p). However, one usually needs to put different weights, up (t), on the different samples, and additional information that is not part of the time-series is often added as extra terms k(p). These extra terms are large if the extra information is violated by the model, and small otherwise. A general least-squares cost function, Vp(p), is thus of the form... 

A key factor in modeling is parameter estimation. One usually needs to fit the established model to experimental data in order to estimate the parameters of the model both for simulation and control. However, a task so common in a classical system is quite difficult in a chaotic one. The sensitivity of the system s behavior to the initial conditions and the control parameters makes it very hard to assess the parameters using tools such as least squares fitting. However, efforts have been made to deal with this problem [38]. For nonlinear data analysis, a combination of statistical and mathematical tests on the data to discern inner relationships among the data points (determinism vs. randomness), periodicity, quasiperiodicity, and chaos are used. These tests are in fact nonparametric indices. They do not reveal functional relationships, but rather directly calculate process features from time-series records. For example, the calculation of the dimensionality of a time series, which results from the phase space reconstruction procedure, as well as the Lyapunov exponent are such nonparametric indices. Some others are also commonly used ... 

Since this monograph is devoted only to the conception of mathematical models, the inverse problem of estimation is not fully detailed. Nevertheless, estimating parameters of the models is crucial for verification and applications. Any parameter in a deterministic model can be sensibly estimated from time-series data only by embedding the model in a statistical framework. It is usually performed by assuming that instead of exact measurements on concentration, we have these values blurred by observation errors that are independent and normally distributed. The parameters in the deterministic formulation are estimated by nonlinear least-squares or maximum likelihood methods. 

Matis, J. and Hartley, H., Stochastic compartmental analysis Model and least squares estimation from time series data, Biometrics, Vol. 27, 1971, pp. 77-102. 

The standard method of least-square fitting the model parameters from experimental data is only applicable if the deviation of estimated parameter-based model results from measured data can be explicitly calculated. To obtain the matrix of model error changes with the individual changes of each parameter for series of time dependent data points as measured in a... 

Assuming that we have measured a series of concentrations over time/ we can define a model structure and obtain initial estimates of the model parameters. The objective is to determine an estimate of the parameters (CLe, Vd) such that the differences between the observed and predicted concentrations are comparatively small. Three of the most commonly used criteria for obtaining a best fit of the model to the data are ordinary least squares (OLS)/ weighted least squares (WLS)/ and extended least squares (ELS) ELS is a maximum likelihood procedure. These criteria are achieved by minimizing the following quantities/... 

Stochastic identification techniques, in principle, provide a more reliable method of determining the process transfer function. Most workers have used the Box and Jenkins [59] time-series analysis techniques to develop dynamic models. An introduction to these methods is given by Davies [60]. In stochastic identification, a low amplitude sequence (usually a pseudorandom binary sequence, PRBS) is used to perturb the setting of the manipulated variable. The sequence generally has an implementation period smaller than the process response time. By evaiuating the auto- and cross-correlations of the input series and the corresponding output data, a quantitative model can be constructed. The parameters of the model can be determined by using a least squares analysis on the input and output sequences. Because this identification technique can handle many more parameters than simple first-order plus dead-time models, the process and its related noise can be modeled more accurately. 

Although this model is very simple, it has the beneficial property that the estimation of its parameters can be performed using least-squares analysis. In many respects, this model is very similar to the autoregressive model previously considered for time series analysis. 

---