Regression, parameter estimation independent variable

One assumption until now has been that the dependent and independent variables are measured without error. The impact of measurement error on the regression parameter estimates depends on whether the error affects the dependent or independent variable. When Y has measurement error, the effect on the regression model is not problematic if the measurement errors are uncorrelated and unbiased. In this case the linear model becomes... 

With linear models, exact inferential procedures are available for any sample size. The reason is that as a result of the linearity of the model parameters, the parameter estimates are unbiased with minimum variance when the assumption of independent, normally distributed residuals with constant variance holds. The same is not true with nonlinear models because even if the residuals assumption is true, the parameter estimates do not necessarily have minimum variance or are unbiased. Thus, inferences about the model parameter estimates are usually based on large sample sizes because the properties of these estimators are asymptotic, i.e., are true as n —> oo. Thus, when n is large and the residuals assumption is true, only then will nonlinear regression parameter estimates have estimates that are normally distributed and almost unbiased with minimum variance. As n increases, the degree of unbiasedness and estimation variability will increase. 

The models in chemical kinetics usually contain a number of unknown parameters, whose values should be determined from experimental data. Regression analysis is a powerful and objective tool in the estimation of parameter values. The task in regression analysis can be stated as follows the value of the dependent variable (y) is predicted by the model a function (/), contains independent variables (x) and parameters (/ ). The independent variable is measured experimentally, at different conditions, i.e. at different values of the independent variables (x). The goal is to find such numerical values of the parameters (/ ) that the model gives the best possible agreement with the experimental data. Typical independent variables are reaction times, concentrations, pressures and temperatures, while molar amounts, concentrations, molar flows... 

To verify such a steric effect a quantitative structure-property relationship study (QSPR) on a series of distinct solute-selector pairs, namely various DNB-amino acid/quinine carbamate CSPpairs with different carbamate residues (Rso) and distinct amino acid residues (Rsa), has been set up [59], To provide a quantitative measure of the effect of the steric bulkiness on the separation factors within this solute-selector series, a-values were correlated by multiple linear and nonlinear regression analysis with the Taft s steric parameter Es that represents a quantitative estimation of the steric bulkiness of a substituent (Note s,sa indicates the independent variable describing the bulkiness of the amino acid residue and i s.so that of the carbamate residue). For example, the steric bulkiness increases in the order methyl < ethyl < n-propyl < n-butyl < i-propyl < cyclohexyl < -butyl < iec.-butyl < t-butyl < 1-adamantyl < phenyl < trityl and simultaneously, the s drops from -1.24 to -6.03. In other words, the smaller the Es, the more bulky is the substituent. The obtained QSPR equation reads as follows ... 

A correlation is based on the proposition that, for a particular population S, the property y is related to one or several independent variables, often called predictors (xi, X2,..., xt), the correlation is based on a particular function y = f(x, X2,. , x. ) and a set of adjustable parameters (co, ci, C2,...). The optimal parameter values for ci are extracted from the data, which minimizes the differences between data and predictions. The principal usefulness of a regression formula, which has been proven valid only in S, is in its success in estimating the property in the bigger population of P (table 5.3). Some investigators would immediately proceed to export this correlation from the training set S to the larger population P without further assurances that it would still work. 

Whether or not any of the possible models are statistically significant is based on several important statistical parameters. Among them are the correlation coefficient (R), the standard error of estimate (.v), the value of the F-test of the overall significance (F), the values of f-test of significance of individual regre.ssion (t) and the cross-correlation coefficients between the independent variables employed in the same regression equation 27]. 

Within the procedure, the major part of the code is virtually identical to the code that would be used to obtain the usual least-squares regression parameters for y = mx + b, namely, obtaining the sum of x values, the sum of squares of x values, etc. The difference is that pairs of values are used the mean value of each pair of x values or of y values is used as the independent variable or dependent variable, respectively, and the estimate of the standard errors for the two sets of data is obtained from the differences between pairs. 

Regression analysis is often employed to fit experimental data to a mathematical model. The purpose may be to determine physical properties or constants (e.g., rate constants, transport coefficients), to discriminate between proposed models, to interpolate or extrapolate data, etc. The model should provide estimates of the uncertainty in calculations from the resulting model and, if possible, make use of available error in the data. An initial model (or models) may be empirical, but with advanced knowledge of reactors, distillation columns, other separation devices, heat exchangers, etc., more sophisticated and fundamental models can be employed. As a starting point, a linear equation with a single independent variable may be initially chosen. Of importance, is the mathematical model linear In general, a function,/, of a set of adjustable parameters, 3y, is linear if a derivative of that function with respect to any adjustable parameter is not itself a function of any other adjustable parameter, that is. 

The Matlab Simulink Model was designed to represent the model stmctuie and mass balance equations for SSF and is shown in Fig. 6. Shaded boxes represent the reaction rates, which have been lumped into subsystems. To solve the system of ordinary differential equations (ODEs) and to estimate unknown parameters in the reaction rate equations, the inter ce parameter estimation was used. This program allows the user to decide which parameters to estimate and which type of ODE solver and optimization technique to use. The user imports observed data as it relates to the input, output, or state data of the SimuUnk model. With the imported data as reference, the user can select options for the ODE solver (fixed step/variable step, stiff/non-stiff, tolerance, step size) as well options for the optimization technique (nonlinear least squares/simplex, maximum number of iterations, and tolerance). With the selected solver and optimization method, the unknown independent, dependent, and/or initial state parameters in the model are determined within set ranges. For this study, nonlinear least squares regression was used with Matlab ode45, which is a Rimge-Kutta [3, 4] formula for non-stiff systems. The steps of nonlinear least squares regression are as follows ... 

A class of algorithms which is specialized for multilinear problems is known as alternating least-squares (ALS). Multilinear models are all conditionally linear in a function of each of the three or so independent variables for example, spectral intensity is linear in concentration if the other variables are fixed. Each step of an ALS algorithm fixes the vectors for all but one independent variable, then applies linear regression to select the vectors for the one variable to minimize the error sum of squares. The algorithm cycles among the sets of parameters to be estimated, updating each in turn. Most applications of multilinear models use ALS code. ... 

The trend now is to determine kinetic parameters by non-linear regression to the rate equation. Non-linear regression is a method of curve fitting to a non-linear estimator of the relationship between dependent and independent variables. Models for non-linear regression can be complex and multi-parameters and there is a vast literature on the subject (Seber and Wild 2003). K and V can be determined directly from the rate equation (Eq. 3.11) and obtain the values that better fit the experimental... 

In the case of multivariate modeling, several independent as well as several dependent variables may operate. Out of the many regression methods, we will learn about the conventional method of ordinary least squares (OLS) as well as methods that are based on biased parameter estimations reducing simultaneously the dimensionality of the regression problem, that is, principal component regression (PCR) and the partial least squares (PLS) method. 

Usually, the matrix of the independent variables, X, is not square, so that the regression parameters B have to be estimated by the generalized inverse. B is given by... 

This latter technique of Himmelblau, Jones, and Bischoff (H-J-B) has proved to be efficient in various practical situations with few, scattered, data available for complex reaction kinetic schemes (see Ex. 1.6.2-1). Recent extensions of the basic ideas are given by Eakman, Tang, and Gay [48,49, 50]. It should be pointed out, however, that the problem has been cast into one of linear regression at the expense of statistical rigor. The independent variables , X jp, do not fulfill one of the basic requirements of linear regression that the Xi p have to be free of experimental error. In fact, the X p are functions of the dependent variables C/tf) and this may lead to estimates for the parameters that are erroneous. This problem will be discussed further in Chapter 2, when the estimation of parameters in rate equations for catalytic reactions will be treated. Finally, all of the methods have been phrased in terms of batch reactor data, but it should be recognized that the same formulas apply to plug flow and constant volume systems, as will be shown later in this book. 

A regression model is a. fitting relationship that allows the estimation of a dependent variable or experimental response for given settings of a specified group of independent variables or factors. The parameters of the model are known as regression coefficients. Typical tests include the following ... 

The first approach to solve such a problem is to perform a regression analysis between the time series y and the regression variable x ignoring the fact of auto-correlated, stochastic variables. Afterwards, the residuals from this hrst step could be obtained and a time series could be fitted for these residuals, e.g. using ARIMA models. Unfortunately, this approach can lead to biased and inefficient estimates even if the sample is large. Due to the time series characteristics of the dependent and independent variables, the cross-correlations between x and y might be overlaid by the individual temporal dependencies of X and y. To obtain (reasonable) estimates for the impulse response parameters V = (vo,, the time series y has to be cleaned from the time series effects of the re-... 

---