Regression Problems

As we have mentioned, the particular characterization task considered in this work is to determine attenuation in composite materials. At our hand we have a data acquisition system that can provide us with data from both PE and TT testing. The approach is to treat the attenuation problem as a multivariable regression problem where our target values, y , are the measured attenuation values (at different locations n) and where our input data are the (preprocessed) PE data vectors, u . The problem is to find a function iy = /(ii ), such that i), za jy, based on measured data, the so called training data. 

The regression problem is here formulated as the optimization problem... 

More correctly, the regression problem involves means instead of averages in (1). Furthermore, when the criterion function is quadratic, the general (usually nonlinear) optimal solution is given by y = [p u ], i.e., the conditional mean of y given the observation u . 

Partial least-squares path modeling with latent variables (PLS), a newer, general method of handling regression problems, is finding wide apphcation in chemometrics. This method allows the relations between many blocks of data ie, data matrices, to be characterized (32—36). Linear and multiple regression techniques can be considered special cases of the PLS method. 

In Section 33.2.2 we showed how LDA classification can be described as a regression problem with class variables. As a regression model, LDA is subject to the problems described in Chapter 10. For instance, the number of variables should not exceed the number of objects. One solution is to apply feature selection or... 

The structure of such models can be exploited in reducing the dimensionality of the nonlinear parameter estimation problem since, the conditionally linear parameters, kl5 can be obtained by linear least squares in one step and without the need for initial estimates. Further details are provided in Chapter 8 where we exploit the structure of the model either to reduce the dimensionality of the nonlinear regression problem or to arrive at consistent initial guesses for any iterative parameter search algorithm. 

The objectives in this chapter are two. The first one is to briefly review the essentials of linear regression and to present them in a form that is consistent with our notation and approach followed in subsequent chapters addressing nonlinear regression problems. The second objective is to show that a large number of linear regression problems can now be handled with readily available software such as Microsoft Excel and SigmaPlot . 

As seen in Chapter 2 a suitable measure of the discrepancy between a model and a set of data is the objective function, S(k), and hence, the parameter values are obtained by minimizing this function. Therefore, the estimation of the parameters can be viewed as an optimization problem whereby any of the available general purpose optimization methods can be utilized. In particular, it was found that the Gauss-Newton method is the most efficient method for estimating parameters in nonlinear models (Bard. 1970). As we strongly believe that this is indeed the best method to use for nonlinear regression problems, the Gauss-Newton method is presented in detail in this chapter. It is assumed that the parameters are free to take any values. 

Having the smoothed values of the state variables at each sampling point and the derivatives, q we have essentially transformed the problem to a "usual" linear regression problem. The parameter vector is obtained by minimizing the following LS objective function... 

The above linear regression problem can be readily solved using any standard linear regression package. 

In this section we shall only present the derivative approach for the solution of the pyrolytic dehydrogenation of benzene to diphenyl and triphenyl regression problem. This problem, which was already presented in Chapter 6, is also used here to illustrate the use of shortcut methods. As discussed earlier, both state variables are measured and the two unknown parameters appear linearly in the governing ODEs which are also given below for ease of the reader. 

In general, a much larger number of parameters [wavelengths, frequencies, or factors] needs to be calculated in overlapping peak systems [some spectra or chromatograms] than in the linear regression problems, (p. 176)... 

The authors describe the use of a Taylor expansion to negate the second and the higher order terms under specific mathematical conditions in order to make any function (i.e., our regression model) first-order (or linear). They introduce the use of the Jacobian matrix for solving nonlinear regression problems and describe the matrix mathematics in some detail (pp. 178-181). 

In any regression problem, there are more equations than unknown parameters. In this instance, there are four equations, representing the four data pairs. The software then returns the optimum values of parameters in the governing equation(s). The Solu-tions/Statistics menu can be consulted to determine the squared residual between each experimental data point and the corresponding predicted value obtained from the parameter estimates. 

The letter b is reserved for the regression equation coefficients in a regression problem... 

A y-sample outlier, based on the sample s property value(s) (for regression problems only). 

Artificial neural networks (ANNs) have been widely applied in the electronic tongue literature both for classification and multivariate regression problems almost one-third of the papers on electronic tongues examined for this review show ANN applications (see Fig. 2.10). 

Solve the regression problem with relative weighting (us Compare the two sequences of residuals. 

The parameter RP among the input data is the ridge parameter that will be exploited in Section 3.5. In normal regression problems RP = 0 should be used. 

To obtain the estimate (3.23) in a multivariate linear regression problem we... 

---