Complexity regression models

For many decision makers, as well as applied researchers, it is one thing to generate a complex regression model, but another entirely to explain its... 

Now, the researcher has a general overview of simple linear regression, a very useful tool. However, not all applied problems can be described with simple linear regression. The rest of this book describes more complex regression models. 

It is also easy to acquire more faith in a complex regression model than it deserves. Even a good model is at best a crude approximation of reality. Yet, by being computer born, it takes on a special aura that may encourage undeserved faith in its utility. 

Often, it is not quite feasible to control the calibration variables at will. When the process under study is complex, e.g. a sewage system, it is impossible to produce realistic samples that are representative of the process and at the same time optimally designed for calibration. Often, one may at best collect representative samples from the population of interest and measure both the dependent properties Y and the predictor variables X. In that case, both Y and X are random, and one may just as well model the concentrations X, given the observed Y. This case of natural calibration (also known as random calibration) is compatible with the linear regression model... 

FIGURE 4.4 Determination of optimum complexity of regression models (schematically). Measure for prediction errors for instance RMSECv in arbitrary linear units. Left, global and local minimum of a measure for prediction performance. Right, one standard error mle. 

The PLS approach to multivariate linear regression modeling is relatively new and not yet fully investigated from a theoretical point of view. The results with calibrating complex samples in food analysis 122,123) j y jnfj-ared reflectance spectroscopy, suggest that PLS could solve the general calibration problem in analytical chemistry. 

Classic univariate regression uses a single predictor, which is usually insufficient to model a property in complex samples. Multivariate regression takes into account several predictive variables simultaneously for increased accuracy. The purpose of a multivariate regression model is to extract relevant information from the available data. Observed data usually contains some noise and may also include irrelevant information. Noise can be considered as random data variation due to experimental error. It may also represent observed variation due to factors not initially included in the model. Further, the measured data may carry irrelevant information that has little or nothing to do with the attribute modeled. For instance, NIR absorbance... 

Concentration-Response Relationship. The simplest model that describes the concentration-response relationship adequately should be used. The selection of weighing and use of a complex regression equation should be justified. 

Mathematical optimization always requires a deterministic process model to predict the future behavior of a process. However, as previously mentioned, it is difficult to construct mathematical models that can cover the entire range of fermentation due to the complexity of intracellular metabolic reactions. As an alternative to the deterministic mathematical models, Kishimoto et al. proposed a statistical procedirre that uses linear multiple regression models [7], as shown below, instead of a deterministic mathematical model such as a Monod equation. 

Dimensionality (or complexity) of the regression model refers to the number of factors included in the regression model. This is probably one of the most critical parameters that have to be optimised in order to obtain good predictions. Unfortunately, there is no unique criterion on how to set it. In this introductory book, only the most common procedure is explained in detail. The reader should be aware, however, that it is just a procedure, not the procedure. We will also mention two other recent possibilities that, in our experience, yield good results and are intuitive. 

There are some distinct advantages of the PLS regression method over the PCR method. Because Y-data are used in the data compression step, it is often possible to build PLS models that are simpler (i.e. require fewer compressed variables), yet just as effective as more complex PCR models built from the same calibration data. In the process analytical world, simpler models are more stable over time and easier to maintain. There is also a small advantage of PLS for qualitative interpretative purposes. Even though the latent variables in PLS are still abstract, and rarely express pure chemical or physical phenomena, they are at least more relevant to the problem than the PCs obtained from PCR. 

As discussed earlier, the two figures of merit for a linear regression model, the RMSEE and the correlation coefficient (Equations 8.11 and 8.10), can also be used to evaluate the fit of any quantitative model. The RMSEE, which is in the units of the property of interest, can be used to provide a rough estimate of the anticipated prediction error of the model. However, such estimates are often rather optimistic because the exact same data are used to build and test the model. Furthermore, they cannot be used effectively to determine the optimal complexity of a model because increased model complexity will always result in an improved model fit. As a result, it is very dangerous to rely on this method for model validation. 

In practice, one is often faced with choosing a model that is easily interpretable but may not approximate a response very well, such as a low-order polynomial regression, or with choosing a black box model, such as the random-function model in equations (l)-(3). Our approach makes this blackbox model interpretable in two ways (a) the ANOVA decomposition provides a quantitative screening of the low-order effects, and (b) the important effects can be visualized. By comparison, in a low-order polynomial regression model, the relationship between input variables and an output variable is more direct. Unfortunately, as we have seen, the complexities of a computer code may be too subtle for such simple approximating models. 

This SPS team was empowered to improve their system. Upper management simply said, OK, see if you can fix it The team s solution used a simple path of least resistance. For a more complex solution set, they may have selected multivariate regression modeling or hired an outside statistical consultant.The main point is that they used a scientifically proven, systematic approach to solving a complex problem, through teamwork and simple observational and analytical techniques. 

---