Multivariate regression basics

Lucic, B., Nadramija, D., Basic, 1. andTrinajstic, N. (2003) Toward generating simpler QSAR models nonlinear multivariate regression versus several neural network ensembles and some related... 

There may exist a mechanical instmment for measuring the softness of the apple material or you may simply use a taste panel to score each apple on a scale from 1 to 10. What the multivariate regression software program then will do for you is basically to produce a model that you can use to calculate the softness of each apple (Y-variable) from the measured impedance data (X-variables) or from any data derived from the measured data, such as Cole parameters. 

TABLE 2.6 The Basic Structure of a QSAR Table, As Provided By the Multivariate Regression Analysis... 

Some methods that paitly cope with the above mentioned problem have been proposed in the literature. The subject has been treated in areas like Cheraometrics, Econometrics etc, giving rise for example to the methods Partial Least Squares, PLS, Ridge Regression, RR, and Principal Component Regression, PCR [2]. In this work we have chosen to illustrate the multivariable approach using PCR as our regression tool, mainly because it has a relatively easy interpretation. The basic idea of PCR is described below. 

If we do have, and include, a priori knowledge in addition to the measurements or numerical data we should use methods from the second family of basic data analysis methods. Here the data are considered to be grouped in respect of the objects, or maybe in respect of the features. Within this family we may further distinguish between non-causally and causally determined data, or by analogy with correlation and regression, we may distinguish between multivariate relationships and dependencies. 

The main goal of this chapter is to present the theoretical background of some basic chemometric methods as a tool for the assessment of surface water quality described by numerous chemical and physicochemical parameters. As a case study, long-term monitoring results from the watershed of the Struma River, Bulgaria, are used to illustrate the options offered by multivariate statistical methods such as CA, principal components analysis, principal components regression (models of source apportionment), and Kohonen s SOMs. 

Multivariate analytical methods have also been applied to the analysis of drug substances. The methods have a significant component of matrix analysis, and the Beer-Lambert law is basically rewritten in matrix form, permitting matrix analysis of absorbance data. A number of other mathematical algorithms have also been developed for the quantitation of analytes in multicomponent mixtures. These have either been iterative methods or methods based on multiple least-squares regression. The multiple least-squares regression methods require a knowledge of all the components of the multicomponent mixture, whereas the iterative methods such as the Kalman or the simplex method are less restrictive in the sense that interferents whose spectra are not known need not be included in the database. 

The basic principle of experimental design is to vary all factors concomitantly according to a randomised and balanced design, and to evaluate the results by multivariate analysis techniques, such as multiple linear regression or partial least squares. It is essential to check by diagnostic methods that the applied statistical model appropriately describes the experimental data. Unacceptably poor fit indicates experimental errors or that another model should be applied. If a more complicated model is needed, it is often necessary to add further experimental runs to correctly resolve such a model. 

Suppose a multivariate time series of dimension L collected at T points of time, say Y The basic types of dependency, auto-regression and moving average, are used to... 

The book that follows is best appreciated by a reader who likes numbers and is comfortable with algebra and multivariate calculus as they apply to situations where decisionmakers are optimizing subject to limitations. Because we try to show how numerical simulation can complement econometric studies a basic understanding of regression is also useful background for the reader. We have structured our presentations to be accessible and interesting to undergraduate honors, masters, and doctoral students in the social sciences, public policy, law,... 

Chatfield and Collins (1980), in the introduction to their chapter on cluster analysis, quote the first sentence of a review article on cluster analysis by Cormack (1971) The availability of computer packages of classification techniques has led to the waste of more valuable scientific time than any other statistical innovation (with the possible exception of multiple-regression techniques). This is perhaps a little hard on cluster analysis and, for that matter, multiple regression but it serves as a note of warning. The aim of this book is to explain the basic principles of the more popular and useful multivariate methods so that readers will be able to understand the results obtained from the techniques and, if interested, apply the methods to their own data. This is not a substitute for a formal training in statistics the best way to avoid wasting one s own valuable scientific time is to seek professional help at an early stage. 

Chapter 4 retrieves the basic ideas of classical univariate calibration as the standpoint from which the natural and intuitive extension of multiple linear regression (MLR), arises. Unfortunately, this generalization is not suited to many laboratory tasks and, therefore, the problems associated with its use are explained in some detail. Such problems justify the use of other more advanced techniques. The explanation of what the multivariate space looks like and how principal components analysis can tackle it is the next step forward. This constitutes the root of the regression methodology presented in the following chapter. 

J. N. Miller, Basic statistical methods for analytical chemistry. Part 2 Calibration and regression methods. A Review, Analyst, 1991, 116, 3-13. R. G. Brereton, Introduction to multivariate calibration in analytical chemistry. Analyst, 2000, 125, 2125-2154. 

The standard method for multivariate data of type 2a (Figure 2, X-matrix and one j -variable) is multiple linear regression (MLR), also called ordinary least squares regression (OLS). Only a few basic principles can be summarized here. The aim of data interpretation in this case is to build a linear model for the prediction of a response y from the independent variables (regressors, features) X, X2... Xp as given in equation (17) ... 

---