Least squares methods experimental design

Finally, a word of caution. In science, we usually have theoretical models to provide a basis for assuming a particular dependence of, say, y on x. Least-squares methods are designed to fit experimental data to such Taws , and to give us some idea of the goodness of their fit. They are at their best when we... 

The expression x (J)P(j - l)x(j) in eq. (41.4) represents the variance of the predictions, y(j), at the value x(j) of the independent variable, given the uncertainty in the regression parameters P(/). This expression is equivalent to eq. (10.9) for ordinary least squares regression. The term r(j) is the variance of the experimental error in the response y(J). How to select the value of r(j) and its influence on the final result are discussed later. The expression between parentheses is a scalar. Therefore, the recursive least squares method does not require the inversion of a matrix. When inspecting eqs. (41.3) and (41.4), we can see that the variance-covariance matrix only depends on the design of the experiments given by x and on the variance of the experimental error given by r, which is in accordance with the ordinary least-squares procedure. 

This model allows us to estimate a response inside the experimental domain defined by the levels of the factors and so we can search for a maximum, a minimum or a zone of interest of the response. There are two main disadvantages of the complete factorial designs. First, when many factors were defined or when each factor has many levels, a large number of experiments is required. Remember the expression number of experiments = replicates x Oevels) " (e.g. with 2 replicates, 3 levels for each factor and 3 factors we would need 2 x 3 = 54 experiments). The second disadvantage is the need to use ANOVA and the least-squares method to analyse the responses, two techniques involving no simple calculi. Of course, this is not a problem if proper statistical software is available, but it may be cumbersome otherwise. 

The coefficients are calculated by multi-linear regression, according to the least squares method. There are a very large number of different programs for doing these calculations. The use of properly structured experimental designs, which are usually quite close to orthogonality, has the result that the more sophisticated methods (partial least squares etc.) are not usually necessary. 

The experimental matrix corresponding to a given design determines the settings of the variables [(-1) or ( + 1)] for each experiment. Once the series of experiments has been carried out, estimates of the coefficients b0, bt, and by-are calculated from the observed response values Y using the method of least squares for fitting the data. 

A useful empirical approach to the design of heterogeneous chemical reactors often consists of selecting a suitable equation, such as one in Table 3.3 which, with numerical values substituted for the kinetic and equilibrium constants, represents the chemical reaction in the absence of mass transfer effects. Graphical methods are often employed to aid the selection of an appropriate equation140 and the constants determined by a least squares approach<40). It is important to stress, however, that while the equation selected may well represent the experimental data, it does not... 

If the experimental runs are completely randomized, then randomization theory (see Hinkelmann and Kempthorne, 1994) tells us that least squares gives us unbiased estimators of any pre-chosen set of n — 1 linearly independent contrasts among the n combinations of factor levels (treatments). In most factorial experiments the pre-chosen treatment contrasts would be main effects and, perhaps, interactions. However, in supersaturated designs there is no rational basis for choosing a set of n — 1 contrasts before the analysis. Any model selection method will lead to selection biases, perhaps large biases, in the estimators of effects. If a2 is assumed known, then we can test the null hypothesis that all n treatment populations have equal means. This would not be of great interest, because even if this null hypothesis were true it would not imply that all main effects are zero, only that a particular set of n - 1 linear combinations of treatment means are zero. Of course, in practice, a2 is not known. 

For inttoductory purposes multiple linear regression (MLR) is used to relate the experimental response to the conditions, as is common to most texts in this area, but it is important to realise that odter regression methods such as partial least squares (PLS) are applicable in many cases, as discussed in Chapter 5. Certain designs, such as dtose of Section 2.3.4, have direct relevance to multivariate calibration. In some cases multivariate methods such as PLS can be modified by inclusion of squared and interaction terms as described below for MLR. It is important to remember, however, diat in many areas of chemistry a lot of information is available about a dataset, and conceptually simple approaches based on MLR are often adequate. 

Several studies have employed chemometric designs in CZE method development. In most cases, central composite designs were selected with background electrolyte pH and concentration as well as buffer additives such as methanol as experimental factors and separation selectivity or peak resolution of one or more critical analyte pairs as responses. For example, method development and optimization employing a three-factor central composite design was performed for the analysis of related compounds of the tetracychne antibiotics doxycycline (17) and metacychne (18). The separation selectivity between three critical pairs of analytes were selected as responses in the case of doxycycline while four critical pairs served as responses in the case of metacychne. In both studies, the data were htted to a partial least square (PLS) model. The factors buffer pH and methanol concentration proved to affect the separation selectivity of the respective critical pairs differently so that the overall optimized methods represented a compromise for each individual response. Both methods were subsequently validated and applied to commercial samples. 

The basic principle of experimental design is to vary all factors concomitantly according to a randomised and balanced design, and to evaluate the results by multivariate analysis techniques, such as multiple linear regression or partial least squares. It is essential to check by diagnostic methods that the applied statistical model appropriately describes the experimental data. Unacceptably poor fit indicates experimental errors or that another model should be applied. If a more complicated model is needed, it is often necessary to add further experimental runs to correctly resolve such a model. 

However, no book on experimental design of this scope can be considered exhaustive. In particular, discussion of mathematical and statistical analysis has been kept brief Designs for factor studies at more than two levels are not discussed. We do not describe robust regression methods, nor the analysis of correlations in responses (for example, principle components analysis), nor the use of partial least squares. Our discussion of variability and of the Taguchi approach will perhaps be considered insufficiently detailed in a few years. We have confined ourselves to linear (polynomial) models for the most part, but much interest is starting to be expressed in highly non-linear systems and their analysis by means of artificial neural networks. The importance of these topics for pharmaceutical development still remains to be fully assessed. 

Meiler et al. [27] adopted the general method of optimizing nonlinear experimental designs by the minimization of the covariance matrix of the least-squares... 

---