Estimation Multiresponse

Unfortunately, there are some technical difficulties associated with the determinant criterion (ref. 28). Minimizing the determinant (3.66) is not a trivial task. In addition, the method obviously does not apply if det[ V(p) ] is zero or nearly zero for all parameter values. This is the case if there exist affine linear relationships among the responses y, y2,. .., yny, as we discussed in Section 1.8.7. To overcome this problem the principal component analysis of the observations is applied before the estimation step. 

Example 5.6 Comparison of the determinant criterion with least squares 

Assuming first order reactions, the mechanism gives rise to a set of first order differential equations. The following solution of the equations gives the component concentrations y, 2,. .., y3 as function of the reaction time t  

The observed concentrations have been listed in Table 1.3. Let us first fit the above response function to the data by the least sqares method with the weighting matrices = I, i.e., without weighting. Module M45 results in the estimates shown in the first row of of Table 3.5. 

As found in Section 1.8.7, there were two affin linear dependences among the data, classified as exact ones. Therefore, Box et al. (ref. 29) considered the principal components corresponding to the three largest eigenvalues as response functions when minimizing the objective function (3.66). By virtue of the eigenvectors derived in Section 1.8.7, these principal components are  

---

Observing a process, scientists and engineers frequently record several variables. For example, (ref. 20) presents concentrations of all species for the thermal isomerization of a-pinene at different time points. These species are ct-pinene (yj), dipentene ( 2) allo-ocimene ( 3), pyronene (y ) and a dimer product (y5). The data are reproduced in Table 1.3. In (ref. 20) a reaction scheme has also been proposed to describe the kinetics of the process. Several years later Box at al. (ref. 21) tried to estimate the rate coefficients of this kinetic model by their multiresponse estimation procedure that will be discussed in Section 3.6. They run into difficulty and realized that the data in Table 1.3 are not independent. There are two kinds of dependencies that may trouble parameter estimation ... 

Kowalski K, Beno M, Bergstrom C, Gaud H. The application of multiresponse estimation to drug stability studies. Drug Dev Ind Pharm 1987 13 2823-2838. 

The following chapters and the package GREGPLUS apply these principles to practical models and various data structures. Least squares, multiresponse estimation, model discrimination, and process function estimation are presented there as special forms of Bayesian estimation. 

Bates, D. M., and D, G. Watts, Multiresponse estimation with special application to systems of linear differential equations, Technometrics, 27, 329-339 (1985). 

Example C.5. Multiresponse Estimation with Analytical Derivatives... 

Criteria of multiresponse maximum likelihood parameter estimation and conditions for their... 

The adaptive parameters in the model were estimated by nonlinear and multiresponse regression, performed using the Fortran subroutine BURENL23 based... 

Experiments that will be used to estimate the behavior of a system should not be chosen in a whimsical or unplanned way, but rather, should be carefully designed with a view toward achieving a valid approximation to a region of the true response surface [Cochran and Cox (1950), Youden (1951), Wilson (1952), Mandel (1964), Fisher (1971)]. In the next several chapters, many of the important concepts of the design and analysis of experiments are introduced at an elementary level for the single-factor single-response case. In later chapters, these concepts will be generalized to multifactor, multiresponse systems. 

D.M Bates and D.G. Watts, A generalized Gauss-Newton procedure for multiresponse parameter estimation, SIAM J. Sci. and Stat. Comp., 8 (1987) 49-55. 

The rate expressions Rj — Rj(T,ck,6m x) typically contain functional dependencies on reaction conditions (temperature, gas-phase and surface concentrations of reactants and products) as well as on adaptive parameters x (i.e., selected pre-exponential factors k0j, activation energies Ej, inhibition constants K, effective storage capacities i//ec and adsorption capacities T03 1 and Q). Such rate parameters are estimated by multiresponse non-linear regression according to the integral method of kinetic analysis based on classical least-squares principles (Froment and Bischoff, 1979). The objective function to be minimized in the weighted least squares method is... 

The kinetic parameters of ammonia oxidation were fitted by multiresponse nonlinear regression, while the parameter estimates for the ammonia adsorption-desorption kinetics were kept unchanged with respect to those obtained from the fit in the previous section. Notably, in this case both the NH3 and the N2 outlet concentrations were regarded as regression responses. 

A global multiresponse non-linear regression was performed to fit Eq. (57) to all the runs with both 2% and 6% v/v 02 feed content to obtain the estimates of the kinetic parameters (Nova et al., 2006a). Figure 37 (solid lines) illustrates the adequacy of the global fit of the TRM runs with 2 and 6% 02 the MR rate law can evidently capture the complex maxima-minima NO and N2 traces (symbols) at low T at both NH3 startup, that a simple Eley-Rideal (ER), approach based on the equation... 

Table 3. Objective functions to be minimized in parameter estimation of multiresponse data and conditions for application [9]. ...

The statistical investigation of a model begins with the estimation of its parameters from observations. Chapters 4 and 5 give some background for this step. For single-response observations with independent normal error distributions and given relative precisions, Bayes theorem leads to the famous method of least squares. Multiresponse observations need more detailed treatment, to be discussed in Chapter 7. 

The methods of Chapter 6 are not appropriate for multiresponse investigations unless the responses have known relative precisions and independent, unbiased normal distributions of error. These restrictions come from the error model in Eq. (6.1-2). Single-response models were treated under these assumptions by Gauss (1809, 1823) and less completely by Legendre (1805), co-discoverer of the method of least squares. Aitken (1935) generalized weighted least squares to multiple responses with a specified error covariance matrix his method was extended to nonlinear parameter estimation by Bard and Lapidus (1968) and Bard (1974). However, least squares is not suitable for multiresponse problems unless information is given about the error covariance matrix we may consider such applications at another time. 

Bayes theorem (Bayes 1763 Box and Tiao 1973, 1992) permits estimation of the error covariance matrix S from a multiresponse data set, along with the parameter vector 0 of a predictive model. It is also possible, under further assumptions, to shorten the calculations by estimating 6 and I separately, as we do in the computer package GREGPLUS provided in Athena. We can then analyze the goodness of fit, the precision of estimation of parameters and functions of them, the relative probabilities of alternative models, and the choice of additional experiments to improve a chosen information measure. This chapter summarizes these procedures and their implementation in GREGPLUS details and examples are given in Appendix C. 

Box and Draper (1965) took another major step by deriving a posterior density function p 6 Y), averaged over S, for estimating a parameter vector 6 from a full matrix Y of multiresponse observations. The errors in the observations were assumed to be normally distributed with an unknown m X m covariance matrix S. Michael Box and Norman Draper (1972) gave a corresponding function for a data matrix Y of discrete blocks of responses and applied that function to design of multiresponse experiments. 

Multiparameter, multiresponse models call for digital optimization. Early workers minimized n(0) by search techniques, which were tedious and gave only a point estimate of 9. Newtonlike algorithms for minimization of i (0) and for interval estimation of 6, were given by Stewart and Sorensen (1976, 1981) and by Bates and Watts (1985, 1987). Corresponding algorithms for likelihood-based estimation were developed by Bard and Lapidus (1968) and Bard (1974), extended by Klaus and Rippin (1979) and Steiner, Blau, and Agin (1986). 

Box and Draper (1965) derived a density function for estimating the parameter vector 6 of a multiresponse model from a full data matrix Y, subject to errors normally distributed in the manner of Eq. (4.4-3) with a full unknown covariance matrix E. With this type of data, every event u has a full set of m responses, as illustrated in Table 7.1. The predictive density function for prospective data arrays Y from n independent events, consistent with Eqs. (7.1-1) and (7.1-3), is... 

The results just given can be transformed in the manner of Section 6.6.2 to get multiresponse inferences for auxiliary functions. One defines a vector (j>a(6) of such functions and computes the matrix A. as in Eqs. (6.6-19) and (6.6-20). The results of Section 7.5.1 can then be applied to the chosen functions by substituting A for A and for 6 in those equations. GREG PLUS does these calculations automatically on request to obtain the interval estimates and covariance matrix for a linearly independent subset of the auxiliary functions. 

In this chapter, Bayesian and likelihood-based approaches have been described for parameter estimation from multiresponse data with unknown covariance matrix S. The Bayesian approaches permit objective estimates of 6 and E by use of the noninformative prior of Jeffreys (1961). Explicit estimation of unknown covariance elements is optional for problems of Types 1 and 2 but mandatory for Types 3 and 4. 

The posterior density function is the key to Bayesian parameter estimation, both for single-response and multiresponse data. Its mode gives point estimates of the parameters, and its spread can be used to calculate intervals of given probability content. These intervals indicate how well the parameters have been estimated they should always be reported. 

Many multiresponse investigations have used procedures based on Eq. (7.1-7) or (7.1-15), which exclude S from the parameter set but implicitly estimate S nonetheless. Additional formulas of this type are provided in Eqs. (7.1-8,9) and (7.1-16,17), which give estimates of 6 that are more consistent with the full posterior density functions for those problem structures see Eqs. (7.1-6) and (7.1-14). 

Bain, R. S., Solution of nonlinear algebraic equation systems and single and multiresponse nonlinear parameter estimation problems, Ph. D. thesis, University of Wisconsin-Madison (1993). 

Box, M. J., and N. R. Draper, Estimation and design criteria for multiresponse non-linear models with non-homogeneous variance, J. Roy. Statist. Soc., Series C (Appl. Statist..) 21, 13-24 (1972). 

Draper, N. R., and W. G. Hunter, Design of experiments for parameter estimation in multiresponse situations, Biometrika, 53, 525-533 (1966). 

Guay, M., and D. D. McLean, Optimization and sensitivity aanalysis for multiresponse parameter estimation in systema of ordinary differential equations, Comput. Chem. Eng., 19, 1271-1285 (1995). 

Mezaki, R., and J. B. Butt, Estimation of rate constants from multiresponse kinetic data, Ind. Eng. Chem. Fundam., 7, 120-125 (1968). 

Stewart, W. E., Multiresponse parameter estimation with a new and noninfor-mative prior, Biometrika, 74. 557-562 (1987). This prior is superseded by Eq. (7.1-5). 

Stewart, W. E., and J. P. Sorensen. Bayesian estimation of common parameters from multiresponse data with missing observations. Technometrics, 23, 131-141 (1981) Errata, 24, 91 (1982). 

---