The Bayesian Approach

As discussed before, in the conventional data reconciliation approach, auxiliary gross error detection techniques are required to remove any gross error before applying reconciliation techniques. Furthermore, the reconciled states are only the maximum likelihood states of the plant, if feasible plant states are equally likely. That is, P x = 1 if the constraints are satisfied and P x = 0 otherwise. This is the so-called binary assumption (Johnston and foamer, 1995) or flat distribution. 

To alleviate these assumptions the maximum likelihood rectification (MLR) technique was proposed by Johnston and Kramer. This approach incorporates the prior distribution of the plant states, P x, into the data reconciliation process to obtain the maximum likely rectified states given the measurements. Mathematically the problem can be stated (Johnston and Kramer, 1995) as 

According to Bayes theorem, the probability of the states, given the data, will be 

The following three cases can be defined according to different assumptions about P y/x and P x and are discussed later  

Case 1 Binary assumption on P x (flat distribution), and sensor errors are assumed to follow a normal distribution 

---

CE Buck, WG Cavanaugh, CD Litton. The Bayesian Approach to Interpreting Archaeological Data. New York Wiley, 1996. 

In brief, the Bayesian approach uses PDFs of pattern classes to establish class membership. As shown in Fig. 22, feature extraction corresponds to calculation of the a posteriori conditional probability or joint probability using the Bayes formula that expresses the probability that a particular pattern label can be associated with a particular pattern. 

The knowledge required to implement Bayes formula is daunting in that a priori as well as class conditional probabilities must be known. Some reduction in requirements can be accomplished by using joint probability distributions in place of the a priori and class conditional probabilities. Even with this simplification, few interpretation problems are so well posed that the information needed is available. It is possible to employ the Bayesian approach by estimating the unknown probabilities and probability density functions from exemplar patterns that are believed to be representative of the problem under investigation. This approach, however, implies supervised learning where the correct class label for each exemplar is known. The ability to perform data interpretation is determined by the quality of the estimates of the underlying probability distributions. 

Monte Carlo data for y were generated according to with mean x, to simulate process sampling data. A window size of 25 was used here and to demonstrate the performance of the Bayesian approach. 

The basic premise of the Bayesian approach is that observations change the state of knowledge of a system. Let us suppose for simplicity that the item of interest is some parameter, 0, describing a state of nature (as in the above example, where 0 was a property of the coin and the conditions under which it was tossed). Figure 5.1 indicates symbolically the development of knowledge. 

The Bayesian approach reverses the role of the sample and model the sample is fixed and unique, and the model itself is uncertain. This viewpoint corresponds more closely to the practical situation facing the individual researcher there is only 1 sample, and there are doubts either what model to use, or, for a specified model, what parameter values to assign. The model uncertainty is addressed by considering that the model parameters are distributed. In other words Bayesian interpretation of a confidence interval is that it indicates the level of belief warranted by the data the... 

The classical or frequentist approach to probability is the one most taught in university conrses. That may change, however, becanse the Bayesian approach is the more easily nnderstood statistical philosophy, both conceptually as well as numerically. Many scientists have difficnlty in articnlating correctly the meaning of a confidence interval within the classical frequentist framework. The common misinterpretation the probability that a parameter lies between certain limits is exactly the correct one from the Bayesian standpoint. 

Apart from this pedagogical aspect (cf Lee 1989, preface), there is a more technical reason to prefer the Bayesian approach to the confidence approach. The Bayesian approach is the more powerfnl one eventnally, for extending a model into directions necessary to deal with its weaknesses. These are various relaxations of distribntional assnmptions. The conceptnal device of an infinite repetition of samples, as in the freqnentist viewpoint, does not yield enongh power to accomplish these extensions. 

For the normal distribution there are analytical solutions allowing the assessment of both FA and HC using frequentist statistics. In contrast, Bayesian solutions are numerical. This highlights the flexibility of the Bayesian approach since it can easily deal with any distribution, which is not always possible with the frequentist approach. 

Aldenberg and Jaworska (2000) demonstrate that frequentist statistics and the Bayesian approach with noninformative prior results in identical confidence intervals for the normal distribution. Generally speaking, this is more the exception than the rule. 

For those who feel more confident with the frequentist approach and find the Bayesian approach controversial to some extent, it is advantageous that both approaches yield the same answers in this simplest case. This might add confidence in the Bayesian approach for some practitioners. 

Howson C, Urbach P. 1989. Scientific reasoning. The Bayesian approach. La SaUe (fL) Open Court. 

O Hagan A. 2001. Uncertainty in toxicological predictions the Bayesian approach to statistics. In Rainbow PS, Hopkin SR Crane M, editors. Forecasting the environmental fate and effects of chemicals. Chichester (UK) John Wiley, p 25—41. 

Robust Bayes redresses some of the most commonly heard criticisms of the Bayesian approach. For instance, robust Bayes relaxes the requirement for an analyst to specify a particular prior distribution and reflects the analyst s confidence about the choice of the prior. Bayesian methods generally preserve zero probabilities. That is, any values of the real line for which the prior distribution is surely zero will remain with zero probability in the posterior, no matter what the likelihood is and no matter what new data may arrive. This preservation of zero probabilities means that an erroneous prior conception about what is possible is immutable in the face of... 

Two very different approaches to inferential statistics exist the classical or fre-quentist approach and the Bayesian approach. Each approach is used to draw conclusions (or inferences) regarding the magnitude of some unknown quantity, such as the intercept and slope of a dose-response model. The key difference between classical... 

Throughout this book, the approach taken to hypothesis testing and statistical analysis has been a frequentist approach. The name frequentist reflects its derivation from the definition of probability in terms of frequencies of outcomes. While this approach is likely the majority approach at this time, it should be noted here that it is not the only approach. One alternative method of statistical inference is the Bayesian approach, named for Thomas Bayes work in the area of probability. 

The Bayesian approach forces the researcher to make certain explicit statements ... 

While the Bayesian approach explicitly acknowledges the role of judgement, the frequentist approach also involves judgement regarding the assumption of a random, representative... 

Thus, despite its limitations, the Bayesian approach is more flexible in dealing with situations in which data are limited or not available, but in which the state of knowledge is adequate to support judgements regarding prior distributions. 

To give a flavor of the Bayesian approach, two summaries of the Bayesian analysis of the glucose data are now presented. A more detailed analysis, including a discussion of prior distributions and computational methods, is given in Section 5.2. 

The Bayesian approach is more than a tool for adjusting the results of the all subsets regression by adding appropriate effects to achieve effect heredity. Take, for example, the sixth model in Table 4 which consists of Al,Bl, AlDq, BlHl, BlHq, BqHq. The AlDq effect identified as part of this model does not appear in the best subsets of size 1-6 in Table 3. The Bayesian procedure has therefore discovered an additional possible subset of effects that describes the data. 

The Bayesian approach described in Section 1.1 can be applied to a wide variety of screening problems, including those with both quantitative and qualitative factors. 

The Bayesian approach to subset selection is outlined in Sections 2 to 4. Section 2 gives the mathematical ingredients of the analysis a probability model for the data, prior distributions for the parameters (J3, a, 5) of the model, and the resultant posterior distribution. 

Obtaining parametric maps necessarily requires estimating the vector of the parameter 0 from K-noised samples. The general theory of estimation59,60 provides solutions that can be applied in the domain of quantitative MRI. In practice, the ML approach is the most commonly used, because it concerns the estimation of non-random parameters, unlike the Bayesian approach, which is mostly applied to segment the images.61 The LS approaches defined by... 

The whole procedure, with or without salts, may not be based upon sound statistical principles. Rather than using various object functions, it appears better to use a reliable statistical technique such as the method of maximum likelihood (24) or the Bayesian approach (25), both of which take into account the errors in all experimental observations in a logically justifiable fashion. The various discrepancies and anomalies noted in the present work would be moderated by using either... 

In this chapter, Bayesian and likelihood-based approaches have been described for parameter estimation from multiresponse data with unknown covariance matrix S. The Bayesian approaches permit objective estimates of 6 and E by use of the noninformative prior of Jeffreys (1961). Explicit estimation of unknown covariance elements is optional for problems of Types 1 and 2 but mandatory for Types 3 and 4. 

With greater sophistication of methods of data acquisition, intuition can play a less important role, and a bayesian philosophical approach becomes more important. In contrast to the classical approach to statistics, which is concerned with the distribution of possible measured values about a unique true value, the bayesian approach is concerned with the distribution of possible true values about the measured value at hand—a concept often greeted with hostility by traditional statisticians. 

A few programs are now available that allow the efficient simultaneous data analysis from a population of subjects. This approach has the significant advantage that the number of data points per subject can be small. However, using data from many subjects, it is possible to complete the analyses and obtain both between- and within-subject variance information. These programs include NONMEM and WinNON-MIX for parametric (model dependent) analyses and NPEM when non-parametric (model independent) analyses are required. This approach nicely complements the Bayesian approach. Once the population values for the pharmacokinetic parameters are obtained, it is possible to use the Bayesian estimation approach to obtain estimates of the individual patient s pharmacokinetics and optimize their drug therapy. 

Lanctot, K.L. Naranjo, C.A. Comparison of the Bayesian Approach and a simple algorithm for assessment of adverse drug events. Clin. Pharmacol. Ther. 1995, 58, 692-698. 

---