Function Bayesian

Statistical pattern recognition is based on the statistical nature of signals and extracted features are represented as probability density functions (Schalkoff, 1992). It therefore requires knowledge of a priori probabilities of measurements and features. Statistical approaches include linear discriminant functions, Bayesian functions and cluster analysis and may be unsupervised or supervised. Supervised classifiers require a set of exemplars for each class to be recognized they are used to train the system. Unsupervised learning, on the other hand, does not require an exemplar set. 

This paper is structured as follows in section 2, we recall the statement of the forward problem. We remind the numerical model which relates the contrast function with the observed data. Then, we compare the measurements performed with the experimental probe with predictive data which come from the model. This comparison is used, firstly, to validate the forward problem. In section 4, the solution of the associated inverse problem is described through a Bayesian approach. We derive, in particular, an appropriate criteria which must be optimized in order to reconstruct simulated flaws. Some results of flaw reconstructions from simulated data are presented. These results confirm the capability of the inversion method. The section 5 ends with giving some tasks we have already thought of. 

Mixmre models have come up frequently in Bayesian statistical analysis in molecular and structural biology [16,28] as described below, so a description is useful here. Mixture models can be used when simple forms such as the exponential or Dirichlet function alone do not describe the data well. This is usually the case for a multimodal data distribution (as might be evident from a histogram of the data), when clearly a single Gaussian function will not suffice. A mixture is a sum of simple forms for the likelihood ... 

The likelihood function is an expression for p(a t, n, C), which is the probability of the sequence a (of length n) given a particular alignment t to a fold C. The expression for the likelihood is where most tlireading algorithms differ from one another. Since this probability can be expressed in terms of a pseudo free energy, p(a t, n, C) x exp[—/(a, t, C)], any energy function that satisfies this equation can be used in the Bayesian analysis described above. The normalization constant required is akin to a partition function, such that... 

For example, Stolorz et al. [88] derived a Bayesian formalism for secondary structure prediction, although their method does not use Bayesian statistics. They attempt to find an expression for / ( j. seq) = / (seq j.)/7( j.)//7(seq), where J. is the secondary structure at the middle position of seq, a sequence window of prescribed length. As described earlier in Section II, this is a use of Bayes rule but is not Bayesian statistics, which depends on the equation p(Q y) = p(y Q)p(Q)lp(y), where y is data that connect the parameters in some way to observables. The data are not sequences alone but the combination of sequence and secondary structure that can be culled from the PDB. The parameters we are after are the probabilities of each secondary structure type as a function of the sequence in the sequence window, based on PDB data. The sequence can be thought of as an explanatory variable. That is, we are looking for... 

Analysis and prediction of side-chain conformation have long been predicated on statistical analysis of data from protein structures. Early rotamer libraries [91-93] ignored backbone conformation and instead gave the proportions of side-chain rotamers for each of the 18 amino acids with side-chain dihedral degrees of freedom. In recent years, it has become possible to take account of the effect of the backbone conformation on the distribution of side-chain rotamers [28,94-96]. McGregor et al. [94] and Schrauber et al. [97] produced rotamer libraries based on secondary structure. Dunbrack and Karplus [95] instead examined the variation in rotamer distributions as a function of the backbone dihedrals ( ) and V /, later providing conformational analysis to justify this choice [96]. Dunbrack and Cohen [28] extended the analysis of protein side-chain conformation by using Bayesian statistics to derive the full backbone-dependent rotamer libraries at all... 

It is an easy exercise to show that if Pn satisfies the Kolmogorov consistency conditions (equations 5.68) for all blocks Bj of size j < N, then T[N- N+LPN) satisfies the Kolmogorov consistency conditions for blocks Bj of size j < N + 1. Given a block probability function P, therefore, we can generate a set of block probability functions Pj for arbitrary j > N hy successive applications of the operator TTN-tN+i, this set is called the Bayesian extension of Pn-... 

The Local Structure Operator By the Kolmogorov consistency theorem, we can use the Bayesian extension of Pn to define a measure on F. This measure -called the finite-block measure, /i f, where N denotes the order of the block probability function from which it is derived by Bayesian extension - is defined by assigning t.o each cylinder c Bj) = 5 G F cti = 6i, 0 2 = 62, , ( j — bj a value equal to the probability of its associated block ... 

The essence of the LST for one-dimensional lattices resides in the fact that an operator TtN->N+i could be constructed (equation 5.71), mapping iV-block probability functions to [N -f l)-block probabilities in a manner which satisfies the Kolmogorov consistency conditions (equation 5.68). A sequence of repeated applications of this operator allows us to define a set of Bayesian extended probability functions Pm, M > N, and thus a shift-invariant measure on the set of all one-dimensional configurations, F. Unfortunately, a simple generalization of this procedure to lattices with more than one dimension, does not, in general, produce a set of consistent block probability functions. Extensions must instead be made by using some other, approximate, method. We briefly sketch a heuristic outline of one approach below (details are worked out in [guto87b]). 

One drag level (Cindiv) can be used with the means and standard deviation (SD) of population parameters (Ppop) as a priori knowledge for an individual parameter estimate using the Bayesian objective function. 

A more general approach to finding a solution than simply assuming some coefficients are zero is provided by a Bayesian approach. Since the displacement d is a linear function of two unknowns, 02 and 07, we can write... 

Simons KT, Kooperberg C, Huang E, Baker D. Assembly of protein tertiary structures from fragments with similar local sequences using simulated annealing and Bayesian scoring functions. J Mol Biol 1997 268 209-25. 

Under certain conditions we may have some prior information about the parameter values. This information is often summarized by assuming that each parameter is distributed normally with a given mean and a small or large variance depending on how trustworthy our prior estimate is. The Bayesian objective function, SB(k), that should be minimized for algebraic equation models is... 

The knowledge required to implement Bayes formula is daunting in that a priori as well as class conditional probabilities must be known. Some reduction in requirements can be accomplished by using joint probability distributions in place of the a priori and class conditional probabilities. Even with this simplification, few interpretation problems are so well posed that the information needed is available. It is possible to employ the Bayesian approach by estimating the unknown probabilities and probability density functions from exemplar patterns that are believed to be representative of the problem under investigation. This approach, however, implies supervised learning where the correct class label for each exemplar is known. The ability to perform data interpretation is determined by the quality of the estimates of the underlying probability distributions. 

Both the a priori and the likelihood functions contain exponentials, so that it is convenient to consider the logarithm of the posterior probability, and maximise the Bayesian score ... 

In this section we briefly discuss an approximate formalism that allows incorporation of the experimental error variances in the constrained maximisation of the Bayesian score. The problem addressed here is the derivation of a likelihood function that not only gives the distribution of a structure factor amplitude as computed from the current structural model, but also takes into account the variance due to the experimental error. 

Under general hypotheses, the optimisation of the Bayesian score under the constraints of MaxEnt will require numerical integration of (29), in that no analytical solution exists for the integral. A Taylor expansion of Ao(R) around the maximum of the P(R) function could be used to compute an analytical expression for A and its first and second order derivatives, provided the spread of the A distribution is significantly larger than the one of the P(R) function, as measured by a 2. Unfortunately, for accurate charge density studies this requirement is not always fulfilled for many reflexions the structure factor variance Z2 appearing in Ao is comparable to or even smaller than the experimental error variance o2, because the deformation effects and the associated uncertainty are at the level of the noise. 

These considerations raise a question how can we determine the optimal value of n and the coefficients i < n in (2.54) and (2.56) Clearly, if the expansion is truncated too early, some terms that contribute importantly to Po(AU) will be lost. On the other hand, terms above some threshold carry no information, and, instead, only add statistical noise to the probability distribution. One solution to this problem is to use physical intuition [40]. Perhaps a better approach is that based on the maximum likelihood (ML) method, in which we determine the maximum number of terms supported by the provided information. For the expansion in (2.54), calculating the number of Gaussian functions, their mean values and variances using ML is a standard problem solved in many textbooks on Bayesian inference [43]. For the expansion in (2.56), the ML solution for n and o, also exists, lust like in the case of the multistate Gaussian model, this equation appears to improve the free energy estimates considerably when P0(AU) is a broad function. 

Another approach is to develop a global model that contains plausible models as special cases, defined by alternative values of particular parameters. This converts model uncertainty into uncertainty about the model parameters. Again this can be done using either Bayesian or non-Bayesian approaches. This approach is favored by Morgan and Henrion (1990), who describe how it can be applied to uncertainty about dose-response functions (threshold versus nonthreshold, linear versus exponential). 

Bayesian interpretation and application of the theorem quantifies the development of information. Suppose that A is a statement or hypothesis, and let p A) stand for the degree of belief in the statement or hypothesis A, based on prior knowledge, it is called the prior probability. Let B represent a set of observations, then p(B A) is the probability that those observations occur given that A is true. This is called the likelihood of the data and is a function of the hypothesis. The left side, p(A B), is the new degree of belief in A, taking into account the observations B, it is called the posterior probability. Thus Bayes theorem tracks the effect that the observations have upon the changing knowledge about the hypothesis. The theorem can be expressed thus ... 

FIG U RE 5.4 Bayesian normal density spaghetti plot random sample of 100 normal probability density functions (pdfs) drawn from the posterior distribution of p and o, given 7 cadmium NOEC toxicity data (dots) from Aldenberg and Jaworska (2000). 

FIGURE 5.5 Bayesian posterior normal probability density function values for SSD for cadmium and its Bayesian confidence limits 5th, 50th, and 95th percentiles (black) and Bayesian posterior probability density of the HC5 (gray). 

To summarize, let 9 be the parameters Rq, Wq, a, and 3 and n be a probability density function. The distribution n describes the variability of the model parameters. The objective of a Bayesian hierarchical model is to generate the distributions of these parameters, based on all available information. 

---