Maximum likelihood solution

Figures 4b and 4c show that neither unconstrained nor non-negative maximum likelihood approaches are able to recover a usable image. Deconvolution by unconstrained/constrained maximum likelihood yields noise amplification — in otfier words, the maximum likelihood solution remains iU-conditioned (i.e. a small change in the data due to noise can produce arbitrarily large changes in the solution) regularization is needed.

We have seen that the maximum likelihood solution ... 

For noiseless projections, it has been shown that each OSEM estimate based on one subset of projections converges to a maximum likelihood solution as close as a full iteration of MLEM using all projections in the subset (Hudson and Larkin, 1994). It is this feature of OSEM that accelerates the computation process, and in general, the computation time is shortened with decreasing number of subsets (i.e., with more projections in each subset). However, there is a tendency of having more image variance with increasing number of subsets when compared to MLEM. So an optimum number of subsets need to be chosen. 

In other words, a maximum likelihood estimate will be found at that value of 0 where the derivative is zero. Notice that the phrase a maximum likelihood estimate will be found was used, not the phrase the maximum likelihood estimate will be found. It is possible for more than one maximum likelihood solution to exist since the derivative can also be at a minimum when it is equal to zero. Technically, 0 should be verified for a maximum by checking the second derivatives, but this is rarely done. 

The prior distribution p 0 C) denotes the prior information of the parameters and it is based on previous knowledge or user s judgement. In some applications, the prior distribution is treated as a constant and it is absorbed into the normalizing constant but this type of prior distribution does not satisfy the property of the PDF that its integral throughout the parametric space is unity. In general, a prior distribution that does not satisfy this property is referred to as an improper prior. Using a constant improper prior distribution yields the maximum likelihood solution. 

Prior distribution does not significantly affect the parametric identification (both identified values and associated uncertainty) results if it is sufficiently fiat in the range with significant likelihood values. Therefore, it is common to absorb the prior distribution into the normalizing constant and the results are equivalent to the maximum likelihood solution. However, it is not appropriate to absorb the prior distribution into the normalizing constant for model class... 

The approach just described can be stated as one which generates the most likely observations from the models in the normal sense. The problem with this is that the states always generate their mean values, which results in jumps at state boundaries. A solution to this problem was presented in a series of articles by Tokuda and colleagues [453], [452], [454] which showed how to generate a maximum likelihood solution that took the natural dynamics of speech into account. The key point in this technique is to use the delta and acceleration coefficients as constraints on what observations can be generated. 

Fig. 6.25. Nonlinear localization results considering the air-filled duct (confidence levels in gray scales and maximum likelihood solution as stars) compared to hnea-rized results for a plain concrete model (black circles as point sonrce with error ellipsoid).

The importance of distinct a priori knowledge account becomes more perceptible if noisy data are under restoration. The noise / ( shifts the solution of (1) from the Maximum Likelihood (ML) to the so called Default Model for which the function of the image constraint becomes more significant. 

The maximum likelihood (ML) solution is the one which maximizes the probability of the data y given the model among all possible x ... 

In the hope that additional constraints such as positivity (which must hold for the restored brightness distribution) may avoid noise amplification, we can seek for the constrained maximum likelihood (CML) solution ... 

Image Space Reconstruction Algorithm. ISRA (Daube-Witherspoon and MuehUehner, 1986) is a multiplicative and iterative method which yields the constrained maximum likelihood in the case of Gaussian noise. The ISRA solution is obtained using the recursion ... 

When started with a smooth image, iterative maximum likelihood algorithms can achieve some level of regularization by early stopping of the iterations before convergence (see e.g. Lanteri et al., 1999). In this case, the regularized solution is not the maximum fikelihood one and it also depends on the initial solution and the number of performed iterations. A better solution is to explicitly account for additional regularization constraints in the penalty criterion. This is explained in the next section. 

While it is perfectly permissible to estimate a and b on this basis, the calculation can only be done in an iterative fashion, that is, both a and b are varied in increasingly smaller steps (see Optimization Techniques, Section 3.5) and each time the squared residuals are calculated and summed. The combination of a and b that yields the smallest of such sums represents the solution. Despite digital computers, Adcock s solution, a special case of the maximum likelihood method, is not widely used the additional computational effort and the more complicated software are not justified by the improved (a debatable notion) results, and the process is not at all transparent, i.e., not amenable to manual verification. 

The above implicit formulation of maximum likelihood estimation is valid only under the assumption that the residuals are normally distributed and the model is adequate. From our own experience we have found that implicit estimation provides the easiest and computationally the most efficient solution to many parameter estimation problems. 

These considerations raise a question how can we determine the optimal value of n and the coefficients i < n in (2.54) and (2.56) Clearly, if the expansion is truncated too early, some terms that contribute importantly to Po(AU) will be lost. On the other hand, terms above some threshold carry no information, and, instead, only add statistical noise to the probability distribution. One solution to this problem is to use physical intuition [40]. Perhaps a better approach is that based on the maximum likelihood (ML) method, in which we determine the maximum number of terms supported by the provided information. For the expansion in (2.54), calculating the number of Gaussian functions, their mean values and variances using ML is a standard problem solved in many textbooks on Bayesian inference [43]. For the expansion in (2.56), the ML solution for n and o, also exists, lust like in the case of the multistate Gaussian model, this equation appears to improve the free energy estimates considerably when P0(AU) is a broad function. 

Thus, when the attention of the mathematicians of the time turned to the description of overdetermined systems, such as we are dealing with here, it was natural for them to seek the desired solution in terms of probabilistic descriptions. They then defined the best fitting equation for an overdetermined set of data as being the most probable equation, or, in more formal terminology, the maximum likelihood equation. 

Now, if (m2 > g), the solution of Eq. (10.24), under the assumption of an independent and normal error distribution with constant variance can be obtained as the maximum likelihood estimator of d and is given by... 

---