Maximum likelihood point

Figure 3-3 The maximum likelihood point d i is a point having the maximum value of the probability distribution P (d). This point will be different from the mean value (d) in general.

The value d L is called the maximum likelihood point (see Figure 3-3). 

Effective MTBF (MTBFeff) excluding NFF cases -based as the maximum likelihood point estimate value of MTBF (MTBFeff = T /k) and upper and lower boundaries of the unknown parameter MTBF with the confidence level of 60%. 

Whereas standard linearized localization methods give a single point solution and uncertainty estimates (Schechinger and Vogel 2005), die result of the nonlinear method is a probability density function (PDF) over the unknown source coordinates. The optimal location is taken as the maximum likelihood point of the PDF. The PDF explicitly accounts for a priori known data errors, which are assumed to be Gaussian. 

In the maximum-likelihood method used here, the "true" value of each measured variable is also found in the course of parameter estimation. The differences between these "true" values and the corresponding experimentally measured values are the residuals (also called deviations). When there are many data points, the residuals can be analyzed by standard statistical methods (Draper and Smith, 1966). If, however, there are only a few data points, examination of the residuals for trends, when plotted versus other system variables, may provide valuable information. Often these plots can indicate at a glance excessive experimental error, systematic error, or "lack of fit." Data points which are obviously bad can also be readily detected. If the model is suitable and if there are no systematic errors, such a plot shows the residuals randomly distributed with zero means. This behavior is shown in Figure 3 for the ethyl-acetate-n-propanol data of Murti and Van Winkle (1958), fitted with the van Laar equation. 

Another class of methods such as Maximum Entropy, Maximum Likelihood and Least Squares Estimation, do not attempt to undo damage which is already in the data. The data themselves remain untouched. Instead, information in the data is reconstructed by repeatedly taking revised trial data fx) (e.g. a spectrum or chromatogram), which are damaged as they would have been measured by the original instrument. This requires that the damaging process which causes the broadening of the measured peaks is known. Thus an estimate g(x) is calculated from a trial spectrum fx) which is convoluted with a supposedly known point-spread function h(x). The residuals e(x) = g(x) - g(x) are inspected and compared with the noise n(x). Criteria to evaluate these residuals are Maximum Entropy (see Section 40.7.2) and Maximum Likelihood (Section 40.7.1). 

The principle of Maximum Likelihood is that the spectrum, y(jc), is calculated with the highest probability to yield the observed spectrum g(x) after convolution with h x). Therefore, assumptions about the noise n x) are made. For instance, the noise in each data point i is random and additive with a normal or any other distribution (e.g. Poisson, skewed, exponential,...) and a standard deviation s,. In case of a normal distribution the residual e, = g, - g, = g, - (/ /i), in each data point should be normally distributed with a standard deviation j,. The probability that (J h)i represents the measurement g- is then given by the conditional probability density function Pig, f) ... 

The parameter values found by the two methods differ slightly owing to the different criteria used which were the least squares method for ESL and the maximum-likelihood method for SIMUSOLV and because the T=10 data point was included with the ESL run. The output curve is very similar and the parameters agree within the expected standard deviation. The quality of parameter estimation can also be judged from a contour plot as given in Fig. 2.41. 

However, as pointed out by Read and others [23-27], use of residual as the target function is only justified for models that are very close to the true structure, which is often not the case in macromolecule refinements. An improved target function can be derived using the maximum likelihood formalism (MLF),... 

The basis upon which this concept rests is the very fact that not all the data follows the same equation. Another way to express this is to note that an equation describes a line (or more generally, a plane or hyperplane if more than two dimensions are involved. In fact, anywhere in this discussion, when we talk about a calibration line, you should mentally add the phrase ... or plane, or hyperplane... ). Thus any point that fits the equation will fall exactly on the line. On the other hand, since the data points themselves do not fall on the line (recall that, by definition, the line is generated by applying some sort of [at this point undefined] averaging process), any given data point will not fall on the line described by the equation. The difference between these two points, the one on the line described by the equation and the one described by the data, is the error in the estimate of that data point by the equation. For each of the data points there is a corresponding point described by the equation, and therefore a corresponding error. The least square principle states that the sum of the squares of all these errors should have a minimum value and as we stated above, this will also provide the maximum likelihood equation. 

One type of common robust estimator is the so-called M-estimator or generalized maximum likelihood estimator, originally proposed by Huber (1964). The basic idea of an M-estimator is to assign weights to each vector, based on its own Mahalanobis distance, so that the amount of influence of a given point decreases as it becomes less and less characteristic. 

These maximum likelihood methods can be used to obtain point estimates of a parameter, but we must remember that a point estimator is a random variable distributed in some way around the true value of the parameter. The true parameter value may be higher or lower than our estimate, ft is often useftd therefore to obtain an interval within which we are reasonably confident the true value will he, and the generally accepted method is to construct what are known as confidence limits. 

Mathematically, all this iirformation is used to calculate the best fit of the model to the experimental data. Two techniques are currently used, least squares and maximum hkelihood. Least-squares refinement is the same mathematical approach that is used to fit the best line through a number of points, so that the sum of the squares of the deviations from the line is at a minimum. Maximum likelihood is a more general approach that is the more common approach currently used. This method is based on the probability function that a certain model is correct for a given set of observations. This is done for each reflection, and the probabilities are then combined into a joint probability for the entire set of reflections. Both these approaches are performed over a number of cycles until the changes in the parameters become small. The refinement has then converged to a final set of parameters. 

The result of the PAM reconstruction is, in general, only an approximation of the exit wave function. Some non-linear terms may be present exactly on the paraboloid surfaces, and, thus result in artifacts for the PAM reconstruction. However, the PAM result is a good approximation to the exit wave function, which, in the present implementation, is used as a starting point for a maximum likelihood (MAL) reconstruction that takes the non-linear image contributions fully into account (Coene et al. 1996, Thust etal. 1996a). 

The estimator must be based on maximum likelihood estimators of the two disturbance variances, so they must be recomputed first. Our initial estimators of them are. v,2 = (1500/9)/50 = 3.3333 and s22 = (4200/9)/50 = 9.3333. Beginning from this point, we iterate between the estimator of the coefficient vector... 

We have considered maximum likelihood estimation of the parameters of this model at several points. Consider, instead, a GMM estimator based on the result that... 

The rationale supporting use of EDi0 as the benchmark dose is that a 10 percent response is at or just below the limit of sensitivity in most animal studies. Use of the lower confidence limit of the benchmark dose, rather than the best (maximum likelihood) estimate (EDio), as the point of departure accounts for experimental uncertainty the difference between the lower confidence limit and the best estimate does not provide information on the variability of responses in humans. In risk assessments for substances that induce deterministic effects, a dose at which significant effects are not observed is not necessarily a dose that results in no effects in any animals, due to the limited sample size. NOAEL obtained using most study protocols is about the same as an LED10. 

A comparison of the various fitting techniques is given in Table 5. Most of these techniques depend either explicitly or implicitly on a least-squares minimization. This is appropriate, provided the noise present is normally distributed. In this case, least-squares estimation is equivalent to maximum-likelihood estimation.147 If the noise is not normally distributed, a least-squares estimation is inappropriate. Table 5 includes an indication of how each technique scales with N, the number of data points, for the case in which N is large. A detailed discussion on how different techniques scale with N and also with the number of parameters, is given in the PhD thesis of Vanhamme.148... 

When experimental data is to be fit with a mathematical model, it is necessary to allow for the fact that the data has errors. The engineer is interested in finding the parameters in the model as well as the uncertainty in their determination. In the simplest case, the model is a linear equation with only two parameters, and they are found by a least-squares minimization of the errors in fitting the data. Multiple regression is just linear least squares applied with more terms. Nonlinear regression allows the parameters of the model to enter in a nonlinear fashion. The following description of maximum likelihood applies to both linear and nonlinear least squares (Ref. 231). If each measurement point has a measurement error Ayi that is independently random and distributed with a normal distribution about the true model y x) with standard deviation then the probability of a data set is... 

An alternate argument for minimizing S 6) is to maximize the function i( 6,a Y) given in Eq. (6.1-10). This maximum likelihood approach, advocated by Fisher (1925). gives the same point estimate 6 as does the posterior density function in Eq. (6.1-13). The posterior density function is essential, however, for calculating posterior probabilities for regions of 6 and for rival models, as we do in later sections of this chapter. 

Schmidt [16] thoroughly analyzed the problem of uncertainties in measuring the mean lifetime with a small number of detected nuclei, including the presence of a stochastic background. The crucial point was that the measurement was supposed to last until complete decay of the nuclide. The treatment was based on the maximum likelihood approach the 90 percent confidence intervals were tabulated. 

Furthermore, when alternative approaches are applied in computing parameter estimates, the question to be addressed here is Do these other approaches yield similar parameter and random effects estimates and conclusions An example of addressing this second point would be estimating the parameters of a population pharmacokinetic (PPK) model by the standard maximum likelihood approach and then confirming the estimates by either constructing the profile likelihood plot (i.e., mapping the objective function), using the bootstrap (4, 9) to estimate 95% confidence intervals, or the jackknife method (7, 26, 27) and bootstrap to estimate standard errors of the estimate (4, 9). When the relative standard errors are small and alternative approaches produce similar results, then we conclude the model is reliable. 

Data in these studies were generated from a so-called giant rat study in our laboratory. Animals were sacrificed to obtain serial blood and tissue samples. Each point represents the measurement from one individual rat and data from all these different rats were analyzed together to obtain a time prohle as though it came from one giant rat. A naive pooled data analysis approach was therefore employed for all model fittings using ADAPT II software (21). The maximum likelihood method was used with the variance model specified as V(a, 6, h) = (j Y(d, where V a, 9, ti) is the variance for the ith point, Y 6, t,) is the ith predicted value from the dynamic model, 9 represents the estimated structural parameters, and oi and 02 are the variance parameters that were estimated. 

In order to avoid the disadvantages, seen or inferred, of the simple addition of q values, various analysts have either calculated or assumed a distribution (for each tumor type) representing the likelihood for the plausible range of estimates of the linear term (q ) of the multistage model (qi), and then they used the Monte Carlo procedure to add the distributions rather than merely adding specific points on the distributions such as the maximum likelihood estimate (MLE) or 95% confidence limit. A combined potency estimate (q for all sites) is then obtained as the 95% confidence limit on the summed distribution. This resembles the approach for multiple carcinogens by Kodell et al. (1995) noted above. 

As an example, consider the data set Y = — 2.1, -1.6, - 1.4, -0.25, 0, 0.33, 0.5, 1, 2, 3. The likelihood function and log-likelihood function, assuming the variance is known to be equal to 1, are shown in Fig. A.6. As p increases from negative to positive, the likelihood increases, reaches a maximum, and then decreases. This is a property of likelihood functions. Also note that both functions parallel each other and the maxima occur at the same point on the x-axis. Intuitively one can then see how maximum likelihood estimation works. Eyeballing the plot shows that the maximum likelihood estimate (MLE) for p is about 0.2. 

The ability of diode array spectrometers to acquire data rapidly also allows the use of measurement statistics to improve the quantitative data. For example, 10 measurements can be made at each point in one second, from which the standard deviation of each point is obtained. The instrument s computer then weights the data points in a least-squares fit, based on their precisions. This maximum-likelihood method minimizes the effect of bad data points on the quantitative calculations. 

---