Maximum likelihood principle

As indicated in Chapter 6, and discussed in detail by Anderson et al. (1978), optimum parameters, based on the maximum-likelihood principle, are those which minimize the objective function... 

The method used here is based on a general application of the maximum-likelihood principle. A rigorous discussion is given by Bard (1974) on nonlinear-parameter estimation based on the maximum-likelihood principle. The most important feature of this method is that it attempts properly to account for all measurement errors. A discussion of the background of this method and details of its implementation are given by Anderson et al. (1978). 

If this criterion is based on the maximum-likelihood principle, it leads to those parameter values that make the experimental observations appear most likely when taken as a whole. The likelihood function is defined as the joint probability of the observed values of the variables for any set of true values of the variables, model parameters, and error variances. The best estimates of the model parameters and of the true values of the measured variables are those which maximize this likelihood function with a normal distribution assumed for the experimental errors. 

The criterion for best fit is based on the maximum likelihood principle (Fisher 1922) where the best estimates of the model parameters should maximise the likelihood function, L, for the observation of N different experimental observations. 

The set of measurements of p. is the most probable set (maximum likelihood principle). 

While the least squares estimator appeared several centuries ago as an independent method giving good results under certain assumptions, we have to dig deeper into mathematical statistics to see its roots, and, in particular, its limitations. This general subject is the maximum likelihood principle, one of the basic concepts of mathematical statistics. The principle is simple ... 

Though the maximum likelihood principle is not less intuitive than the least squares method itself, it enables the statisticans to derive estimation criteria for any known distribution, and to generally prove that the estimates have nice properties such as asymptotic unbiasedness (ref. 1). In particular, the method of least absolute deviations introduced in Section 1.8.2 is also a maximum likelihood estimator assuming a different distribution for the error. 

Since the final form of a maximum likelihood estimator depends on the assumed error distribution, we partially answered the question why there are different criteria in use, but we have to go further. Maximum likelihood estimates are only guaranteed to have their expected properties if the error distribution behind the sample is the one assumed in the derivation of the method, but in many cases are relatively insensitive to deviations. Since the error distribution is known only in rare circumstances, this property of robustness is very desirable. The least squares method is relatively robust, and hence its use is not restricted to normally distributed errors. Thus, we can drop condition (vi) when talking about the least squares method, though then it is no more associated with the maximum likelihood principle. There exist, however, more robust criteria that are superior for errors with distributions significantly deviating from the normal one, as we will discuss... 

The above criterion can be derived from the maximum likelihood principle (refs. 33-34). 

S. , J. Manczinger, S. Skold-Jorgensen and K. T6th, Reduction of thermodynamic data by means of the multiresponse maximum likelihood principle, AIChE J. 28 (1982) 21-30. 

Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory, pages 267-281. Akademia Kiado, Budapest. 

The calculated values Ycalc are given by the evaluation of the integral in Eq. (13) using an estimate of the distribution function, P(r). The weighting matrix W is chosen on the basis of maximum-likelihood principles 12 in our work, we assume that the errors have mean zero and are identically distributed, so that W is the identity. 

The training problem determines the set of model parameters given above for an observed set of wavelet coefficients. In other words, one first obtains the wavelet coefficients for the time series data that we are interested in and then, the model parameters that best explain the observed data are found by using the maximum likelihood principle. The expectation maximization (EM) approach that jointly estimates the model parameters and the hidden state probabilities is used. This is essentially an upward and downward EM method, which is extended from the Baum-Welch method developed for the chain structure HMM [43, 286]. 

Hence, Akaike, who considered the AIC ... a natural extension of the maximum likelihood principle, created an empirical function that linked Kullback-Leibler distance to maximum likelihood, thereby allowing information theory to be used as a practical tool in data analysis and model selection. 

Chylla RA, Markley JL (1995) Theory and application of the maximum likelihood principle to NMR parameter estimation of multidimensional NMR data. J Biomol NMR 5 245-258... 

If the Bayes strategy should be used instead of the maximum likelihood principle the a priori probabilities p(m) of all classes m must be considered. For this purpose, the right side of equation (83) is multi-lied by p(m). The ratio of Bayes probabilities for a binary classification is given in equation (86) C333]. 

The maximum likelihood principle is based on the following train of thought. Suppose that the shape of the model fimction is exactly known (in principle), but the exact value of the parameter vector a (providing the fitting function common to all spectra that could be measured under the given conditions) remains hidden from the experimenter. One can only state with certainty (because of physical considerations) that such a parameter vector does exist. The measured spectrum... 

The above maximum condition is an expression of the maximum likelihood principle. Note that the maximum likelihood principle - rationality aside - does not provide a guarantee that the real a will be found, since it is based on belief rather than on strict mathematical foundations. However, the results shown below can be interpreted mathematically as well, speaking for the strength of the principle. 

The condition Q = minimum (10.1.26) for the adjusted measured values is, in general terms, the condition (9.1.14). It can be again motivated as a minimum distance condition, or via the maximum likelihood principle (9.1.12) where 9A is the set of x obeying the solvability condition. In the latter case, the distribution of measurement errors is assumed Gaussian and with zero mean then the condition is derived in the same manner as after formula (9.1.13). Observe again that the covariance matrix F is assumed constant, thus also independent of the true values of the variables. Then the problem generally reads as follows. 

Rather academic is the problem of reconciliation when applying rigorously the maximum likelihood principle to a general (not Gaussian) probability distribution. One then assumes a given (twice continuously differentiable) probability density for the random (vector) variable e of measurement errors and solves the problem (9.1.12)... 

---