Cramer-Rao bound

One of the major questions that should always be asked of any estimation technique is whether is is optimal. This problem can be addressed using Cramer-Rao bounds and we illustrate this in Section 24.4. We conclude this chapter in Section 24.5 with the application of these statistical techniques to the Shack-Hartmann and curvature sensors. 

A. Nehorai A. Dogandzic. Cramer-Rao bounds for estimating range, velocity and direction with an active array. IEEE transactions on Signal Processing, 49(6) 1122—1137, June 2001. 

The essential criteria for a good fit are that the returned parameters should be as accurate as permitted by the data and that the fitting process should be robust. The theoretical limit on the uncertainties of each of the parameters in the model is given by the Cramer Rao lower bound. The Cramer Rao bound applies to any unbiased estimator 0(y) of a parameter vector, 9, using measurements, y. The measurements are described by their joint probability density function p(y 6), which is influenced by 9. 

Providing p y 9) is sufficiently well behaved the Cramer-Rao bound is given by... 

In an analysis of errors, Barkhuijsen and colleagues109 showed that the Cramer-Rao bounds can be used as estimates of the random errors in the parameter estimates. 

Provided the estimator is unbiased, this expression indicates that the diagonal elements of the inverse information matrix, the Cramer-Rao bound (CRB), give the highest attainable variance of the unknown parameters. For zero-mean noise and a correct model r, no difference is expected between the values of the estimates and the true values. These minimal variances depend on the pdf of the signal, because the Fisher information matrix I is given by... 

Bergman N, Doucet A, Gordon N (2001) Optimal estimation and Cramer Rao bounds for partial non-Gaussian state space models. Ann I Stat Math 53(1) 97-112... 

Abstract Wavefront sensing for adaptive optics is addressed. The most popular wavefront sensors are described. Restoring the wavefront is an inverse problem, of which the bases are explained. An estimator of the slope of the wavefront is the image centroid. The Cramer-Rao lower bound is evaluated for several probability distribution function... 

Finally we need to compare the variance of our estimator with the best attainable. It can be shown that The Cramer-Rao lower bound (CRLB) is a lower bound on the variance of an unbiased estimator (Kay, 1993). The quantities estimated can be fixed parameters with unknown values, random variables or a signal and essentially we are finding the best estimate we can possibly make. 

It should be pointed out that the use of the Fisher matrix here is an approximation. It really corresponds to the Cramer-Rao lower bound on the estimator for the target from this measurement. It can be shown that the estimator here is asymptotically efficient (see[2], pp. 38-39) in that the covariance matrix approaches the Cramer-Rao lower bound over a large number of measurements (loc. cit.). 

The parameters A,k and b must be estimated from sr The general problem of parameter estimation is to estimate a parameter, 0, given a number of samples, x,-, drawn from a population that has a probability distribution P(x, 0). It can be shown that there is a minimum variance bound (MVB), known as the Cramer-Rao inequality, that limits the accuracy of any method of estimating 0 [55]. There are a number of methods that approach the MVB and give unbiased estimates of 0 for large sample sizes [55]. Among the more popular of these methods are maximum likelihood estimators (MLE) and least-squares estimation (LS). The MLE... 

Before a fitting routine is used on experimental data, it should always be tested first on trial data with known parameters and SNR. For each set of parameters for a given SNR, a set of, say, 50 spectra should be generated with different noise having the same standard deviation.32 The distribution of each fitted parameter should be checked to verify that it is free from bias, relative to its known value, as shown in Fig. 17. If the fitting procedure has been correctly applied, then the uncertainties in the fitted parameters should satisfy the Cramer-Rao lower bounds. This procedure also checks the robustness of the fitting technique. 

Liu XQ, Sidiropoulos ND, Cramer-Rao lower bounds for low-rank decomposition of multidimensional arrays, IEEE Transactions on Signal Processing, 2001,49, 2074—2086. 

The Cramer-Rao lower bound for a parameter estimate provides a bound on how low the variance of the estimated parameters can be. Achieving the lower bound implies that we have a minimum variance estimate. The Cramer-Rao lower bound is defined as... 

A parameter estimator achieving the Cramer-Rao lower bound will have an efficiency of 1. 

The variance of the estimate attains the Cramer-Rao lower bound ... 

---