Poisson data, normalization

Mathematical Models for Distribution Curves Mathematical models have been developed to fit the various distribution cur ves. It is most unlikely that any frequency distribution cur ve obtained in practice will exactly fit a cur ve plotted from any of these mathematical models. Nevertheless, the approximations are extremely useful, particularly in view of the inherent inaccuracies of practical data. The most common are the binomial, Poisson, and normal, or gaussian, distributions. 

Bialkowski, S. E., Data Analysis in the Shot Noise Limit 1. Single Parameter Estimation with Poisson and Normal Probability Density Functions, Anal. Chem. 61, 1989, 2479-2483. 

What about the question of whether a detector or counting system is working properly For example, the data in Table 18.2 do not exactly match a Poisson or normal distribution. Was the counting system malfunctioning One parameter that we can calculate that will help us answer such questions is x (chi squared). Formally,... 

Note 5. The relative standard deviation [RSD] of based on observed counts is for Poisson data, or about 6% for -E(N)=60. Equivalent precision for based on replication would require about 2fi or 120 degrees of freedom. The same is true for confidence intervals for n, hence, based on counts, vs based on replication. For more detail, Including the use of x bo derive both types (counts, replication) of Cl s see Ref. and the monograph by Cox and Lewis (100). Adequacy of the large count (normal) approximation, and the exact treatment for extreme low-level data (n 10 or less) are covered in Ref. and the references therein. 

There are many complications with interpreting MWCO data. First, UF membranes have a distribution of pore sizes. In spite of decades of effort to narrow the distribution, most commercial membranes are not notably sharp. What little is known about pore-size distribution in commercial UF membranes fits the Poisson distribution or log-normal distribution. Some pore-size distributions may be polydisperse. 

The principle of Maximum Likelihood is that the spectrum, y(jc), is calculated with the highest probability to yield the observed spectrum g(x) after convolution with h x). Therefore, assumptions about the noise n x) are made. For instance, the noise in each data point i is random and additive with a normal or any other distribution (e.g. Poisson, skewed, exponential,...) and a standard deviation s,. In case of a normal distribution the residual e, = g, - g, = g, - (/ /i), in each data point should be normally distributed with a standard deviation j,. The probability that (J h)i represents the measurement g- is then given by the conditional probability density function Pig, f) ... 

F(x), here, is (Es + AEs)/ Er + A )), as we just noted. In the previous case, the weighting function was the Normal distribution. Our current interest is the Poisson distribution, and this is the distribution we need to use for the weighting factor. The interest in our current development is to find out what happens when the noise is Poisson-distributed, rather than Normally distributed, since that is the distribution that applies to data whose noise is shot-noise-limited. Using P to represent the Poisson distribution, equation 49-59 now becomes... 

Fit data to a recognized mathematical distribution (e.g., normal, Poisson, binomial). When appropriate, transform the data (e.g., log 10 transformation). Calculate confidence limits. 

Poisson distribution can also be approximated by a normal distribution for a large number of counts. Therefore, nonlinear LS regression analysis is an efficient estimation procedure for the data even though the counts are Poisson distributed. 

Clearly these large fluctuations are due to cyclic variations not turbulent fluctuations. The dashed curve is an attempt to remove this cyclic variation effect by using the most probable density value as the mean value of a normal distribution. The standard deviation of the distribution is determined from fitting the data to the side of the new mean that has not been distorted by flame arrival. The reduction of the apparent fluctuations near the flame arrival crank angle is dramatic. Both curves of Figure 5 have had the Poisson statistical fluctuations subtracted. 

The variation that is observed in experimental results can take many different forms or distributions. We consider here three of the best known that can be expressed in relatively straightforward mathematical terms the binomial distribution, the Poisson distribution and the Gaussian, or normal, distribution. These are all forms of parametric statistics which are based on the idea that the data are spread in a specific manner. Ideally, this should be demonstrated before a statistical analysis is carried out, but this is not often done. 

The purpose of statistical evaluation of sample data is to extrapolate from a collection of individual events (e.g., 30 min of process time) to the entire population of events (e.g., 8-h shift). Because microbial monitoring data usually measure the impact of human activity, which is not reproducible exactly from one event to the next, results usually do not fit standard statistical models for normal distributions. In spite of this limitation, it is necessary to summarize the data for comparison to limits. The best statistical methods of evaluation are determined by the nature of the data. Wilson suggests that microbial monitoring data histograms generally resemble Poisson or negative... 

The Poisson distribution tends to symmetry as A increases. For A > 10, the Poisson distribution is reasonably well represented by a normal distribution. This has implications for analysis in cases where the mean number of counts is expected to be high, in which case traditional analyses for continuous data may be sufficient. 

Control charts for processes in which the data are not Normally distributed are also possible. In the ISO 8258 standard [39] examples of control charts are presented that are based on binomial and Poisson distributions. These distributions could also apply... 

A study is considered valid if the results obtained with positive and negative controls are consistent with the laboratory s historical data and with the literature. Statistical analysis is usually applied to compare treated and negative control groups. Both pairwise and linear trend tests can be used. Because of the low background and Poisson distribution, data transformation (e.g., log transformation) is sometimes needed before using tests applicable to normally distributed data. Otherwise, nonparametric analyses should be preferred. 

When a statistical population has a distribution that is approximately normal, a chi-square test can be performed on a random sample from the population to check its variance. Since the Poisson distribution is approximately normal whenever its mean is large enough (e.g., 20 or more), the chi-square test can be adapted to check whether the mean of a set of counting data equals its variance. For example, if a long-lived source is counted n times with the same detector to generate the counts Cl, C2,..., C , then one may calculate the chi-square statistic given by Eq. (10.16) ... 

Data requirements The software must allow for both deterministic and stochastic input Sampling from system-supplied distribution functions should include theoretical statisticed distributions such as exponential, normal, triangular, uniform, Poisson, beta, gama, erltmg, and... 

---