Uncertainty confidence limits

The assembly process (Figure 10-1) brings together all of the assessment tasks to provide the risk, its significance, how it was found, its sensitivity to uncertainties, confidence limits, and how it may be reduced by system improvements. Not all PSAs use fault trees and event trees. This is especially true of chemical PSAs that may rely on HAZOP or FMEA/FMECAs. Nevertheless the objectives are the same accident identification, analysis and evaluation. Figure 10-1 assumes fault tree and event tree techniques which should be replaced by the equivalent methods that are used. 

The LD q is calculated from data obtained by using small groups of animals and usually for only a few dose levels. Therefore, there is an uncertainty factor associated with the calculation. This can be defined by determining the 95% confidence limits for the particular levels of mortaUty of interest (Fig. 7). The 95% confidence limits give the dose range for which there is only a 5% chance that the LD q will be outside. 

If there is a lack of specific, appropriate data for a process facility, there can be considerable uncertainty in a frequency estimate like the one above. When study objectives require absolute risk estimates, it is customary for engineers to want to express their lack of confidence in an estimate by reporting a range estimate (e.g., 90% confidence limits of 8 X 10 per year to 1 X 10 per year) rather than a single-point estimate (e.g., 2 X 10per year). For this reason alone it may be necessary for you to require that an uncertainty analysis be performed. 

Uncertainty - displays distribution and confidence limits of a system, sequence, or end state for both base and current data values. 

Confidence limits for the conclusions cannot be expressed simply because of the complexity of hazard assessments. The estimation of uncertainties is itself a process subject to professional judgment. However, the team s estimates of probability are believed to be realistic, but may be pessimistic by a factor of perhaps two or three, but less than a factor of ten. Uncertainties also exist... 

If the lower values in the brackets are applied, an additional 0.5 uncertainty (error on 5% risk level) has to be added arithmetically to the flow coefficient confidence limits. The use of flow straighteners is recommended in cases when a nonstandard type of upstream fitting disturbs the flow velocity profile. 

Every measured quantity or component in the main equations, Eqs. (12.30) and (12.31), influence the accuracy of the final flow rate. Usually a brief description of the estimation of the confidence limits is included in each standard. The principles more or less follow those presented earlier in Treatment of Measurement Uncertainties. There are also more comprehensive error estimation procedures available.These usually include, beyond the estimation procedure itself, some basics and worked examples. 

If a result is quoted as having an uncertainty of 1 standard deviation, an equivalent statement would be the 68.3% confidence limits are given by Xmean 1 Sjc, the reason being that the area under a normal distribution curve between z = -1.0 to z = 1.0 is 0.683. Now, confidence limits on the 68% level are not very useful for decision making because in one-third of all... 

Example 57 The three files can be used to assess the risk structure for a given set of parameters and either four, five, or six repeat measurements that go into the mean. At the bottom, there is an indicator that shows whether the 95% confidence limits on the mean are both within the set limits ( YES ) or not ( NO ). Now, for an uncertainty in the drug/weight ratio of 1%, a weight variability of 2%, a measurement uncertainty of 0.4%, and fi 3.5% from the nearest specification limit, the ratio of OOS measurements associated with YES as opposed to those associated with NO was found to be 0 50 (n == 4), 11 39 (n = 5), respectively 24 26 (u = 6). This nicely illustrates that it is possible for a mean to be definitely inside some limit and to have individual measurements outside the same limit purely by chance. In a simulation on the basis of 1000 sets of n - 4 numbers e ND(0, 1), the Xmean. Sx, and CL(Xmean) were calculated, and the results were categorized according to the following criteria ... 

Confidence limits for the parameter estimates define the region where values of bj are not significantly different from the optimal value at a certain probability level 1-a with all other parameters kept at their optimal values estimated. The confidence limits are a measure of uncertainty in the optimal estimates the broader the confidence limits the more uncertain are the estimates. These intervals for linear models are given by... 

Determination of confidence limits for non-linear models is much more complex. Linearization of non-linear models by Taylor expansion and application of linear theory to the truncated series is usually utilized. The approximate measure of uncertainty in parameter estimates are the confidence limits as defined above for linear models. They are not rigorously valid but they provide some idea about reliability of estimates. The joint confidence region for non-linear models is exactly given by Eqn. (B-34). Contrary to ellipsoidal contours for linear models it is generally banana-shaped. 

There are a number of other problems relating to the manipulation and interpretation of data that cause difficulty. The most common are (i) uncertainty about the number of replicate results required for proper comparison of the certified reference value, and (2) the actual analytical result and how gross outlier results should be handled. These issues and how to deal with data that falls outside the confidence limit are reviewed in detail by Walker and Lumley (1999), who conclude that whilst customer requirements may provide answers the judgement of the analyst must always be the final arbiter in any decision ... 

Safety factors are used in engineering design to reduce the design load (or equivalent parameter) to allow for the predicted degradation, to allow for a confidence limit based on statistical evaluation, or to allow simply for uncertainty. It is clear that any predicted degradation should be allowed for in full. Confidence limits can be calculated. Uncertainties must be estimated. For example, to allow for the extrapolation necessary... 

In terms of confidence limits the two Grand Means can be written as 38.5 + 6.4 mg/m2 for the EC plot and 49.9 + 32.7 mg/m for the GF plot at the 90 level. This statement emphasizes the extent to which sampling variability can affect the confidence with which an analytical result is known. Unless the sampling program is designed to measure and identify the source of the variability much effort towards improvement of the quality of the chemical analyses can be wasted (4). The difficulty of improving the sampling procedures to reduce the variability is illustrated by calculation of the number of samples that would have to be analyzed to obtain estimates known to have an uncertainty less than 10 at the 90 confidence level (4). This would require 106 analyses from the EC plot and 2140 from the GF. Both sample sizes... 

The uncertainties refer to 95% confidence limits. These small error limits are indicative of the high precision of the readings obtainable from both instruments. 

Normally the population standard deviation a is not known, and has to be estimated from a sample standard deviation s. This will add an additional uncertainty and therefore will enlarge the confidence interval. This is reflected by using the Student-t-distribution instead of the normal distribution. The t value in the formula can be found in tables for the required confidence limit and n-1 degrees of freedom. 

Confidence intervals nsing freqnentist and Bayesian approaches have been compared for the normal distribntion with mean p and standard deviation o (Aldenberg and Jaworska 2000). In particnlar, data on species sensitivity to a toxicant was fitted to a normal distribntion to form the species sensitivity distribution (SSD). Fraction affected (FA) and the hazardons concentration (HC), i.e., percentiles and their confidence intervals, were analyzed. Lower and npper confidence limits were developed from t statistics to form 90% 2-sided classical confidence intervals. Bayesian treatment of the uncertainty of p and a of a presupposed normal distribution followed the approach of Box and Tiao (1973, chapter 2, section 2.4). Noninformative prior distributions for the parameters p and o specify the initial state of knowledge. These were constant c and l/o, respectively. Bayes theorem transforms the prior into the posterior distribution by the multiplication of the classic likelihood fnnction of the data and the joint prior distribution of the parameters, in this case p and o (Fignre 5.4). 

FIGURE 6.11 Kolmogorov-Smirnov confidence limits (black) acconnting for both mea-snrement uncertainty and sampling uncertainty about the p-box (gray) from Figure 6.9. 

Risk managers are interested in both variability and uncertainty they want to know how the expected impacts will vary (how frequent and widespread will impacts be ), and they want to know how certain the assessment is (how sure are you, what are the confidence limits ). 

Trueness or exactness of an analytical method can be documented in a control chart. Either the difference between the mean and true value of an analyzed (C)RM together with confidence limits or the percentage recovery of the known, added amount can be plotted [56,62]. Here, again, special caution should be taken concerning the used reference. Control charts may be useful to achieve trueness only if a CRM, which is in principle traceable to SI units, is used. All other types of references only allow traceability to a consensus value, which however is assumed not to be necessarely equal to the true value [89]. The expected trueness or recovery percent values depend on the analyte concentration. Therefore, trueness should be estimated for at least three different concentrations. If recovery is measured, values should be compared to acceptable recovery rates as outlined by the AOAC Peer Verified Methods Program (Table 7) [56, 62]. Besides bias and percent recovery, another measure for the trueness is the z score (Table 5). It is important to note that a considerable component of the overall MU will be attributed to MU on the bias of a system, including uncertainties on reference materials (Figures 5 and 8) [2]. 

In reading various HO measurement reports [including Hard et al. (78)], we find too little attention given to clearly expressing how the uncertainty limits are derived. This negligence may be the result of assuming such uncertainties are trivial to calculate. However, future reports should state explicitly whether la, 2a, 90% confidence limits, and so forth employ Gaussian or Poisson statistics and whether the quoted uncertainties are internal to the ambient HO data or include calibration uncertainties. 

The use of MLEs of probability coefficients, rather than upper confidence limits (UCLs), to classify waste can be justified, in part, on the grounds that the assumed exposure scenarios for hypothetical inadvertent intruders at waste disposal sites are expected to be conservative compared with likely on-site exposures at future times. However, uncertainties in probability coefficients should still be considered in classifying waste. When risk is calculated using MLEs of... 

The rationale supporting use of EDi0 as the benchmark dose is that a 10 percent response is at or just below the limit of sensitivity in most animal studies. Use of the lower confidence limit of the benchmark dose, rather than the best (maximum likelihood) estimate (EDio), as the point of departure accounts for experimental uncertainty the difference between the lower confidence limit and the best estimate does not provide information on the variability of responses in humans. In risk assessments for substances that induce deterministic effects, a dose at which significant effects are not observed is not necessarily a dose that results in no effects in any animals, due to the limited sample size. NOAEL obtained using most study protocols is about the same as an LED10. 

For the most frequently used low-dose models, the multi-stage and one-hit, there is an inherent mathematical uncertainty in the extrapolation from high to low doses that arises from the limited number of data points and the limited number of animals tested at each dose (Crump et al., 1976). The statistical term confidence limits is used to describe the degree of confidence that the estimated response from a particular dose is not likely to differ by more than a specified amount from the response that would be predicted by the model if much more data were available. EPA and other agencies generally use the 95 percent upper confidence limit (UCL) of the dose-response data to estimate stochastic responses at low doses. 

Although rarely presented in a dose-response assessment, in nearly all cases the lower bound on the incremental probability of a response will be zero or less (see Figure 3.7). That is, the statistical model that accounts for the uncertainty in the results of an animal study also accommodates the possibility that no response may occur at low doses and that, in fact, there may be fewer responses (e.g., cancers) than observed in the control population at some low doses. The possibility of reduced responses at low doses also is shown by the lower confidence limit of data on radiation-induced cancers in some organs of humans including, for example, the pancreas, prostate, and kidney (Thompson et al., 1994). 

---