Confidence in the Database

In the hazard assessment, it is important to evaluate the toxicological database with regard to its adequacy. The adequacy of a study includes its validity and its relevance. The relevance refers to what has been studied in relation to what is needed for the hazard and risk assessment, and the validity refers to how the study was performed, e.g., conforming with a particular test guideline. The validity and the relevance of a study, or a whole database, has to be considered in relation to the reliability and thus the confidence. The data for hazard assessment are described in detail in Chapter 3. 

It has been suggested to apply an assessment factor for the confidence in the database in case there are limitations in the database, including lack of data for children, which are important in relation to the purpose of the assessment. This section gives an overview of such proposals and evaluations. Then, the key issues are summarized and our recommendations are presented. The question of an extra assessment factor in the hazard and risk assessment for chemicals of concern for children is specifically addressed in Section 5.2.1.13. 

Dourson et al. (1996) noted that if data are only available from one chronic study on which to base the estimation of a sub-threshold dose, the question could be asked whether data from chronic studies in other species or data from different types of bioassays (e.g., reproductive or developmental toxicity) would yield lower NOAELs. The uncertainty related to this issue must therefore be addressed and, according to the authors, the default approach to address this uncertainty is to apply a 3- or 10-fold UF, based on the assumption that the critical effect can be discovered in a reasonably small selection of toxicity studies. With a reference to some analyses performed within this area, the authors suggested the use of a UF to account for missing bioassays however, the quantification of this UF was considered to require additional work. 

High degree of confidence The database contains high quality human or animal studies, i.e., two or more studies with the same endpoint. The database should be sufficiently extensive to give confidence that the correct critical effect has been selected, and that there are no major uncertainties in this respect. No additional numerical UF required, i.e., the default factor is 1. 

Medium degree of confidence The database falls short of the quality described above in some significant respect, which limits the overall confidence to medium . Assess on a case-by-case basis, perhaps consider to use a low numerical UF, in the range of 1-2. 

---

The EU TGD (EC 2003) pointed out that the overall confidence in the database needs to be considered in the evaluation of the MOS (can be interpreted as an overall assessment factor). 

TNO has stated that the size, quality, completeness, and consistency of the database should be considered (Hakkert et al. 1996). Major aspects for the evaluation of the quality of the data supporting the NOAEL are (1) deviations from official guidelines, which are not properly substantiated, (2) number of animals used, (3) number of dose levels tested, and (4) adequacy of hematological, biochemical, and pathological examinations. Indications for doubts on the confidence in the database are (1) the absence of certain types of smdies, (2) conflicting results between studies, and (3) doubts on the reliability of the route-to-route extrapolation. However, consistency of results from different studies, consistency of animal and human data, and rehable mechanistic data are indicative for a high-confidence database. The default assessment factor for confidence of the database is 1. 

Regarding consistency of the database, conflicting results between studies are an indication of a lower confidence in the database, whereas consistency of results from different studies, consistency of animal and human data, and reliable mechanistic data are indicative of a high-confidence database. 

In conclusion, the uncertainty related to the confidence in the database should be taken into account by the use of an assessment factor. Since the quality, completeness, and/or consistency of different databases vary, the assessment factor will also vary and can only be assigned on the basis of expert judgment, preferably made transparent through the application of a set of criteria. In any case, the size of the factor should be considered in terms of other information in the database. The default value should be 1 in case of a high-confidence database, and a factor of 10 would be appropriate where major deficiencies in the data exist, e.g., a lack of chronic and reproductive toxicity studies when setting a tolerable intake. 

The confidence in RfD is a composite of the confidence in the principal study and in the database. In assigning confidence to RfD, EPA gives precedence to confidence in the database. If the principal study has a medium confidence rating and the database a low confidence rating, EPA assigns low confidence to RfD. The confidence level is a part of dose-response characterization discussed in Section 3.2.1.4. 

Figure 6. Statistics for 5 selected substructures of the 500 tested on the EPA IR database. Values of the Reliability, False Positives, and Recall (see text) are compared at the 45% confidence level. The number of compounds in the database containing each substructure is given beneath the substructure name. Note the expanded scale used to plot the False Positive measure.

Average results for 500 IR-active substructures are shown in Figure 7 at four different confidence levels. The average compound in the database contains 8.1 of the 500 substructures. At a confidence level of > 45%, only 1.4 (of 492) incorrect substructures are reported, while 4.6 of 8.1 substructures actually present are reported. In other words, a t3rpical" analysis will report 6.0 substructures at > 45% confidence, of which 4.6 are correct. 3.5 substructures actually present in the compound will fail to be reported. In an actual analysis, infrared data is combined with other t3rpes of data, so that many of the substructures undetected by infrared would be found by other techniques. 

Comparative Toxicokinetics. Based on available data, there do not appear to be significant differences in the toxicokinetics of barium between species (Chou and Chin 1943 Cuddihy and Griffith 1972 McCauley and Washington 1983). However, there are not enough similar studies on different species to determine this with certainty. Studies on different species would increase confidence in the reliability of the existing database. 

LOAEL where a NOAEL is not available, and database deficiencies. Lack of reproductive and developmental toxicity data is often used as a basis for including a database uncertainty factor. The default value for any one uncertainty factor is 10, but this may be reduced depending on the confidence in the data or information that provides assurance of reduced intra- or interspecies variability (Renwick et al., 2000). As noted above, chemical-specific data on toxicokinetics and toxicodynamics may be used to replace part or all of these uncertainty factors, and this strategy has been used by WHO/IPCS (IPCS, 1994, 2004b, 2005). 

It is important that both the qualitative and quantitative characterization be clearly communicated to the risk manager. The qualitative characterization includes the quality of the database, along with strengths and weaknesses, for both health and exposure evaluations the relevance of the database to humans the assumptions and judgements that were made in the evaluation and the level of confidence in the overall characterization. The quantitative characterization also includes information on the range of effective exposure levels, dose-response estimates (including the uncertainty factors applied), and the population exposure estimates. Kimmel et al. (2006) reviewed many of the components of the risk characterization for reproductive and developmental effects and provided a comprehensive list of issues to be considered for each of the components of the risk assessment. 

UFs for reproductive and developmental toxicity applied to the NOAEL often include 10-fold factors for interspecies and intraspecies variation. Additional factors might be applied to account for other uncertainties or for additional information that might exist in a database. For example, in circumstances in which only a LOAEL is available, it might be necessary to use an additional UF uncertainty factor of up to 10, depending on the sensitivity of the endpoints evaluated, the adequacy of the tested dose, or general confidence in the LOAEL. An additional uncertainty factor of 3-10 has been used by EPA (1996a) to account for database deficiencies, particularly the lack of reproductive and developmental toxicity studies. 

Generic databases, snch as PHED, EUROPOEM, ARTF and others, have been developed to increase the confidence in the exposure estimates since many more data points from a wide range of different stndies are included. Details on how these databases are constrncted, the different ways in which the data are normalized and recommendations on how databases can be improved and harmonized are discussed in Chapters 5 and 6. 

In 1995, the EU commissioned Maurice Palmer Associates (MPA) to undertake a survey of packaging of foodstuffs across Europe. They primarily focused on the UK and Italy, but the data were not correlated with foodstuff consumption. The data collected from the UK and Italy were extrapolated to the remaining 15 EU Member States, but because many assumptions were necessary, the confidence in the results varied from about 60% for Ireland, Einland and Sweden to a high of 90% for the UK market (ILSI 1996). MPA concluded that the different databases for food packaging materials for 15 EU Member States had a considerable amount of detailed information and probably contained far more detail than would be needed to derive food consumption factors, although the information could if necessary be refined on a country by country market share or packaging type basis. It is believed that this data has not been put to many, if any, uses to date as far as packaging of foodstuffs consumed is concerned. 

The Subcommittee on Permissible Exposure Levels for Military Fuels judged that, on the basis of available data, DOD s PEL of 350 mg/m3 for the fuel vapors is adequate to protect the health of naval personnel exposed to them occupationally (NRC 1996). However, because of uncertainties in the database, the PEL should still be considered interim until further research has been completed. The subcommittee recommended that data be obtained on exposures during operational procedures, including exposure to respirable aerosols of unburned fuels that studies be conducted on the possible effects of high-level acute and low-level chronic exposure to fuel vapors on the central nervous system and that research be conducted on the effect of fuel vapors on hepatotoxicity in experimental animals to help to identify a no-observed-adverse-effect level for JP-8 with greater confidence. 

---