Expert judgements

Lawrence, D., and D. W. Edwards (1994). "Inherent Safety Assessment of Chemical Process Routes by Expert Judgement. The 1994 IChemE Research Event, 886-88. 

The considerably higher effort of carrying out LCAs would be worthwhile in two situations. First, individual LCAs may be valuable in cases where complex and potentially relevant trade-offs occur in mass balancing which cannot be resolved by expert judgement. Second, the routine use of LCA could be aimed for. This would necessitate a dedicated effort to compile inventory data for frequently used processes, such as waste treatment. Once this is achieved, calculating LCAs for existing or new processes is relatively easy. 

Edwards and Lawrence (1993) have presented a list of sixteen chemical and physical properties and process parameters which are available at the process route selection stage (Table 5). Seven of these sixteen parameters were included to their index method (PUS). The selected parameters concentrate very much on the chemical process route and chemistry. They have also tested their selection by an expert judgement, which gave support to their work (Edwards et al., 1996). 

In general, only valid studies were used to derive predicted no effect concentrations (PNECs). Because in some cases only a few valid data were available, studies valid with restrictions have been used based on expert judgement. Invalid or non-assignable studies have not been used. In instances of volatile compounds, valid studies were generally those using closed, flow-through systems, preferably with analytical measurements. 

Expert judgment A critical source of information based npon the collective experience of a scientist or expert in a particular field of study. For Bayesians, expert judgement is frequently used to form the prior distribution, thns formally incorporating an expert s degree of belief into statistical procednres. 

This method makes use of a test battery to derive a toxicity index that can be employed to classify effluents as a function of their overall toxicity. A formula is given as an example and a procedure to calculate the index using expert judgements and a PLS (Partial Least Square) regression procedure is described using data on 30 effluents. 

This was carried out by suppressing tests and calculating the resulting sum of squares of deviation of the index from expert judgements. The more the sum of squares increases after suppression of one test, the more important the test is. 

The wide range in EC10 values obtained for the different effluents is shown in Table 4. Some effluents were found non toxic with some tests. Since a numerical value for computations was needed, the value of 100% as a "virtual" EC 10 was used. It only served for computation but not for the expert judgements. 

Table 7. Average value of the experts judgements on the toxicity and genotoxicity of the effluents. The experts had to rank their relative hazard as an integer varying from 1 to 5 (not toxic = 1, highly toxic = 5).

Table 8. Average experts judgements on the adequacy of bioassays and other parameters in term of their usefulness for effluent toxicity assessment. Each expert had to rank usefulness as an integer value between 1 (least useful) to 5 (most useful) and they were also allowed to suggest other tests or parameters deemed useful. Some experts recommended tests without numerical ranking (Lemna minor and chronic Daphnia magna).

The result of this first analysis is shown in Fig. 6. The graph shows a fairly good linear relationship between the experts judgement and the calculated index. This can be visualized by the position of the circles compared to the line y = x. This comparative analysis can also be used to pass judgment on the proposed battery of tests, with the degree of departure from the line being a criterion to estimate its adequacy. 

The last step in calculating an index was to simplify the numerical values of the coefficients from each test in order to have an easier way to calculate the model. Fig. 7 compares the calculated results with the experts judgements. We also considered the possibility of including effluent flow in the index formula such that the quantity of toxic material (/. e., toxic loading) discharged to the receiving environment can be estimated. 

Figure 6. Calculated index using all tests versus the expert judgement. The small dots represent the individual experts judgements large squares show the average of experts judgements as an abscissa and the index value as an ordinate for each effluent. The line drawn shows the identity y =x.

Fig. 7 also compares the values of the experts judgement index with the corresponding Canadian PEEP index values (see Chapter 1 of this volume). It shows that the value differences for both indexes are minor and suggests that the index defined in this work was a good estimator of an expert consensus. 

Figure 7. Comparison of the index that we developed (called index ) and the Canadian PEEP index. The correlation coefficient between the two indexes is 0.95. In both cases we draw a regression line to show the good agreement between the experts judgements and the indexes.

An index of toxicity is intended to be a simple tool that allows integrating and summarizing several variables into a single value. Realistically, this cannot be inferred without a judgement by environmental protection experts who consider all parameters available for their classification. PLS regression helped calculate an index fitted to expert judgement. The loss of information owing to the transformation of a multivariate situation to a univariate one was thus minimized since it is an inherent characteristic of multivariate analytical tools. 

Vindimian, E., Garric, J., Flammarion, P., Thybaud, E. and Babut, M. (1999) An index of effluent aquatic toxicity designed by PLS regression, using acute and chronic tests and expert judgements, Environmental Toxicology and Chemistry 18, 2386-2391. 

An aspect of matrix RMs which is of considerable importance is the question of commutability or horizontal traceability. This refers to the scope of the materials, i.e. the extent to which a matrix RM of a particular composition may reliably be used to evaluate a measurement procedure that is applied to a routine test sample of a different composition. The differences in composition between a reference matrix and a routine test sample matrix must not cause the two materials to behave differently when a particular analytical method is applied. At present, the extent to which this is true is largely a matter of expert judgement based on knowledge of the measurement application. A better and more systematic understanding of the factors affecting horizontal traceability will enable users to select appropriate matrix RMs more reliably and producers to target their production activities more efficiently. 

BOX 5.2 Example of a spreadsheet calculation of toxic risk (msPAF) for a species assemblage in an imaginary aquatic pond as the result of exposure to a mixture of toxicants with diverse and species-dependent toxic modes of action. Note Overall risk values (msPAF) per species group were calculated assuming concentration addition within common modes of action and response addition between modes of action. This example only serves to demonstrate the method of calculation. The SSD information on the mixture constituents as well as the total and bioeffective concentrations in pond water were randomly selected by realistic expert judgement. The gray cells contain examples of the formulas applied. 

Among others, crucial to the human expert judgement is (a) access to all the relevant literature, and (b) the ability to search across toxicity databases using both biological and chemical criteria. 

Commercially available prediction software packages can be a useful tool to support expert judgement, provided that they offer transparent predictions and not black-box responses. 

Are the estimates based upon measurements, modelling or expert judgement ... 

Data uncertainties depend on the type of data being used in the model, such as surrogate data, expert judgement, default data, modelled (extrapolated) data, etc. ... 

---