Validity criteria, safety performance measurements

Audit instruments can be evaluated on the basis of three different types of validity when linking the audit to safety performance. These are content-related, criterion-related, and construct-related procedures for accumulating evidence of validity (Anastasi 1988, 139). Content-related validity involves the examination of the audit content to determine whether it covers a representative sample of the behaviors to be measured. Criterion-related validity procedures indicate the effectiveness of an audit in predicting performance in specified activities. The performance on the audit is checked against a criterion that is considered a direct and independent measure of what the audit was intended to measure. The third type of validity, construct validity, is the extent to which the audit is said to be measuring a theoretical construct or trait. Construct validation requires accumulation of information from a variety of sources. 

Validity relates to whether you are measuring what you want to measure. When we measure the width of a room, the question of validity usually does not arise. When we are measuring a complex process, such as aptitude to perform well in college or the ability of the safety-management system in a plant to prevent future loss, validity becomes a serious question. Scientists (Chronbach and Meeh 1955) generally define three categories of validity content-related, criterion-related, and construct-related validity. 

Validity is another question. We have said that audits typically have content-based validity. Testing for criterion-based validity may be beyond the scope of many situations for which safety professionals would like to apply audits. Certainly in a company with a small population and relatively low accident rates, there would probably not be enough data to find a relationship between audit scores and accident rates. An alternative would be to look for a relationship between audit scores and the scores on our safety survey (see Chapter 8). Since the scores of this survey are extensively validated as measures of system performance, they can serve as a criterion against which to test audits. 

---