Some Conclusions About Audits

It appears that audits can be made reliable. To gain this reliability appears to reqnire some training of the auditor and ensuring that the scoring criteria are operationally defined. If audits are to be used in a company, the audit proponents and practitioners should test reUabiUty constantly and should not pnblish resnlts until demonstrated reliability is achieved. 

The validity should also be tested whenever possible. Do the plants with the best long-term safety records have the best audit scores In this test it is essential that auditors not know the long-term record of plants they are auditing. Otherwise, the validity test is biased. 

We are inclined to believe that audits have some value. Both of us regularly take our cars to the dealer to have them go through the checklist. However, the role of an audit is usually to maintain conformity to rules, regulations, and standards. This is not nnimportant. However, it is nnUkely that the data obtained from audits will serve in a planning process to enable the development of targeted improvement actions. 

In order to serve as a key element in improvement planning, an audit process would need to have reliability, criterion-based validity, and construct-based validity. This is a hurdle that does not appear to have been cleared up to this point. 

---

There is some evidence that supports the criterion-based validity of other audits. Uusitalo and Mattila (1989) found a fair correlation between audit scores and accident rates in 11 industrial companies. There are other studies described by Uusitalo that show some evidence of validity. Pringle and Brown (1990) report a 12 percent drop in the incident rates among the 2,395 North American companies that used the ISRS in the period 1978-1979. The apparent lack of a control group of similar companies that did not use the ISRS and the fact that the improvonent was relatively small leads us to some reservation about the conclusion that the ISRS was responsible for this change. It would not be surprising to find that a sample of similar companies that used methods for safety improvement that did not involve audits showed the same 12 percent reduction in accident rates. 

---