Safety-audit validity

A one-page, memory-jogging safety assessment checklist was made available by Trevor Kletz in 1976. [6] The checklist is still valid today. After a review of several Management of Change policies from several major companies, it appears that Kletz s checklist or a similar checklist was used as a basis for a few companies procedures, Frank Lees [14] has cataloged and presented a number of checklists in Hazard Identification and Safety Audit. Lees states ... 

Advertising successes of the process safety improvement effort demonstrates that improvement is possible. Well-crafted stories also explain the benefits that accrue to everyone in the organization. Of particular interest are stories where a process safety weakness was observed, possibly during a process safety audit, and an improvement effort corrected the identified weakness before it could manifest into an accident. Metrics can validate such improvements. Another example is improved reliability from timely maintenance of safety devices as demonstrated by metrics that educate personnel not only about the hazards, but also about the importance of reliable safety systems in managing those hazards. 

The safety audit is a review by the FMCSA of the carrier s written procedures and records to validate the accuracy of information and certifications provided in the application and determine whether the carrier has established or exercised the basic safety management controls necessary to ensure safe operations. The FMCSA will evaluate the results of the safety audit using the criteria in Part 365, Subpart E, Appendix A. 

The other scenario is where the recipient of the report does not accept that the identified problem exists. In this case, the Exception Report should produce some evidence as to why the problem is not valid. It may be that the Road Safety Audit Team did not have all the information available, or that the scheme design has changed since the plans used in the audit were prepared. 

Risk assessment and safety audit A risk assessment result can be used to predict whether the facility is safe or not. If it is not acceptable then additional control measures must be used to keep the facility safe. A safety audit is a process by which such safety claim is verified for consistency in results. Each case-by-case validation is done. 

Huang, Yueng-Hsiang, and Stanford A. Brubaker. Safety Auditing Applying Research Methodology to Validate a Safety Audit Tool. Des Plaines, IL Professional Safety, 2006. 

The requirements of safety culture will be a challenge for auditing validity, but if successful, it may be an useful supplement to more formal systems approaches. 

The need for validation planning is stressed in the Standard and this should be visible in the project Quality/Safety Plan which will include reference to the Functional Safety Audits. 

PrH A improves the safety, reliability and quality of the design and construction of a new or old process. P IDs must be correct as constructed operating, startup and shutdown procedures must be validated, and the operating staff must be trained before startup. Incident inr estigation recommendations, compliance audits or PrHA recommendations need resolution before startup. 

All aspects of the laboratory s work which might affect the validity of the final result should be inspected. This will include, for example, documentation, equipment, calibrations, methods, materials, record keeping, sample recording, labelling, quality control checks and log of daily checks, among many others. Some aspects, however, are outside the scope of such an audit, such as safety and security matters, which usually have separate arrangements for auditing. 

Gathering audit data can be accomplished through observations, documents, and interviews. The data obtained is used to verify and validate that the process safety management systems are implemented and functioning as designed. Data gathering can be aided oy the use of audit samples, where a representative number of items are audited to draw a conclusion, and by using self-evaluation questionnaires. 

When implementing a metrics system, it is important to ensure the process safety data is reviewed for accuracy. Inaccurate data can lead to poor decisions and focus improper priority to issues. Worse, inaccurate data may focus attention away from serious performance deficiencies. The metrics system designer needs to define the methods that will be used to validate data entered into the metrics system. There are several techniques for validating data many of the techniques have been developed through quality-based efforts and auditing methods. The following is not a detailed how to for developing a validation method, but rather introduces topics for further research. 

Audit notes are indispensable to allow QA auditors to write an accurate report after the audit. Detailed notes allow the auditor to prepare a meaningful audit report which is based on verified observations. All information collected during an audit is considered audit evidence. Information sources in an audit are, for example, document review, interviews and observation of activities. If applicable, sampling techniques may be applied, for example for SDV and verification of information in tables and listings. Audit observations are only considered audit findings if it is determined after comparison with audit criteria that these are not or insufficiently fulfilled. And finally, audit conclusions can be drawn to assess whether the audit findings impact the validity of the clinical data and the safety of the trial subjects. 

Medicinal products must be fit for their intended use, comply with the requirements of Market Authorization and not place patients at risk because of inadequate safety, quality or efficacy. These objectives are easy to state, but to achieve them requires a comprehensively designed and correctly implemented system of quality assurance. It is important that this is documented and its effectiveness monitored. Records must be kept that are open to inspection by validating bodies, and there must also be procedures for selfinspection and quality audit that allow appraisal of the quality assurance system. Management must be adequately trained and their responsibilities minutely defined. Two key posts are the Heads of Production and of Quality Control. These posts are required by the guidelines to be independent of one another. 

Regulatory Guide 1.168, Verification, Validation and Audits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants, Sept. 1997... 

AR437 1.168 Verification, validation, reviews and audits for digital computer software used in safety... 

The development of performance measures requires a collaborative effort from various levels of the organization, as well as personnel from different departments, such as safety, environmental, finance, personnel, maintenance, and engineering. The inputs the various departments can provide include appropriate benchmark levels, validity of performance measures, data needs (such as methods for obtaining data, available data, and data collection means), audit procedures, and evaluation techniques. Input from the various departments can be useful in determining available sources so that duplication does not result in wasted time and money. 

Audit instruments can be evaluated on the basis of three different types of validity when linking the audit to safety performance. These are content-related, criterion-related, and construct-related procedures for accumulating evidence of validity (Anastasi 1988, 139). Content-related validity involves the examination of the audit content to determine whether it covers a representative sample of the behaviors to be measured. Criterion-related validity procedures indicate the effectiveness of an audit in predicting performance in specified activities. The performance on the audit is checked against a criterion that is considered a direct and independent measure of what the audit was intended to measure. The third type of validity, construct validity, is the extent to which the audit is said to be measuring a theoretical construct or trait. Construct validation requires accumulation of information from a variety of sources. 

Inspections are conducted to identify new or previously missed hazards and failures in hazard controls an effective safety and health program will include regular site inspections. Inspections are planned and overseen by certified safety or health professionals. Statistically valid random audits of compliance with all elements of the safety and health program are conducted. Observations are analyzed to evaluate progress. 

---