Audits and Performance Assessments

Three general types of feedback are commonly used audits and performance assessments reporting systems and anomaly, incident, and accident investigation. 

Once again, audits and performance assessments should start from the safety constraints and design assumptions and rationale. The goal should be to determine whether the safety constraints are being enforced in the operation of the system and whether the assumptions underlying the safety design and rationale are still true. Audits and performance assessments provide a chance to detect whether the behavior of the system and the system components still satisfies the safety constraints and whether the way the controllers think the system is working—as reflected in their process models—is accurate. 

The entire safety control structure must be audited, not just the lower-level processes. Auditing the upper levels of the organization will require buy-in and commitment from management and an independent group at a high enough level to control audits as well as explicit rules for conducting them. 

Audits are often less effective than they might be. When auditing is performed through contracts with independent companies, there may be subtle pressures on the audit team to be unduly positive or less than thorough in order to maintain their customer base. In addition, behavior or conditions may be changed in anticipation of an audit and then revert back to their normal state immediately afterward. 

Overcoming these limitations requires changes in organizational culture and in the use of the audit results. Safety controllers (managers) must feel personal 

---

Setting up a safety information system for a single project or product may be easier. The effort starts in the development process and then is passed on for use in operations. The information accumulated during the safety-driven design process provides the baseline for operations, as described in chapter 12. For example, the identification of critical items in the hazard analysis can be used as input to the maintenance process for prioritization. Another example is the use of the assumptions underlying the hazard analysis to guide the audit and performance assessment process. But first the information needs to be recorded and easily located and used by operations personnel. 

Some measures of PSM and ESH performance are easy to identify, establish and track. These include accident rates, effluent tonnages and composition and number of days lost to illness. Almost all of these traditional performance measures are end-of-pipe that is, they measure the output of the management system and allow corrective action only after a failure has occurred. The ideal measurement system identifies potential problems ahead of actual failure allowing corrective action to be taken. This requires using techniques such as audits and hazard assessments. 

Management reviews are conducted based on the review inputs and any decisions and actions taken as a result of the management review are the review output . The information taken as input to the management reviews could be the results from audits and self-assessment of the laboratory, feedback from customers and results from benchmarking activities, the laboratory activity performance, the status of corrective and preventive action... 

There is an industry trend toward dedicating specific personnel to the function of internal audits and quality assessments. This approach allows the assessor or auditor to focus on the compliance of the company and its vendors and contractors as well as maintain an independent role allowing for impartial assessment of all areas, including the QA unit. Those companies whose assessors and auditors also perform QA responsibilities might seriously consider the regular use of an independent assessment group, such as corporate auditors or an independent consultant. A fresh pair of eyes, coupled with extensive exposure to multiple worldwide operations, makes using external auditors, preferably with some former FDA experience a tremendous asset to any company. 

The ISO has also produced a set of quality standards specifically for environmental management. This is the ISO 14000 series. The areas addressed by ISO 14000 are Environmental Management Systems, Environmental Performance Evaluations, Environmental Auditing, Life Cycle Assessment, and Environmental Labeling. 

Cl Assessment and response actions C2 Reports to management 4.6.3 Systems and performance audits... 

The performance of the laboratory QC system is assessed through internal and external systems audits. Laboratory QA section performs internal audits and identifies the weaknesses of the quality system or the deviations from approved internal procedures. The state, the EPA, a client or any other body that oversees the quality of laboratory work conducts external audits as part of initial and ongoing laboratory... 

From a perspective of the data user, a combination of audits and PE samples may be a cost-effective approach to laboratory quality assessment. Single-blind and double-blind PE samples are typically used on long-term, large sample volume projects. Because of the additional costs, they are not sent to the laboratory with every shipment of field samples. PE samples for the project contaminants of concern analyzed at the start of project activities will enable the data user to establish the laboratory s performance. Depending on the project duration and on the volume and importance of the collected samples, additional PE samples may be analyzed on a quarterly or semiannual basis. 

Closely related to preventive action is the process audit. Such an audit is performed as part of a laboratory assessment for each test on the proposed scope of accreditation. Performance of a test procedure is observed relative not only to the requirements of the ISO 17025 standard (or ISO Guide 25) but also with respect to performing the ASTM or ISO test method properly. As a minimum, the laboratory must have properly calibrated equipment, the current test method, and trained testing personnel. In addition, a representative number of test demonstrations will reveal whether good laboratory practices are observed in sample preparation, using the correct test parameters and calculating the results. When operators and technicians accept responsibility through such demonstrations, they tend to commit to the quality process. Real improvement occurs with involvement at all levels. 

Prepare design specification including any hardware configuration. Verify installation and performance of hardware components. Assess (audit) hardware development capability maturity of supplier. 

QA audits — to be performed by specified (independent) personnel as per the agreed schedule. These audits will evaluate the use and performance of the system in its operational and maintenance environment and will also address system users their competency assessment, training, accreditation, proficiency, and supervision. 

The second step is to honestly and accurately assess one s current state. The performance of process assessments has proven to be an effective tool for doing this. This method differs significantly from the traditional compliance audit. It focuses on efficiency as well as effectiveness in achieving standards. It looks beyond observations or symptoms for root causes. It focuses on the quality of the processes and their outputs. It borrows from the Malcolm Baldrige award the evaluation of ... 

ISO includes environmental management systems, environmental auditing, environmental labeling, environmental performance evaluation, and lifecycle assessment. It is a process rather than a performance-based system. 

Systems audits in this late phase in clinical trials aim at assessing related procedures to ensure that capable procedures exist for managing and cleaning clinical trial data, for conducting statistical analyses and for preparing the final study report which represents properly the data collected and reported in the clinical trial. Such systems audits are performed across functional boundaries. Such systems audit can be combined with a database audit and/or an audit of the final study report. 

The Malaysian Department of Environment recommended that the different project phases of exploration, development, operation, and rehabilitation are evaluated due to their environmental, biological, and socioeconomic impact. Once the possible environmental impacts are assessed, the project initiator must identify and indicate the possible mitigation measures to be taken with a purpose of controlling the environmental pollution. Compliance auditing is performed during the assessment to check whether the project complies with environmental protection standards. 

Deciding whether a company has carried out an audit will in turn depend to some degree on the company s own definition. A number of different types of audit have been identified. An environmental review is an initial analysis of the environmental issues, impact and performance related to activities at a site and should be seen as the first step in the audit programme. Following the review, a system audit will assess the effectiveness of the company s environmental management system. Other audits will cover particular sites, acquisition or divestment and the production impacts of making a specific product. 

To measure environmental performance to conduct regular environmental audits and assessments of compliance with company requirements, legal requirements, and these principles and periodically to provide appropriate information to the directors, shareholders, employees, the authorities and the public. ... 

This can be more complicated, since it will rely on the assessment of a company s potential ability, separate from the actual context of historical performance. Ideally any new supplier should be certificated to a national or international quality standard (e.g. ISO 9000), with an already audited and proven quality system. (During the 1990s many UK packaging companies have been independently certificated to the Pharmaceutical Supplier Codes of Practice, which are linked to ISO 9002.)... 

Compliance with the requirements of ISO/IEC 17025 2005 is essential if a laboratory is to become accredited for particular analytical procedures. External audit by a recognized national or international body is a prerequisite for this process. For example, in the United Kingdom, accreditation is awarded by the UK Accreditation Service (UKAS) and in Canada by the Standards Council of Canada. The benefits of accreditation will be discussed in more detail below, but accreditation for a particular function by a recognized body means that the laboratory has been assessed against internationally recognized standards to demonstrate their competence, impartiality, and performance capability. 

Waste and energy audits are performed to identify existing problems. In the next four subsections, new approaches are presented that focus on prevention through Ufe cycle design, fife cycle assessment, production planning models with environmental considerations, and environmental management systems. 

Clause 8 contains five subclauses, the first of which, 8.1, states that the oigemization shall define, plan, and implement the measuring and monitoring activities needed to ensure conformity emd improvement. Clause 8.2 on measurement and monitoring requires the organization to measure customer satisfaction, conduct internal audits, and measure and monitor all processes and the product. The internal audits must be performed at reguleu intervals and must assess whether the quality memage-ment system is effective and conforms to the standard. 

---