Error recovery

To avoid accidents altogether, a complete elimination of human errors may be seen as the ultimate goal. This goal is not very practical, however, and will have severe side effects. Due to the intrinsic variability in human performance, errors will occur. Errors also provide the operators with task feedback and on-the-job learning about the systems that they operate. This experience is extremely valuable in situations when the operators have to handle unanticipated situations to avoid shut-down or accidents. This has often to be done under tight time constraints and psychological stress. 

Stop in order to avoid the development of a harmful situation. A third possibility is to activate redundant equipment in order to bring the system to the desired goal. 

Monitoring of systems feedback provides an opportunity for error detection and recovery when there is a mismatch between detected and observed outcome of an action. Human errors are not easily detected in complex systems. There may be a significant time lag between action and observed effects that makes detection difficult. The effects of human actions may be masked by actions taken by the technical control system. Biased error attribution on the part of the operator may also impede error recovery. Co-workers and supervisors are important resources for error recovery in this context. There are, however, some important preconditions for colleagues and supervisors to be able to contribute. They must co-operate closely with the erring operator to be able to observe performance and distinguish between erroneous and correct acts. There must also be a climate of trust and willingness to correct each other s behaviour. 

The operator may be able to recover errors at the execution stage. This is when the operator catches himself/herself in the act or immediately after it has been executed and corrects it before the system has been adversely affected. Different strategies are applied in accomplishing this error-recovery mechanism. The operator may notice that there is a mismatch between his/ her observation of his/her own act and his/her expectations of how it should have been performed. A special case is when the operator is able to avoid an error by observing that the situation is similar to an earlier situation in which he/she committed an error. We also here have the case when colleagues or supervisors observe erroneous acts and react to them. 

Finally, error recovery may take place at the planning stage. This is when the operator recognises mismatches between his/her own intentions and formulated plans. Self-checking and vigilance for possible errors and planning 

Feedback that confirms I am doing the right thing is important for error recovery as well as for error prevention. It is important to display the actual position of what the operator is manipulating, as well as the state of the variable he/she is worried about. 

Critical Safe Operating Parameters Never Deviate actions prevent reaching the Never Exceed limit. 

For critical, high consequence systems, simulators are useful to practice diagnosis and correction of errors and abnormal conditions in emergency conditions (CCPS, 1994a). 

Similarly, unit operating staffs can be trained to work together during a process upset using all the skills and resources available. Such training is part of nuclear submarine training ( Submarine , 1992) and 

In the Three Mile Island incident, the command signal to close the reactor relief valve was displayed, not the actual position of the valve (Kletz, 1988). Since the valve was actually open, the incident was worse than otherwise. 

---

Error recovery by the operators is only one of several layers of protection to prevent undesired consequences (see Figure 2.1). Process and equipment designs (discussed in previous chapters) that prevent undesired process excursions are inherently safer than designs that require operator intervention. Likewise, designs that enable the operators to intervene before an upset becomes serious are inherently safer than those that do not. 

An opportimity for error recovery would have been to implement a checking stage by a supervisor or independent worker, since this was a critical maintenance operation. However, this had not been done. Another aspect of the unforgiving environment was the vulnerability of the system to a single human error. The fact that the critical water jacket flow was dependent upon a single pump was a poor design that would have been detected if a hazard identification technique such as a hazard and operability study (HAZOP) had been used to assess the design. 

Because errors are frequently recoverable, it is also appropriate to define another category of errors, recovery failures. These are failures to recover a chain of events leading to a negative consequence (assuming that such a recovery was feasible) before the consequence occurs. This includes recovery from both active and latent failures. 

In the skill-based mode, recovery is usually rapid and efficient, because the individual will be aware of the expected outcome of his or her actions and will therefore get early feedback with regard to any slips that have occurred that may have prevented this outcome being achieved. This emphasizes the role of feedback as a critical aspect of error recovery. In the case of mistakes, the mistaken intention tends to be very resistant to disconfirming evidence. People tend to ignore feedback information that does not support their expectations of the situation, which is illustrated by case study 1.14. This is the basis of the commonly observed "mindset" syndrome. 

Potential errors, recovery points and error consequences are identified... 

Care should also be taken in the use of recovery factors, because these can exert a significant effect. In general, recovery paths are appropriate where there is a specific mechanism to aid error recovery, that is an alarm, a supervising check, or a routine walk round inspection. 

The application of human error analysis (HEA) techniques is to predict possible errors that may occur in a task. The next stage of error analysis is to identify error recovery possibilities implicit within the task, and to specify possible... 

Evaluation level create a system, in accordance with ergonomic criteria, that is error tolerant and supports error recovery redesign charging manifold (see Figure 7.7) using functional grouping corresponding to the actual layout of system. 

Of all the requirements that have to be fulfilled by a manufacturer, starting with responsibilities and reporting relationships, warehousing practices, service contract policies, airhandUng equipment, etc., only a few of those will be touched upon here that directly relate to the analytical laboratory. Key phrases are underlined or are in italics Acceptance Criteria, Accuracy, Baseline, Calibration, Concentration range. Control samples. Data Clean-Up, Deviation, Error propagation. Error recovery. Interference, Linearity, Noise, Numerical artifact. Precision, Recovery, Reliability, Repeatability, Reproducibility, Ruggedness, Selectivity, Specifications, System Suitability, Validation. 

Recently, the miniaturization procedures of bioanalytical studies have become an important research area with particular focus on modem concept of lab-on-a-chip technology [48], with a reduction in manufacturing costs, easy transport, minimal space and minimal maintenance requirements (and costs) in the laboratory and in the fields, even if this progress require a long design and implementation time, non-stable robotic operation, and limited error recovery abilities. 

Note (i) that an error may be judged to have multiple causes (for example the occurrence of an attempted attack on the system, and the existence of an exploitable vulnerability within the system, left there by a failing system design and implementation process), and (ii) an error does not necessarily lead to a failure (for example, error recovery might be attempted successfully and failure averted). 

Both transactions and conversations are examples of atomic actions [Lomet 1977], in that viewed from the outside they appear to perform their activity as a single indivisible action. (In practice transaction-support systems also implement other properties, such as durability , i.e., a guarantee that the results produced by completed transactions will not be lost as a result of a computer hardware fault.) And both rely on backward error recovery. 

However, systems are usually not made up just of computers -rather they will also involve other entities (e.g., devices and humans) which in many cases will not be able to simply forget some of their recent activity, and so simply go straight back to an exact earlier state when told that an error has been detected. Thus forward error recovery (the typical programming mechanism for which is exception handling), rather than backward recovery will have to be used. Each of these complications individually makes the task of error recovery more difficult, and together they make it much more challenging. This in fact is the topic that I and my colleagues have concentrated on these last few years. 

The Idealized Fault-Tolerant Component diagram (see Figure 3) is a simple, indeed simplistic, structuring technique that shows one approach to distinguishing between various sorts of system interactions, in particular identifying and classifying those that relate to system activity aimed at error recovery. 

Campbell and Randell 1986] R.H. Campbell and B. Randell, Error Recovery in... 

Xu et al. 1995] J. Xu, B. Randell, A. Romanovsky, RJ. Stroud and Z. Wu. Fault Tolerance in Concurrent Object-Oriented Software through Coordinated Error Recovery, in Proceedings 25th Int. Symp. Fault-Tolerant Computing (FTCS-25), Los Angeles, IEEE Computer Society Press, 1995. 

The standard required a site review, and a head-office level review of all reported occurrences. There was good cooperation between head-office and the plant to get the system underway. Apparent inconsistencies between the definition of the class, and the examples were fairly quickly resolved, with the help of head-office. A data-base of events was established, with an extensive sort facility. The system became very well respected and well used, and much improvement of performance was achieved through experience feedback. Near-misscs were reported freely, from which much information was gained in order to facilitate performance improvement and the development of error tolerant. and error recovery systems. 

A more experimental approach is given by Masson (1991) which deals with simulation facilities. These may be used to generate errors, recoveries, near misses and accidents on the basis of suitable scenarios. Because the conditions are under the control of the experimenter very efficient data collection is possible, but the question is always whether these data are valid and therefore generalisable to the real world. 

Identification of halt states, error rontines, and error recovery... 

---