Safe software

Human expertise in complex systems is constantly changing and a New Paradigm for software safety assurance is considered. As the development of Safety Critical Systems is guided by standards, the standards are to be updated3. In what follows we present a general view of how the development of safe software systems is currently practiced and show two specific solutions aimed at efficient support of the efforts. Responsibility of organizations, processes and culture, not just efforts of specific members of the organizations, is emphasized. 

Reviewing every single line of code will produce safe software. That would be a Herculean task, however, and hyper-expensive. We should spend the majority of our efforts reviewing lines of code in software-critical systems. 

The hardware expenses for both channels, the oonparator and the interface amount to three to four Europe cards. The power svpply is included in this figure. Some figures about the development of safe software systemis can be found in the references /7/. According to these date, the expenses per comitend are between 30 and 700 corresponding to safety relevant requiremients. 

Safe software is software that executes within a system context and environment with an acceptable level of potential mishap risk. This means the software will not cause any system hazards or prevent system design safety mechanisms from performing correctly, or the likelihood of causing these conditions is within acceptable bounds. 

Figure 2.80 shows the overall approach to SwS. It should be noted that these SwS tasks do not provide a quantitative estimate of the potential mishap risk associated with the software. What this approach does provide is a level of confidence that the software can be considered as being safe. Software-related hazards can be accepted for risk based on the conclusions drawn from the safety case, where the safety case is built upon the results of both the functional and developmental completion evidence. 

The author has particular experience of the use of robots in medicine [1]. This activity incorporates, in essence, most of the problem areas that can be met in complex medical systems. Not only are data gathered on the basis of whidi actions are taken, but these actions are performed by a robotic system which involves prime movers (often of considerable power) being used next to people. These robots involve a computer controlled mechanism, dependent on safe software operating on... 

It should be noted that all these strategies for producing safe software are vulnerable to an error in the original specification, i.e. when there is a mismatch between the software requirement and the real world need. This unfortunately also limits the potential for accelerated testing of software against the requirement to reduce the dangerous failure rate bound as the tests will omit the same key features of real-world behaviour. 

Safety transformations transform unsafe original software into safe software that, in contrast to the unsafe version, detects if its execution was incorrect due to errors in the infrastructure used. This error detection facilitates error tolerance. Several safety transformations were developed recently ... 

Objective. So far, safety transformations are just claimed and believed to be correct, since the applied techniques of replication or arithmetic encoding are well known. However, the implementation is complicated in many places. Hence, there is no guarantee that the resulting safe software produces the same results as the unsafe original software in an error-free execution or that the implementation ensures the best error detection theoretically possible. Thus, before applying these transformations in safety-critical applications, we need to check the following properties ... 

In order to operate a process facility in a safe and efficient manner, it is essential to be able to control the process at a desired state or sequence of states. This goal is usually achieved by implementing control strategies on a broad array of hardware and software. The state of a process is characterized by specific values for a relevant set of variables, eg, temperatures, flows, pressures, compositions, etc. Both external and internal conditions, classified as uncontrollable or controllable, affect the state. Controllable conditions may be further classified as controlled, manipulated, or not controlled. Excellent overviews of the basic concepts of process control are available (1 6). 

Although the traditional point of reference for safety interlock systems is a hard-wired implementation, a programmed implementation is an alternative. The potential for latent defects in software implementation is a definite concern. Another concern is that solid-state components are not guaranteed to fail to the safe state. The former is addressed by extensive testing the latter is addressed by manufacturer-supplied and/or user-supplied diagnostics that are routinely executed by the processor within the safety interlock system. Although issues must be addressed in programmable implementations, the hard-wired implementations are not perfect either. 

The view is therefore growing that we should try to design plants so that they are safe even if there is a fault in the software. This can be done by adding on independent safety systems, such as relief valves and hardwired trips and interlocks, or by designing inherently safer plants that remove the hazards instead of controlling them (see Chapter 21). 

Safe handling of chemicals demands a combination of hardware and software such as operating procedures, staff selection and training. Systems of work will generally include ... 

An application of transport and compartment-type models to hazard analysis is described in the paper by Honeycutt and Ballantine (19). The compound CGA-72662 running off from agricultural areas into surface waters was modeled in order to set safe application procedures consistent with the protection of aquatic environments. Patterson, et al (2 0) have adapted the UTM model to a software package that is generally applicable to fate assessments of toxic substances in air, water, soil and biota. Their work, now in working draft form, is being used by Dr. William Wood and Dr. Joan Lefler in the Office of Toxic Substances of the U.S. Environmental Protection Agency. 

Finally the algorithm decides what to produce next. If product changes are flexible, this is the product with the shortest coverage, measured by the mean or by the safe coverage. If a product cycle has to be realized then the next product is the next one in the cycle. However, the software recommends skipping this product if the inventory covers the demand until this product appears in the next cycle. 

The SIS is normally designed to fail-safe on loss of power and takes action only when the process demands that it do so. These demands often occur when safe operating limits are exceeded due to BPCS failures. Therefore, the SIS is designed and managed to be independent of the BPCS in terms of its hardware and software and its user interfaces, such as operator, maintenance, and engineering interfaces. 

The lack of adequate software tools to develop and safely implement chemometric models in a process analytical environment, and... 

---