Treating the computer as a black box

The most common types of errors are probably those that occur because operators treat the computer as a black box, that is, something that will do what we want it to do without the need to understand what goes on inside it. There is no fault in the hardware or software, but nevertheless the system does not perform in the way that the designer or oper- 

Chapter 2 stressed the need to consider the results of plant modifications before they are made and to prevent unauthorized ones. This applies to computers as well as traditional plant. No change should be made to hardware or software unless authorized by a professionally competent person who has carried out a systematic survey of possible consequences. It is easier to change a software control system than a traditional one and therefore harder to control the changes, but it is just as important to do so. Section 20.5 describes an unauthorized change to hardware that could have had serious results. 

Sections 9.1.6 (c) and 9.2.1 (h) drew attention to the hazards of using old equipment. Similar remarks apply to old software except that, unfortunately, it never wears out. I do not know of any incidents in the process industries due to this cause, but it was responsible for the loss of the European space rocket Ariane 5. A function that no longer served any purpose was left in for commonality reasons, and the decision to do so was not analyzed or fully understood [14]. In another incident, cancer patients received excessive doses of radiation because operators were able to enter data faster than the computer could process them. This had always been the case, but originally a hardwired interlock had prevented 

---