Formal methods software testing

The assurance of critical systems was discussed for approaches that employ formal methods and testing for applications as well as underlying software layers. In addition, a number of results that target specific domains such as military systems, safety-critical product lines and peer-to-peer control and data acquisition systems were presented. These papers provided a good introduction into the specific requirements of these domains and presented specific solutions for their domain. Furthermore, the interplay of architecture modeling and existing domain-specific safety standards was discussed in the context of automotive systems. 

The proposed methodology integrates CBD and MBT as depicted in Figure 2 the system architecture is enriched with a contract refinement, which is proved correct with formal methods the software components are implemented with a state machine and with a concrete program the state machine is proved to satisfy the component contracts with model checking and is used to generate a test suite for the program. 

It is concluded that it would be reasonable to claim that available systems and application software design techniques were suitable for safety related applications. The limiting factors are perhaps the lack of an integrated development environment extending to formal methods and the poor support for testing. The provisos are less significant when PLCs are used in simple plications. 

Hamon G, de Moura L, Rushby J (2004) Generating efficient test sets with a model checker. In 2nd International Conference on Software Engineering and Formal Methods (SEFM), Beijing, China. IEEE Computer Society... 

The safety analysis of safety-critical software provides the safety requirements which need to be tested. Safety verification shall be performed to verify a correct incorporation of software safety requirements [24]. Verification must show that hazards have been ehminated or controlled to an acceptable level of risk. Figure 1 shows the proposed method of software safety verification based on STPA at the system level. The method includes three main step>s (1) safety analysis of software at the system level (2) formalization of safety requirements and constraints and (3) verification and testing at the code level. 

Unit testing is a software verification and validation method in which a programmer tests if individual units of source code are fit for use. A unit is the smallest testable part of an application. In procedural programming, a unit may be an individual function or procedure. Unit tests are typically written and run by software developers to ensure that code meets its design and behaves as intended (or specified). Its implementation can vary from being very manual (pencil and paper) to being formalized as part of build automation. 

Other forms of evidence are usually provided, often generically called process (or indirect) evidence that methods believed to reduce the risk of defects were applied in development and verification and validation and that the faults of concern are indeed likely to be absent (e.g., certain properties have been formally proved, stringent inspection or testing methods failed to detect faults, any fault revealed and considered important was fixed). Such process precautions are required by most standards for safety-critical (and security-critical) software. 

---