System verification and validation

Lopez, O, Automated Process Control Systems Verification and Validation, Pharmaceutical Technology, September 1997. 

Lopez, O. (1997). Automated process control systems verification and validation. Pharm. Technol. 21(9), 100-108. 

This chapter provides an overview of expert-system verification and validation (V V) techniques. Several methods are presented. First, many of the conventional software V V techniques such as requirements analysis and unit testing can be applied to expert-system development. Second, an expert-system developer can use automated tools to test rule consistency and structure. A more viable alternative, however, is for the developer to create his own set of consistency and completeness tests. Finally, a developer should rely on qualitative judgment to determine the validity of a knowledge base. This judgment could include expert opinion as well as specialized tests designed to determine knowledge-base certification. The chapter suggests that methods should be combined into an optimal mix in order to best undertake V V. 

Richardson, Keith and Wong, Carla, "Knowledge Based Systems Verification and Validation" Proceedings of the Second Annual Artificial Intelligence Research Forum, April 1987 NASA/Ames. 

Evidence from system Verification and Validation (V V) may be based on, inter alia, analysis and/or testing. 

Instrumentation shall be provided and tests shall be specified to be performed to ensure that the means of shutdown are always in the state stipulated for the given condition of the reactor. For computer based digital reactivity control systems, verification and validation of software shall be performed. 

GLP regulations require QA personnel to inspect/audit each study conducted, but the extent to which QA personnel are involved in software development and the val-idation/verification process varies from company to company. In some companies, there is little or no QA involvement in these processes, whereas in others QA personnel are involved. QA personnel can provide assistance in the area of vendor audits for purchased software or can conduct inspections of in-house software development to ensure that internal procedures are being followed. QA personnel, who conduct in-process inspections and review the resulting data and validation report for accuracy, could provide inspection support during the validation and verification process. During system development and validation, properly trained QA personnel can provide the regulatory advice needed to ensure that the system will meet government standards. QA personnel become more familiar with the system(s) that will be used when they are involved early in the validation process. 

The final step of method development is validation of the HPLC method. Optimisation of chromatographic selectivity [110], performance verification testing of HPLC equipment [591], validation of computerised LC systems [592] and validation of analysis results using HPLC-PDA [34] were reported. The feasibility of automated validation of HPLC methods has been demonstrated [593]. Interlaboratory transfer of HPLC methods has been described [594]. 

Customer support Quality management/standards Software/system development methodology Testing methods/verification and validation Technical personnel... 

Compnters are widely used in analytical laboratories for instrument control, data evaluation, and data management, and are subject to all validation and verification activities. Verification and validation activities to assess compnter systems used in analytical laboratories cover all life-cycle phases, from nser Reqnirements Specifications, design, development, and manufacturing to installation and operation. In this case stndy, users of computer systems will find guidelines on ... 

Verification and validation methodologies must be determined at this stage. Verification techniques are the methods used to determine that the expert system has been built correctly. 

STUNDER Verification and Validation of Environmental Expert Systems 41... 

Expert Systems are being designed or considered for both on-line process control and off-line decision support in the handling of hazardous materials. There are possibilities for both positive and negative impacts on chemical industry fire/explosion loss potentials. We will review some guidelines and tools available for the verification and validation (V V) of expert systems. 

MILLER Verification and Validation of Decision Support Expert Systems 127... 

The purpose here is not to present a tutorial on software reliability or quality assurance, but to pose some of the questions relating to the verification and validation of knowledge-based systems, to outline some of the current research in this area, and review a few pieces of recent literature. 

Because of cost and time constraints, we have not had the opportunity to actually work with and evaluate many of the tools mentioned. Consequently our conclusions are based in large measure on the comments received from those working in the fields of AI/Expert Systems and Software Verification and Validation. Also, my conclusions are those of a potential user rather than of a specialist in the field. 

The principle conclusion is that the development of "expert" systems and the necessary audit tools for verification and validation of those systems is still in infancy. There are a number of interesting concepts under development which bear promise but which will probably see their first application within critical space and defense applications. Tools useful to those of us interested in more generalized risk and reliability analyses are probably at least two to three years away. 

The acceptance of expert systems in process control applications and in risk assessment, risk management and loss prevention engineering will depend to a large extent on the degree to which verification and validation questions are addressed early on in the design of the systems. 

---