Validation expert system

O Keefe R.M., Bald O. and Smith E.P. 1987. Validating expert system performance . IEEE Expert Winter 81 7. 

Uses raw data from field tests to compute hydraulic conductivity computed value is evaluated by the expert system for its correctness with regard to these considerations site-specific geological characteristics, validity of test procedures, accuracy of the raw data, and the computational method. System is written in Arity-Prolog on a PC. 

Dearden JC, Barratt MD, Benigni R, Bristol DW, Combes RD, Cronin MTD et al. The development and validation of expert systems for predicting toxicity. ATE A 1997 25 223-52. 

The most popular representation scheme in expert systems is the mle-based scheme. In a rule-based system the knowledge consists of a number of variables, also called attributes, to which a number of possible values are assigned. The rules are the functions that relate the different attributes with each other. A mle base consists of a number of If... Then... mles. The IF part contains the conditions that must be satisfied for the actions or conclusions in the THEN part to be valid. As an example, suppose we want to express in a mle that a compound is unstable in an alkaline solution if it contains an ester function. In a semi-formal way the mle can be written ... 

In this example this combination is straightforward. One has to be aware, however, that a medium-sized expert system easily contains several hundreds of rules. In addition, several rules can be valid at the same time. The inference engine should also have a strategy for deciding on priorities conflict resolution). 

M. Mulholland, N. Walker, J.A. van Leeuwen, L. Buydens, H. Hindriks and P.J. Schoenmakers, Expert systems for method development and validation in HPLC. Microchim. Acta, 2 (1991) 493-503. 

Evaluation of data and validation multivariate data analysis (MULTI-VAR, Wienke et al. [1991]), evaluation of interlaboratory studies (INTERLAB, Wienke et al. [1991]), ruggedness expert system (RES, van Leeuwen et al. [1991]). 

The relevant calculations are commonly handled poorly, because the equilibrium equations involved are difficult to solve manually (but not with computers). The few calculations that are actually reported in the biochemical literature use simplified methods of limited and frequently unknown validity. Large excesses of magnesium ion are frequently used in experiments, perhaps in an attempt to avoid such calculations. The relevant theory is well worked out and there are excellent reviews. The limitation appears to involve diffusion to the (mathematically) inexpert user, which is one of the motivations of building expert systems. 

A number of software packages or expert systems for ruggedness testing has been developed. RES (commercialized under the name Shaiker ) is an expert system created by Van Leeuwen et al. [4,23] and has been validated and evaluated [42,43]. It uses fractional factorial and Plackett-Burman designs and allows to test the factors at two or three levels. The interpretation criteria used here are the predefined values (see Section 3.4.8). 

J.A. Van Leeuwen, L.M.C. Buydens, B.G.M. Vandeginste, G. Kateman, P.J. Schoenmakers, M. Mulholland, RES, an expert system for the set-up and interpretation of a ruggedness test in HPLC method validation. Part 1 The ruggedness test in HPLC method validatioa Chemometrics and Intelligent Laboratory systems, 10 (1991) 337-347. 

Technology is one of many trends constantly affecting computer validation. One area requiring special attention is the introduction of expert systems into the FDA-regulated environment. The simplest form of artificial intelligence generally used in applications (such as mortgages, credit card authorization, fraud detection, e-commerce, personalization) is the rule-based system, also known as the expert system (ES). 

Early agreement should be reached between customer/owner and supplier with regard to the documentation necessary to establish the validity of system parts under supplier control. In addition, the supplier should agree to inspection(s) by the owner, or if this would compromise the supplier s commercial interests, by an independent expert, employed by the owner. 

F. Maris, R. Hindriks, J. Vink, A. Peeters, N. Vanden Driessche, and L. Massart, Validation of an expert system for the selection of initial HPLC conditions for analysis of basic drugs, J. Chromatogr., 506 211 (1990). 

It is most important that expert systems should be properly validated, just like any other test method. This is especially so at the present time, because (Q)SAR and expert systems potentially offer the most realistic and practical way to address the requirements of the recent European Union White Paper on chemicals testing (Anon., 2001 Worth and Balls, 2003). Unfortunately, this is not happening and in some situations, such as COMPACT and CASE, the systems are being used only by one research group. The validation of these approaches will require special considerations as outlined in Chapter 20 as well as Worth et al. (1998) and Worth and Cronin (2004). 

Table 19.1 A Summary of Selected Validation Studies of QSARs and Expert Systems Performed By Regulatory Agencies... 

Most pharmaceutical and healthcare companies will split the roles mentioned above into several specific jobs as appropriate to their organizational structures. For instance, the Quality and Compliance role may be split into an Operations Quality Representative and a Validation Expert. Equally, the Developer role may consist of Project Manager, Project Engineers, and Supplier Representative. Similarly, System Owners may sometimes delegate responsibilities such as specific maintenance activities to a system administrator function that may be internal or external to the company. The Quality and Compliance role, however, must not be combined with other roles otherwise its independence will be compromised. 

---