Proficiency-testing evaluation

Over the past decade, a major trend has been the development of the use of proficiency testing (PT) or evaluation materials (Fox 2000). PT materials are a type of reference material, which aid in assessment of analytical laboratory measurement quality. There will be an increased use of such materials as part of laboratory accreditation programs and other new quality assurance efforts, including internal audits. At the same time, a number of providers have used PT schemes to produce a form of RM intended to meet the ever-growing need for RMs required for routine QC use (Jenks 1995,1997). 

The principle of proficiency testing schemes consists in analyzing one or more samples sent to the laboratories by an external body. The analytical results returned to the organizer are evaluated in comparison to the assigned value(s) of the sample(s). 

Proficiency testing can be assessed in different ways. One of the most used evaluation system is that of so-called z-scores which are defined as follows (Lawn et al. [1993] Thompson and Wood [1993]) ... 

This chapter deals with handling the data generated by analytical methods. The first section describes the key statistical parameters used to summarize and describe data sets. These parameters are important, as they are essential for many of the quality assurance activities described in this book. It is impossible to carry out effective method validation, evaluate measurement uncertainty, construct and interpret control charts or evaluate the data from proficiency testing schemes without some knowledge of basic statistics. This chapter also describes the use of control charts in monitoring the performance of measurements over a period of time. Finally, the concept of measurement uncertainty is introduced. The importance of evaluating uncertainty is explained and a systematic approach to evaluating uncertainty is described. 

The previous chapters of this book have discussed the many activities which laboratories undertake to help ensure the quality of the analytical results that are produced. There are many aspects of quality assurance and quality control that analysts carry out on a day-to-day basis to help them produce reliable results. Control charts are used to monitor method performance and identify when problems have arisen, and Certified Reference Materials are used to evaluate any bias in the results produced. These activities are sometimes referred to as internal quality control (IQC). In addition to all of these activities, it is extremely useful for laboratories to obtain an independent check of their performance and to be able to compare their performance with that of other laboratories carrying out similar types of analyses. This is achieved by taking part in interlaboratory studies. There are two main types of interlaboratory studies, namely proficiency testing (PT) schemes and collaborative studies (also known as collaborative trials). 

There are two main types of proficiency testing scheme. First, there are those set up to assess the competence of a group of laboratories to undertake a very specific analysis, e.g. lead in blood or the number of asbestos fibres in air collected on membrane filters. Secondly, there are those schemes used to evaluate the performance of laboratories across a certain sector for a particular type of analysis. Because of the wide range of possible analyte/matrix combinations it is not practicable to assess the performance of laboratories when analysing all the possible sample types. Instead, a representative cross-section of analyses is chosen (e.g. determination of different pesticide residues in a range of foodstuffs or the determination of trace levels of metals in water samples). 

All proficiency testing schemes should have a statistical protocol which states clearly how the data will be processed and how laboratory performance will be evaluated. This protocol should also describe how the assigned value for any parameter in a test sample is estimated. This is an important consideration, as the performance of individual laboratories is gauged by comparison with the assigned value. 

Although the providers of proficiency testing schemes should have a quality management system in place, on occasions problems can arise which will affect the quality of the data evaluation being carried out. These can include transcription errors during data entry, mistakes in the report, software problems and inappropriate criteria for evaluation being used. Such problems should be remedied by the provider once the problem has been identified. 

This chapter has considered two of the types of interlaboratory comparison exercise in which your laboratory may participate. It is important to remember that proficiency testing schemes and collaborative studies have different aims. The former is a test of the performance of the laboratory, whereas the latter is used to evaluate the performance of a particular analytical method. Laboratories should participate in proficiency testing schemes (where an appropriate scheme is available) as this provides an independent check of the laboratory s performance. This chapter has described the key features of proficiency testing schemes and explained how the results from participation in a scheme should be interpreted. 

First of all it takes time to evaluate a PT. So it is quite a long time before the laboratory knows the result. The interlaboratory test is retrospective. It is therefore not possible to base quality management only on proficiency tests. 

In ISO 13528 details on possible statistical methods for the evaluation of proficiency tests are given. 

Usually different levels and different samples in proficiency tests are evaluated separately. This can lead to injustice if by chance good laboratories are grouped together in one concentration level and less good laboratories in another. This results in variations in the tolerance limits as shown in this slide. A calculation procedure for a variance function was introduced in the German standard DIN 38402-45 and also pubhshed in ISO/TS 20612. 

The NVLAP is comprised of a series of laboratory accreditation programs (LAPs), depending on what is requested and what is needed. Each LAP includes specific calibration and/or test standards and related methods and protocols assembled to satisfy the unique needs for accreditation in a field of testing or calibration. NVLAP accredits public and private laboratories based on evaluation of their technical qualifications and competence to carry out specific calibrations or tests. All are based on the ISO/IEC Guide 25 discussed in Section 2. The process involves an application and the payment of fees. This is followed by an on-site assessment. If deficiencies are found, these must be resolved to the satisfaction of the assessment team. The laboratory then participates in proficiency testing (Section 5.7) and a technical evaluation. 

Participance in Proficiency Testing Schemes Proficiency testing (PT) is the periodic assessment of the competency or the analytical performance of individual participating laboratories [23]. An independent coordinator distributes individual test portions of a typical uniform test material. The participating laboratories analyze the materials by their method of choice and return the results to the coordinator. Test results obtained by different laboratories are subsequently compared with each other and the performance of each participant evaluated based on a single competency score [64,107]. International harmonized protocols exist for the organization of PT schemes [59,60,64,69,79]. 

Evaluations of various soil and sediment samplers have been reported [56, 57]. The sediment shovel proved highly practical, but was limited because small particles tend to be lost when the shovel is lifted [56]. A cryogenic sediment sampler was less convenient to use, but allowed the collection of almost undisturbed samples. Houba described a different device for the automatic subsampling of soil, sediment and plant material for proficiency testing [57]. In another study, Thoms showed that freeze-sampling collects representative sediment samples, whereas grab-sampling introduces a bias into the textural composition of the 120 mesh fraction, due to washout and elutriation of the finer fractions [58]. 

Most situations where proficiency evaluation is applied are rather narrowly defined as to scope. For example, clinical laboratories may be asked to demonstrate that they can determine certain constituents occurring in human serum with uncertainties not to exceed specific limits. Accrediting bodies will require successful participation in periodic proficiency tests conducted by a reference laboratory which they recognize. 

More recently, proficiency evaluation programs have produced similar traceability linkages, but with a novel twist. The proficiency test material (transfer standard) is sent without attachment of results. The laboratory under test receives both an independent confirmation of measurement ability and, indirectly, a traceability linkage when the test is concluded and the assigned value of the measurand is disclosed. It is unlikely that proficiency evaluation will supplant traditional forms of providing traceability links. However, the concept does provide metrologists with an additional tool. 

This paper has described new approaches to providing quality assurance within the operation of proficiency evaluation programs. The approaches are used in a new NIST program to monitor providers of proficiency testing in an environmental field. A part of that monitoring is conducted by means of indirect proficiency evaluation of the providers. The new program demonstrates that both direct and indirect proficiency evaluation can provide traceability links, in chemical metrology, to measurements carried out at NIST. 

Target values obtained by reference procedures such as isotope dilution mass spectrometry are in use since 1988 serving as a basis for the evaluation of participants results in the German proficiency testing system. It may be of some interest to know how the introduction of the concept of traceability improved the performance of diagnostic procedures since 1988. 

In Germany, the Federal Medical Association prescribes the use of IFCC enzyme reference procedures in clinical laboratory practice. Accordingly, the evaluation of participants results in the proficiency testing system is based on IFCC reference procedure values. 

Fig. 4 Elements in the proficiency testing organised to evaluate matrix bias and method validation...

There is one thing that we can definitely say is not a source of reference values and that is interlaboratory comparison. Interlaboratory comparison, proficiency testing and the like are very useful tools, depending on the protocols and intended purposes. They can be used to detect and diagnose problems, evaluate competencies and pro-... 

The initial procedures for the conduct of official OPCW proficiency tests have been developed by the Special Task Force on Analytical Issues in the period 1994 to 1996, and subsequently recommended for adoption through Working Group B to the Conference of States Parties the respective procedures were adopted by the First Conference of States Parties in May 1997 (5). The evaluation criteria have been further developed since, through consultation with participants in the proficiency-testing scheme. These consultations were often initiated by practical problems posed during individual proficiency tests. The numerous amended documents have recently been compiled into three comprehensive quality system documents that describe the process of the OPCW proficiency test, which has been accredited by the Dutch accreditation body in 2001 ... 

Work Instruction for the Evaluation of Results of OPCW Proficiency Tests , QDOC/LAB/WI/ PRO003 (8). 

---