Proficiency Testing performance measures

This chapter deals with handling the data generated by analytical methods. The first section describes the key statistical parameters used to summarize and describe data sets. These parameters are important, as they are essential for many of the quality assurance activities described in this book. It is impossible to carry out effective method validation, evaluate measurement uncertainty, construct and interpret control charts or evaluate the data from proficiency testing schemes without some knowledge of basic statistics. This chapter also describes the use of control charts in monitoring the performance of measurements over a period of time. Finally, the concept of measurement uncertainty is introduced. The importance of evaluating uncertainty is explained and a systematic approach to evaluating uncertainty is described. 

Participation in proficiency testing schemes is an ongoing activity. It is therefore useful to monitor performance over a period of time and to look for trends. Performance over time can be demonstrated statistically by using measures such as RSZ and SSZ (see Section 7.3.4) but as mentioned previously, these can be misleading. It is better to monitor performance scores by plotting them on a... 

Participation in proficiency tests is not a prerequisite or an absolute substitute for IQC measures or vice versa. However, participance in proficiency tests is meaningless without a well-developed IQC system. IQC underlies participance in PT schemes, while IQC and participance in PT schemes are both important substitutes of AQA (Figure 6). It is shown that laboratories with the strongest QC procedures score significantly better in PT schemes [8,50]. Participance in PT can to a certain extent improve the laboratory s performance however unsatisfactory performance in schemes (up to 30% of all participants) has been reported. This means that there... 

Some important assays commonly used in biochemical genetics laboratories do not provide quantitative data (e.g. MPS-EP, qualitative urinary organic acid analysis, AA-TLC). In addition, all successful investigations depend heavily upon selection of the correct analytes to measure and the appropriate interpretation of the quantitative or qualitative results in their clinical context. These challenges suggest a requirement for external quality assessment or proficiency testing schemes that can inform participants about their performance in these areas when compared with other centres. 

The need to carry out both QA and QC in order to achieve the expected quality of analytical results immediately generates the requirement for clearly defined performance criteria. These criteria enable comparability to be achieved via the traceability of analytical results to national or international standards along an unbroken chain of comparisons. Validation is the central task in the development of any analytical method whose capabilities in specific applications can be assessed with the aid of measurement uncertainty. Finally, proficiency testing (PT) serves to demonstrate comparability in terms of the scatter of the results.3... 

Figure 14. Overall performance of participants in proficiency tests 1 to 10, measured in the absolute number of spiking chemicals...

Laboratory performance study An interlaboratory study that consists of one or more measurements by a group of laboratories on one or more homogeneous, stable, test samples by the method selected or routinely used by each laboratory. The reported results are compared with the consensus value. Note, this is very similar to a Proficiency Testing scheme (see below). 

THE PERFORMANCE OF UK AND OVERSEAS LABORATORIES IN PROFICIENCY TESTS FOR THE MEASUREMENT OF Am... 

E623 Hassemer, D., Woodlief, G., Lanphear, B. and Laessig, R. (1990). An assessment of the quality of tests performed in physician office laboratories as measured through proficiency testing using proposed CLIA-88 requirements. Clin. Chem. 56, 1005, Abstr. 252. 

Ion-selective electrodes (ISEs) represent the current primary methodology in the quantification of S-Li [11-13], Moreover, ISE modules are parts of large and fully automated clinical chemistry analysers. In practice, the validation parameters are most often chosen in terms of judging the acceptability of the new measurement system for daily use. For this reason, the first approach was to study whether the detected imprecision fulfilled the desired analytical quality specifications. Secondly, proficiency testing (PT) results from past samples were of great value in predicting future bias. The identity of the three ISE methods was evaluated using patient samples. The analytical performance was checked after 6 months routine use. Without any exception, method validations always mean an extra economical burden. Therefore, the validation parameters chosen and processed have to be considered carefully. 

Quality measurements have several elements. Quality assurance plans and quality control procedures are an essential beginning. In addition, it is necessary to have qualified scientists whose training needs to be documented and updated on a continuous basis. Quality measurements also require proper use of reference materials where available, and laboratories must repeatedly test their ability to perform through taking part in proficiency testing schemes. The provision of another essential element in quality measurements, namely validated methods, is the primary contribution from the work of AOAC. 

Although AFP was traditionally measured by RIA, newer methods use lEMA or chemiluminescent immunoassay (CIA) because of their lower detection lunits, better precision, speed, avoidance of radioactivity, and ease of automation. Most laboratories in the United States measure AFP by use of automated systems available from Abbott Laboratories (Abbott Park, 111,), Bayer (Medfield, Mass.), Beckman Coulter (Fullerton, Calif.), Diagnostic Products (Los Angeles), and Perkin Elmer (Norwalk, Conn.). Judging from the results of proficiency testing sponsored by the Foundation for Blood Research and College of American Pathologists (Survey FP), all of these systems perform satisfactorily. ... 

The variation of sensitivity between different sensors was also checked. Calibration curves with five different sensors were performed. A Relative Standard Deviation of 13, 13 and 42% of calibration slopes (sensitivity) were obtained for Cu, Pb and Cd respectively. These variations should have limited consequence on bias and precision when the standard addition method is used. However, for Cd, variations in the limit of quantification between two electrodes could be expected. Finally, the accuracy of the method was evaluated by the measurement of a SWIFT reference material used during the 2nd SWIFT-WFD Proficiency Testing exercise (Table 4.2.2). The reference value was chosen as the consensus value of the selected data population obtained after excluding the outliers. The performances of the device were estimated according to the Z-score (Z) calculation. Based on this score, results obtained with the SPEs/PalmSens method were consistent with those obtained by all methods for Pb and Cu ( Z < 2) while the result was less satisfactory for Cd (2 < Z < 3). 

In general, the results of these proficiency testing programs indicate a considerable improvement in the performance of PbB determination in most laboratories (for review see Parsons, 1992). According to the practice in clinical chemistry, quality assurance including internal and external quality control should, therefore, be an integrated part of PbB measurements. 

Quality issues related to analytical measurements have been described in full detail in several books [1-3] and highlighted several principles, e.g. validation of methods, quality assurance protocols incorporating the use of CRMs, independent assessment of method performance by participation in proficiency testing schemes, and accreditation. Method performance studies also represent a very important aspect to evaluate the state-of-the-art of a particular type of analysis at the development stage e.g. for testing the applicability of a standard method) or to improve the quality of measurements, e.g. prior to the certification of reference materials. 

There are other routinely practiced procedures in terms of intralaboratory and interlaboratory quality control, such as the control chart, round-robin interlaboratory testing, and various proficiency tests offered by different agencies, such as the PATs (Proficiency Analytical Testing) by AHA in the US and WASPs (Workplace Analysis Scheme for Proficiency) by the Health and Safety Laboratory in Great Britain. In normal circumstances, a combination of these quality control measures should be used in performing workplace air analysis. 

---