External quality assessment

EXTERNAL QUALITY ASSESSMENT AND PROFICIENCY TESTING PROGRAMS 

Initial method evaluation studies are essential to ensure that systematic errors are not present before the baseline period and the determination of the mean and the control limits. The accuracy of the method should be initially established by comparison with other analytical methods (and recovery and interference studies) and should continue to be monitored by comparison with other analytical methods. Ongoing comparison-of-methods studies are desirable to ensure that systematic errors do not slowly increase and go undetected by internal QC procedures. These ongoing comparison studies are provided by the external QA programs, which in turn form the basis for proficiency testing programs. 

The reports often include extensive data analysis, statistical summaries, and plots. The mean of all results or the mean of results from peer laboratories (those performing the test with similar methods) is taken as the target value and is used for comparison with the individual laboratory s result. Different programs do this in different ways. For example, the statistical significance of any difference between an individual laboratory s observed result and the group mean can be tested by use of the t-test. When the difference is significant, the laboratory is alerted that its results are biased compared with the results of most of the other laboratories. Another approach is to divide the difference by the overall standard deviation of the group, and then to express the difference in terms of the number of standard deviations 

Additional information about the nature of the systematic error is obtained when there are two different control materials analyzed by each laboratory. For example, the laboratory s observed mean for material A is plotted on the y axis versus its observed mean for material B on the x-axis these graphs are called Youden plots. Ideally the point for a laboratory should fall at the center of the plot. Points falling away from the center but on the 45° line suggest a proportional analytical error. Points falling away from the center but not on the 45° line suggest either an error that is constant for both materials or an error that occurs with just one material. 

The report may also include Levey-Jennings plots of the data, but because this information is not available in real time, it does not effectively serve the purposes of internal QC. Blank control charts that are set up for each analyte and each control material save the laboratory the time that is required when these charts are prepared manually. 

---

Agency. A second example of an external method of quality assessment is the voluntary participation of the laboratory in a collaborative test (Chapter 14) sponsored by a professional organization such as the Association of Official Analytical Chemists. Finally, individuals contracting with a laboratory can perform their own external quality assessment by submitting blind duplicate samples and blind standard samples to the laboratory for analysis. If the results for the quality assessment samples are unacceptable, then there is good reason to consider the results suspect for other samples provided by the laboratory. 

It must be remarked that terminology is not consistent and there are many widely used synon)ms. Quality control in this Chapter refers to practices best described as internal quality control. Quahty assessment is often referred to as external quality control, proficiency testing, interlaboratory comparisons, round robins or other terms. Internal Quality Control and External Quality Assessment are preferred because they best describe the objectives for which the RMs are being used, i.e. the immediate and active control of the results being reported from an analytical run or event, and an objective, retrospective assessment of the quality of those results. 

The primary purposes for which reference materials are employed are encompassed within the laboratory Quality Assurance Procedures. Quality assurance comprises a number of management responsibilities which focus on how the laboratory is organized, how it deals with situations, how it interacts with users, together with analytical responsibilities re internal quality control and external quality assessment (Sargent 1995 Burnett 1996). Ideally each component follows a documented protocol and written records of all activities are maintained. 

Evaluation of a new method may include comparison of results obtained by the analysis of reference materials using established procedures. The purpose is to demonstrate that the novel approach provides results that are at least as reliable as an accepted technique. This same approach may also be used to discover which, of a number of techniques, are preferred. Comparison of analytical techniques and/or methods is considered further in the discussion of external quality assessment, below. 

External quality assessment schemes involve the distribution by the organizer, to participating laboratories, of aliquots from the same sample. Participants analyze the received specimen and send the results back to the organizer who prepares a report to summarize all data. 

In 1978 the International Federation of Clinical Chemistry produced a comprehensive summary of the objectives of External Quality Assessment (IFCC 1978). The first six points of Table 4.1 summarize the IFCC objectives the seventh has developed since that time and is of considerable importance in many countries where strict control over laboratories undertaking particular types of work is required by legislation or by appropriate professional organizations. 

Measurements of individual laboratory performance provides for comparisons between laboratories. It then follows to ask why some laboratories report data that are more accurate and precise than do their peers, and a well designed external quality assessment scheme allows investigation of some of the important factors (see below). A comparison of performance between individual laboratories also helps to stimulate those who are not so successful to improve (or abandon the assay) and those who do well to continue with their expertise. Finally, changes of performance may be monitored as a consequence of some new factor, e.g. purchase of a new piece of equipment, work carried out by a different analyst, change to the methodology etc. 

Bullock DG, Smith NJ, Whitehead TP (1986) External quality assessment of assays of lead in blood. Glin Chem 32 1884-1889. 

Moller j, Rasmussen K, Christensen L (1999) External quality assessment of methylmalonic acid and total homocystein. Clin Chem 45 1536-1542. 

Morisi G, Menditto A, Patriarca M, Taylor A, eds. (1996) European external quality assessment schemes in occupational and environmental medicine. Ann 1st Sup di Sanita 32(2) i9i-3i6. 

Taylor A (1996) Reference materials and analytical standards to stimulate improved laboratory performance Experience from the external quality assessment scheme for trace elements in biological samples. Mikrochim Acta 123 251-260. 

Various External Quality Assessment Working Groups... 

Proficiency testing is a special type of laboratory performance studies defined as study of laboratory performance by means of ongoing interlaboratory test comparisons (ISO Guide 33 [1989] ISO/REMCO N 280 [1993] IUPAC [1993] Prichard et al. [2001]). Proficiency testing is an essential part of external quality assessment schemes and performance checks. 

Rhodes A, Jasani B, Balaton AJ, et al. Immunohistochemical demonstration of oestrogen and progesterone receptors correlation of standards achieved on in house tumours with that achieved on external quality assessment material in over 150 laboratories from 26 countries. J. Clin. Pathol. 2000 53 292-301. 

Merdol Ibrahim, Manager, United Kingdom National External Quality Assessment Service Immunocytochemistry In situ Hybridization, London, UK... 

Keith D. Miller, Chief Scientific Officer, UCL-Advanced Diagnostics, Cancer Institute, Director of the UK National External Quality Assessment Scheme for Immunocytochemistry In-situ Hybridisation and, Fellow of the Institute of Biomedical Science, London, UK... 

Take part in External Quality Assessment, formally through Proficiency Testing schemes and informally by taking part in intercomparison studies. 

Quality control describes the measures used to ensure the quality of individual results or a batch of results. The measures used will vary according to the particular application. It is a means of evaluating the current performance of the method being used and the general procedures used in the laboratory. There are two types of quality control, namely internal quality control and external quality assessment. These are covered in detail in Chapters 6 and 7, respectively. 

In some sectors, particularly clinical analysis, proficiency testing is referred to as External Quality Assessment (EQA). 

To monitor the laboratory performance against both its own requirements and the norms of peer laboratories there is a need for intercomparison of laboratory results. This is usually done by participating in proficiency testing (PT), i.e. an external quality assessment. In this way, the laboratory performance at national or international level are monitored and, furthermore, the reproducibility is highlighted. 

Certification, accreditation or some other information, such as external quality assessment and internal quaUty control procedures, indicating... 

NEQAS National External Quality Assessment Service... 

The belief in some quarters that patients have proprietary rights over their samples and any information generated therefrom, and that even anonymised data should not be divulged without permission may cause difficulties even for routine services with no overt research brief, putting interlaboratory audits and external quality assessment schemes at risk. 

Some important assays commonly used in biochemical genetics laboratories do not provide quantitative data (e.g. MPS-EP, qualitative urinary organic acid analysis, AA-TLC). In addition, all successful investigations depend heavily upon selection of the correct analytes to measure and the appropriate interpretation of the quantitative or qualitative results in their clinical context. These challenges suggest a requirement for external quality assessment or proficiency testing schemes that can inform participants about their performance in these areas when compared with other centres. 

Heinrich-Rumm, R., G. Lehnert, and J. Angerer. 1996. The German external quality assessment scheme in occupational and environmental medicine. Ann. 1st. Super Sanita. 22(2)12 47-251. 

---