Proficiency testing results

Vani K, Sompuram SR, Fitzgibbons P, et al. National HER2 proficiency test results using standardized quantitative controls. Arch. Pathol. Lab. Med. 2008 132 211-216. 

AMC (2000) How to combine proficiency test results with your own uncertainty estimate - the zeta score. AMC Technical Brief No. 2 (Nov 2000). Analytical Methods Committee, Royal Society of Chemistry, London, 2 pp. 

Radiation monitoring laboratories seeking to achieve optimum proficiency test results with an accreditation standard must use calibration methods that duplicate or at least closely approximate the irradiation protocols described in the accreditation standard. This requirement is particularly important for calibrations using photons with energies below 200 keV where irradiation conditions must recreate the scattered radiation that contributes significantly to the response of the monitoring device. 

Laboratories enrolling in the CTQ program (to satisfy the last of the above criteria) must remit, with the enrollment application, an initial fee of approximately 100 per analyte. (Note that this fee is only an estimate, and is subject to revision without notice.) Laboratories should indicate on the application that they agree to have proficiency test results sent by the CTQ directly to the physicians designated by participating laboratories. 

Proficiency test results and related materials received while participating in the CTQ interlaboratory program over the past 2 years results also should be tabulated to provide a serial record of relative error (derived per Section 3.3.3 below). 

The following criteria are designed solely for the purpose of determining performance of laboratories in proficiency testing. Results from these tests will be used by the Director-General for designating laboratories. No such criteria shall be considered as constituting any interpretation of, or precedent for, related provisions of the Convention. 

It must be remarked that terminology is not consistent and there are many widely used synon)ms. Quality control in this Chapter refers to practices best described as internal quality control. Quahty assessment is often referred to as external quality control, proficiency testing, interlaboratory comparisons, round robins or other terms. Internal Quality Control and External Quality Assessment are preferred because they best describe the objectives for which the RMs are being used, i.e. the immediate and active control of the results being reported from an analytical run or event, and an objective, retrospective assessment of the quality of those results. 

Assuming an analytical error of 2 % which is based on the analysis of the results of a number of proficiency tests and collaborative trials (Daas and Miller 1998), a maximum uncertainty of 0.24% would imply a 0.1% probability of rejecting a good result. Given that the uncertainty of a content to be assigned is below a predetermined value, the results of the collaborative trial are acceptable otherwise it is recommended to repeat the trial in whole or in part. 

Table 14 can be regarded as providing a reasonable overall picture, even if the results cannot applied to any particular case. However, if the underlying principle is accepted, it becomes clear that improvements in a single stage, for example the reduction of instrument variation, has a negligible beneficial effect (if this variation was not outside the normal range ). Even if the contribution of repeatability is re-duced to zero, the cumulative uncertainty is reduced by 10% only, i.e. from 2.2 to y(0.0)2 (0.8)2 (1.0)2 + (1.5)2 = 2.0. This statistical view of errors should help to avoid some unnecessary efforts to improve, e.g., calibration. Additionally, this broad view on all sources of error may help to detect the most important ones. Consequently, without participation in proficiency tests, any method validation will remain incomplete. 

False-positive results with bDNA have been observed with proficiency testing specimens for HTV-1 in the College of American Pathologists HIV-1 viral load survey and HCV in the viral quality control program administered by the Netherlands Red Cross. The reason for the false-positive results with these proficiency testing specimens is not known but may be sample matrix effects. The extent to which this problem occurs with clinical samples has not been determined. However, both the HIV-1 and HCV bDNA assays were designed to have a false-positive rate of 5%. 

The principle of proficiency testing schemes consists in analyzing one or more samples sent to the laboratories by an external body. The analytical results returned to the organizer are evaluated in comparison to the assigned value(s) of the sample(s). 

Kahlert, B. (2005b). Third ISTA proficiency test on GMO testing on Zea mays (MON819+T25) summary of results , Seed Testing International, 129, 10-12. www.seedtest.org/upload/cms/user/STI1291.pdf, Accessed July 18, 2005. 

Organizations whose measurement results are with 25% of the known exposure level will be considered to have met the proficiency test requirements. 

The equation representing this curve was introduced in Section 4.4 (equation (4.4)). However, a more contemporary model based on results from Proficiency Testing schemes has shown that the relationship is best represented if three equations are used to cover from high to low concentrations, as shown in... 

The previous chapters of this book have discussed the many activities which laboratories undertake to help ensure the quality of the analytical results that are produced. There are many aspects of quality assurance and quality control that analysts carry out on a day-to-day basis to help them produce reliable results. Control charts are used to monitor method performance and identify when problems have arisen, and Certified Reference Materials are used to evaluate any bias in the results produced. These activities are sometimes referred to as internal quality control (IQC). In addition to all of these activities, it is extremely useful for laboratories to obtain an independent check of their performance and to be able to compare their performance with that of other laboratories carrying out similar types of analyses. This is achieved by taking part in interlaboratory studies. There are two main types of interlaboratory studies, namely proficiency testing (PT) schemes and collaborative studies (also known as collaborative trials). 

There is no experimentally established optimum frequency for the distribution of samples. The minimum frequency is about four rounds per year. Tests that are less frequent than this are probably ineffective in reinforcing the need for maintaining quality standards or for following up marginally poor performance. A frequency of one round per month for any particular type of analysis is the maximum that is likely to be effective. Postal circulation of samples and results would usually impose a minimum of two weeks for a round to be completed and it is possible that over-frequent rounds have the effect of discouraging some laboratories from conducting their own routine quality control. The cost of proficiency testing schemes in terms of analysts time, cost of materials and interruptions to other work has also to be considered. 

If the analytical method used by participants in the proficiency testing round has been validated by means of a formal collaborative trial, then the repeatability and reproducibility data from the trial can be used. The repeatability standard deviation gives an estimate of the expected variation in replicate results obtained in a single laboratory over a short period of time (with each result produced by the same analyst). The reproducibility standard deviation gives an estimate of the expected variation in replicate results obtained in different laboratories (see Chapter 4, Section 4.3.3 for further explanation of these terms). 

En numbers are used when the assigned value has been produced by a reference laboratory, which has provided an estimate of the expanded uncertainty. This scoring method also requires a valid estimate of the expanded uncertainty for each participant s result. A score of En < 1 is considered satisfactory. The acceptability criterion is different from that used for z-, z - or zeta-scores as En numbers are calculated using expanded uncertainties. However, the En number is equal to zeta/2 if a coverage factor of 2 is used to calculate the expanded uncertainties (see Chapter 6, Section 6.3.6). En numbers are not normally used by proficiency testing scheme providers but are often used in calibration studies. 

It is always useful to consider the performance in a proficiency testing round in a wider context. One of the main factors to consider is the performance of all of the participants in the round. If the majority of the results are satisfactory, but yours is not, this is likely to indicate a problem in your laboratory. However, it is worth remembering that your laboratory may have got the correct result and the other participants are in error In addition, if many other participants also have unsatisfactory results, there is still a problem, but it is less likely to be in your laboratory. 

This chapter has considered two of the types of interlaboratory comparison exercise in which your laboratory may participate. It is important to remember that proficiency testing schemes and collaborative studies have different aims. The former is a test of the performance of the laboratory, whereas the latter is used to evaluate the performance of a particular analytical method. Laboratories should participate in proficiency testing schemes (where an appropriate scheme is available) as this provides an independent check of the laboratory s performance. This chapter has described the key features of proficiency testing schemes and explained how the results from participation in a scheme should be interpreted. 

The nonconforming work that will require evidence of implementation of corrective and preventive action within a given timescale will include matters, such as, no corrective action taken when the results from a round of a Proficiency Testing scheme indicated the laboratory s result was an outlier, or the competency records of staff do not indicate they are competent to do the accredited work. Listings of nonconformities can be found in a publication produced by the International Laboratory Accreditation Cooperation (ILAC) [9]. 

---