Proficiency testing data evaluation

This chapter deals with handling the data generated by analytical methods. The first section describes the key statistical parameters used to summarize and describe data sets. These parameters are important, as they are essential for many of the quality assurance activities described in this book. It is impossible to carry out effective method validation, evaluate measurement uncertainty, construct and interpret control charts or evaluate the data from proficiency testing schemes without some knowledge of basic statistics. This chapter also describes the use of control charts in monitoring the performance of measurements over a period of time. Finally, the concept of measurement uncertainty is introduced. The importance of evaluating uncertainty is explained and a systematic approach to evaluating uncertainty is described. 

All proficiency testing schemes should have a statistical protocol which states clearly how the data will be processed and how laboratory performance will be evaluated. This protocol should also describe how the assigned value for any parameter in a test sample is estimated. This is an important consideration, as the performance of individual laboratories is gauged by comparison with the assigned value. 

Although the providers of proficiency testing schemes should have a quality management system in place, on occasions problems can arise which will affect the quality of the data evaluation being carried out. These can include transcription errors during data entry, mistakes in the report, software problems and inappropriate criteria for evaluation being used. Such problems should be remedied by the provider once the problem has been identified. 

It was recommended that the target values for each proficiency test should be released soon after the deadline for data submission for the world-wide test. It was also requested that other field intercomparison exercises (gamma does rate evaluation by in-situ gamma-spectrometry, sampling in aquatic systems, etc.) should be organized. 

The data were assessed by the ISO 5725-2 protocol [3] implemented in the software package Prolab98 (Dr Uhlig, Quo Data, Dresden, Germany), which is routinely used by the German Federal Environmental Agency for evaluation of laboratory proficiency tests. Outliers were rejected by use of Grubbs test (P=l%) and Cochran s test (P=l%). 

The variation of sensitivity between different sensors was also checked. Calibration curves with five different sensors were performed. A Relative Standard Deviation of 13, 13 and 42% of calibration slopes (sensitivity) were obtained for Cu, Pb and Cd respectively. These variations should have limited consequence on bias and precision when the standard addition method is used. However, for Cd, variations in the limit of quantification between two electrodes could be expected. Finally, the accuracy of the method was evaluated by the measurement of a SWIFT reference material used during the 2nd SWIFT-WFD Proficiency Testing exercise (Table 4.2.2). The reference value was chosen as the consensus value of the selected data population obtained after excluding the outliers. The performances of the device were estimated according to the Z-score (Z) calculation. Based on this score, results obtained with the SPEs/PalmSens method were consistent with those obtained by all methods for Pb and Cu ( Z < 2) while the result was less satisfactory for Cd (2 < Z < 3). 

The Army conducted an evaluation of the proficiency of its medical first-responders (medics). Their readiness to treat NBC casualties was lowest of all the skills evaluated (16 percent passed a cognitive test with a grade of 70 percent or better). Hands-on skills were not tested in this evaluation. There are no equivalent data for the Navy. 

---