Performance control chart

The principal tool for performance-based quality assessment is the control chart. In a control chart the results from the analysis of quality assessment samples are plotted in the order in which they are collected, providing a continuous record of the statistical state of the analytical system. Quality assessment data collected over time can be summarized by a mean value and a standard deviation. The fundamental assumption behind the use of a control chart is that quality assessment data will show only random variations around the mean value when the analytical system is in statistical control. When an analytical system moves out of statistical control, the quality assessment data is influenced by additional sources of error, increasing the standard deviation or changing the mean value. 

Using Control Charts for Quality Assurance Control charts play an important role in a performance-based program of quality assurance because they provide an easily interpreted picture of the statistical state of an analytical system. Quality assessment samples such as blanks, standards, and spike recoveries can be monitored with property control charts. A precision control chart can be used to monitor duplicate samples. 

Once a control chart is in use, new quality assessment data should be added at a rate sufficient to ensure that the system remains in statistical control. As with prescriptive approaches to quality assurance, when a quality assessment sample is found to be out of statistical control, all samples analyzed since the last successful verification of statistical control must be reanalyzed. The advantage of a performance-based approach to quality assurance is that a laboratory may use its experience, guided by control charts, to determine the frequency for collecting quality assessment samples. When the system is stable, quality assessment samples can be acquired less frequently. 

Documentation verifying job set-ups should include documentation to perform the setup and records that demonstrate that the set-up has been performed as required. This requires that you record the parameters set and the sample size and retain the control charts used which indicate performance to be within the central third of the control limits. These records should be retained as indicated in clause 4.16 of the standard. 

Figure 21.1 shows an example of a typical control chart. Here the date is plotted on the x axis so the operator can keep track of how the instrument is performing. 

DEGRAD STABILjcIs Section 1.8.4 The analysis of stability reports often suffers from the fact that the data for each batch of product is scrutinized in isolation, which then results in a see-no-evil attitude if the numerical values are within specifications. The analyst is in a good position to first compare all results gained under one calibration (usually a day s worth of work) irrespective of the products/projects affected, and then also check the performance of the calibration samples against experience, see control charts, Section 1.8.4. In this way, any analytical bias of the day will stand out. For this purpose a change in format from a Time-on-Stability to a Calendar Time depiction is of help. 

The program must require the vendors to measure a number of reference samples and/or duplicates submitted in a planned sequence. It should require prompt measurement and reporting of these data and should maintain the results in a control chart format. Prompt feedback and follow-up of any apparent data discrepancies and reconciliation of the results with control charts maintained by the vendors are required to minimize the length of uncertain performance. The quality assurance plan should include random sampling of the vendors data for their validity and conformance with quality assurance requirements. If quality assurance is properly practiced at all levels, an inspection of 5 percent of the total data output should be adequate. 

This type of verification should be distinguished from the periodic performance verification, which monitors performance, e.g., in control charts. 

Establish control charts of instrumental performance. Day-to-day variations in pump flow rate, relative response factors, absolute response to a standard, column plate counts, and standard retention times or capacity factors are all useful monitors of the performance of a system. By requiring that operators maintain control charts, troubleshooting is made much easier. The maintenance of control charts should be limited to a few minutes per day. 

The underlying calibration procedure of a newly developed analytical method has to be examined by basic validation studies to determine the reliability of the method and its efficiency in comparison with traditional methods. In order to ensure long-term stability, it is necessary to perform revalidations, which can be combined with the use of quality control charts, over meaningful time periods. 

Identify available information, including information from quality control charts, performance in proficiency testing rounds, literature and validation information on related methods and data concerning comparison with other methods. Use the available information and professional judgement to review each relevant validation issue and sign-off issues adequately addressed and documented. 

Method validation provides information concerning the method s performance capabilities and limitations, when applied under routine circumstances and when it is within statistical control, and can be used to set the QC limits. The warning and action limits are commonly set at twice and three times the within-laboratory reproducibility, respectively. When the method is used on a regular basis, periodic measurement of QC samples and the plotting of these data on QC charts is required to ensure that the method is still within statistical control. The frequency of QC checks should not normally be set at less than 5% of the sample throughput. When the method is new, it may be set much higher. Quality control charts are discussed in Chapter 6. 

This chapter deals with handling the data generated by analytical methods. The first section describes the key statistical parameters used to summarize and describe data sets. These parameters are important, as they are essential for many of the quality assurance activities described in this book. It is impossible to carry out effective method validation, evaluate measurement uncertainty, construct and interpret control charts or evaluate the data from proficiency testing schemes without some knowledge of basic statistics. This chapter also describes the use of control charts in monitoring the performance of measurements over a period of time. Finally, the concept of measurement uncertainty is introduced. The importance of evaluating uncertainty is explained and a systematic approach to evaluating uncertainty is described. 

The previous chapters of this book have discussed the many activities which laboratories undertake to help ensure the quality of the analytical results that are produced. There are many aspects of quality assurance and quality control that analysts carry out on a day-to-day basis to help them produce reliable results. Control charts are used to monitor method performance and identify when problems have arisen, and Certified Reference Materials are used to evaluate any bias in the results produced. These activities are sometimes referred to as internal quality control (IQC). In addition to all of these activities, it is extremely useful for laboratories to obtain an independent check of their performance and to be able to compare their performance with that of other laboratories carrying out similar types of analyses. This is achieved by taking part in interlaboratory studies. There are two main types of interlaboratory studies, namely proficiency testing (PT) schemes and collaborative studies (also known as collaborative trials). 

Where control charts are used, performance has been maintained within acceptable criteria. 

One method of data presentation which is in widespread use is the control chart. A number of types of chart are used but where chemical data are concerned the most common types used are Shewhart charts and cusum charts. Only these types are discussed here. The charts can also be used to monitor the performance of analytical methods in analytical laboratories. 

Note This experiment assumes that a permanent log and a quality control chart are constantly maintained for each analytical balance in use in the laboratory. Each day you use a given analytical balance and log in with your name and date. The following calibration check should be performed weekly on all balances. If, according to the log, the calibration of the balance you want to use has not been checked in over a week, perform this procedure. Review Section 3.3 for basic information concerning the analytical balance. 

FIGURE 19 Control charts of six quantitative method performance indicators. 

Several method performance indicators are tracked, monitored, and recorded, including the date of analysis, identification of equipment, identification of the analyst, number and type of samples analyzed, the system precision, the critical resolution or tailing factor, the recovery at the reporting threshold level, the recovery of a second reference weighing, the recovery for the control references (repeated reference injections for evaluation of system drift), the separation quality, blank issues, out of spec issues, carry over issues, and other nonconformances. The quantitative indicators are additionally visualized by plotting on control charts (Figure 23). 

If we don t have such an ideal control sample, but only one with a matrix different from the routine sample (e.g. a standard solution) than we have to consider also the uncertainty component arising from changes in the matrix. For this purpose we use the (repeatability) standard deviation calculated from repeated measurements of our routine samples (performed e.g. for a range control chart). When we estimate the reproducibility within laboratory we now have to combine both contributions by calculating the square root of the sum of squares. 

The analysis of quality control (QC) samples with the construction of quality control charts has been suggested as another way of performing PQ. Control samples with known amounts are interdispersed among actual samples at intervals determined by the total number of samples, the stability of the system, and the precision specified. The advantage of this procedure is that the system performance is measured more or less continuously under conditions that are very close to the actual application. 

An established external quality control (QC) scheme is not currently available. Pooled disease control CSF retained from other analyses is used. Aliquots of the pooled CSF are made and stored at -70°C. This CSF is analysed on five separate occasions and the mean and standard deviation determined. For an analytical/diagnostic run to proceed, analysis of QC material must provide concentration values that are within two standard deviations (plus and minus) of the calculated mean for that particular QC. Construction of Levy-Jennings type control charts provide historical information of overall performance and highlight potential deterioration in the performance of the system. 

Specifications How good do the numbers have to be Write specifications Pick methods to meet specifications Consider sampling, precision, accuracy, selectivity, sensitivity, detection limit, robustness, rate of false results Employ blanks, fortification, calibration checks, quality control samples, and control charts to monitor performance Write and follow standard operating procedures... 

Documentation is critical for assessment. Standard protocols provide directions for what must be documented and how the documentation is to be done, including how to record information in notebooks. For labs that rely on manuals of standard practices, it is imperative that tasks done to comply with the manuals be monitored and recorded. Control charts (Box 5-1) can be used to monitor performance on blanks, calibration checks, and spiked samples to see if results are stable over time or to compare the work of different employees. Control charts can also monitor sensitivity or selectivity, especially if a laboratory encounters a wide variety of matrixes. 

Once the mechanics of retrospective validation are mastered, a decision is required as to how data analysis will be handled. The illustrated calculations may be performed manually with the help of a programmable calculator and the control charts may be hand-drawn, but computer systems are now available that can shorten the task. If the computer route is chosen, commercially available software should be considered. There are many reasonably priced programs that are more than up to the task [17]. 

---