Bias and precision

Bland (2004) provides a review and some examples of cluster randomised trials while Campbell, Donner and Klar (2007) give a comprehensive review of the methodology. 

CHOI BASIC IDEAS IN CLINICAL TRIAL DESIGN 

ICH E9 (1998) Note for Guidance on Statistical Prindples for Clinical Trials  

What particular features in the design of a trial help to eliminate bias  

---

Overall uncertainty of a measuring procedure or of an instrument The quantity used to characterize the uncertainty of results given by an apparatus or a measuring procedure, expressed on a relative basis by a combination of bias and precision, according to a formula. 

Tellinghuisen, J. and Wilkerson, C. W. (1993). Bias and precision in the estimation of exponential decay parameters from sparse data. Anal. Chem. 65, 1240-6. 

Accuracy is often used to describe the overall doubt about a measurement result. It is made up of contributions from both bias and precision. There are a number of definitions in the Standards dealing with quality of measurements [3-5]. They are only different in the detail. The definition of accuracy in ISO 5725-1 1994, is The closeness of agreement between a test result and the accepted reference value . This means it is only appropriate to use this term when discussing a single result. The term accuracy , when applied to a set of observed values, describes the consequence of a combination of random variations and a common systematic error or bias component. It is preferable to express the quality of a result as its uncertainty, which is an estimate of the range of values within which, with a specified degree of confidence, the true value is estimated to lie. For example, the concentration of cadmium in river water is quoted as 83.2 2.2 nmol l-1 this indicates the interval bracketing the best estimate of the true value. Measurement uncertainty is discussed in detail in Chapter 6. 

Here the concept of statistical control is not applicable. It is assumed, however, that the materials in the run are of a single type. Carry out duplicate analysis on all of the test materials. Carry out spiking or recovery tests or use a formulated control material, with an appropriate number of insertions (see above), and with different concentrations of analyte if appropriate. Carry out blank determinations. As no control limits are available, compare the bias and precision with fitness-for-purpose limits or other established criteria. 

Interlaboratoiy tests are a test for accuracy. Inaccuracy arises from systematic and random effects that are related to bias and precision respectively. As a result of a PT the laboratory should be able to determine, whether imprecision or bias is the reason for its inaccuracy. 

Figure 8.10. Bias and precision in chemical analysis. Contributions to bias when averaged by many repeated measurements become part of the variance of the measurement at the level (run, laboratory, method) being studied. (Adapted from O Donnell and Hibbert 2005.)...

The limit of quantitation is the minimum concentration of an analyte in a specific matrix that can be determined above the method detection limit and within specified bias and precision limits under routine operating conditions (EPA, 1998a). The limit of quantitation is often referred to as the practical quantitation limit (PQL). The concept of PQL is discussed in Chapter 4.5.1. 

CDC and the CRMLN established a traceability scheme (Fig. 1). The CRMLN uses this approach in a certification program for manufacturers. In this program, NCCLS protocol EP9-A is used as a basis for comparison using fresh serum samples [31]. The manufacturer collects a minimum of 40 specimens and analyzes them in duplicate in five separate runs. The specimens are then shipped to a CRMLN laboratory for analytical and statistical analysis. When the NCEP performance criteria for bias and precision are met, the manufacturer is issued a Certificate of Traceability for the... 

The two parameters most often used to assess measurement quality objectives are bias and precision. Bias is defined as a systematic deviation (error) in data. Precision is defined as random variation in data. One objective of any sampling quality... 

Reference materials give an indication of bias and precision for your method. They can be purchased from a number of vendors (e.g., NIST, Environment Canada) and should be analyzed on a monthly basis. When reference materials are not available, a reference standard can be used. A reference standard is a highly purified compound that is well characterized. 

Accuracy is often used instead of bias and trueness. It can be seen from Figure 2 that it involves bias and precision. 

Accuracy is a term that is used loosely in daily conversation. It has particular meanings in analytical measurement. For example, under the current ISO definition, accuracy is a property of a result and comprises bias and precision. 

E357 Koch, T.R., Mehta, U., Lee, H., Aziz, K., Temel, S., Donlon, J.A. and Sherwin, R. (1987). Bias and precision of cholesterol analysis by physician s office analyzers. Clin. Chem. 33, 2262-2267. 

However, one important point should be kept in mind when statistically testing the model fit The higher the precision of a method, the higher the probability to detect a statistically significant deviation from the assumed calibration model [1, 6, 9]. Therefore, the practical relevance of the deviation from the assumed model should also be taken into account. If the accuracy data (bias and precision) are within the required acceptance limits or an alternative calibration model is not applicable, slight deviations from the assumed model may be neglected [6, 9],... 

The variation of sensitivity between different sensors was also checked. Calibration curves with five different sensors were performed. A Relative Standard Deviation of 13, 13 and 42% of calibration slopes (sensitivity) were obtained for Cu, Pb and Cd respectively. These variations should have limited consequence on bias and precision when the standard addition method is used. However, for Cd, variations in the limit of quantification between two electrodes could be expected. Finally, the accuracy of the method was evaluated by the measurement of a SWIFT reference material used during the 2nd SWIFT-WFD Proficiency Testing exercise (Table 4.2.2). The reference value was chosen as the consensus value of the selected data population obtained after excluding the outliers. The performances of the device were estimated according to the Z-score (Z) calculation. Based on this score, results obtained with the SPEs/PalmSens method were consistent with those obtained by all methods for Pb and Cu ( Z < 2) while the result was less satisfactory for Cd (2 < Z < 3). 

Box plots were generated for each scenario comparing the performance of the two imputation methods. Area under the curve extrapolated to infinity (AUCo-inf), % area extrapolated, and terminal half-life (Lambda Z HL) were plotted and compared across different methods. Also the bias and precision associated with the estimation of each of these parameters were compared for the two methods. 

FIGURE 9.2 Bias and precision of the conditional multiple imputation and fractional single multiple imputation (LLOQ/n) methods under the fourth scenario (i.e., assuming 45% interindividual variability and 25% residual variability), presented as percent relative prediction errors (%RPE) (+SD) for the following parameters (A) AUQ-inf, (B) % AUC extrapolated, and (C) terminal half-life (Lambda Z HL). 

Numerical methods used to fit experimental data should, ideally, give parameter estimates that are unbiased with reliable estimates of precision. Therefore, determining the reliability of parameter estimates from simulated PPK studies is an absolute necessity since it may affect study outcome. Not only should bias and precision associated with parameter estimation be determined but also the confidence with which these parameters are estimated should be examined. Confidence interval estimates are a function of bias, standard error of parameter estimates, and the distribution of parameter estimates. Use of an informative design can have a significant impact on increasing precision. Paying attention to these measures of parameter estimation efficiency is critical to a simulation study outcome (6, 7). 

In the authors opinion, the first definition is the most concise and self-explanatory. All analysis should be fit for the intended purpose. A laboratory should be able to determine the concentration of the specified parameter with sufficient accuracy (lack of bias) and precision (repeatability) so that the result can be satisfactorily applied in any relevant risk assessment or remediation verification. The laboratory should also be able to detect the specified parameter at concentrations where there is no significant risk from that parameter on the site in question to the receptor(s) of concern. 

Another internal technique used to validate models, one that is quite commonly seen, is the bootstrap and its various modifications, which has been discussed elsewhere in this book. The nonparametric bootstrap, the most common approach, is to generate a series of data sets of size equal to the original data set by resampling with replacement from the observed data set. The final model is fit to each data set and the distribution of the parameter estimates examined for bias and precision. The parametric bootstrap fixes the parameter estimates under the final model and simulates a series of data sets of size equal to the original data set. The final model is fit to each data set and validation approach per se as it only provides information on how well model parameters were estimated. 

Acceptable bias and precision in the structural model parameters were observed with two samples per subject across any two time points. However, better precision and accuracy in estimating clearance was obtained when the second sample was collected at later times. Volume of distribution was not as affected by the choice of sample times. Across all time points, the variance components were often significantly underestimated with the range of estimates being quite large. 

When the number of samples per individual was increased to three, regardless of where the middle point was collected in time, the structural model parameters remained unbiased but the bias in the variance components was removed. When the number of subjects was increased to 100 and then 150, the bias and precision in the structural model parameters remained unchanged, but improved the estimation of the variance components. Hence, under these conditions, neither more data per subject nor more subjects improved the estimates of the fixed effects in the model. What were affected were the variance components. Both more data within a subject and more subjects resulted in better variance component estimation. 

---