Reliability, replicate measurements

Random deviations (errors) of repeated measurements manifest themselves as a distribution of the results around the mean of the sample where the variation is randomly distributed to higher and lower values. The expected mean of all the deviations within a measuring series is zero. Random deviations characterize the reliability of measurements and therefore their precision. They are estimated from the results of replicates. If relevant, it is distinguished in repeatability and reproducibility (see Sect. 7.1)... 

For tests designed to detect the presence or absence of an analyte, the threshold concentration that can be detected can be determined from replicate measurements over a range of concentrations. These data can be used to establish at what concentration a cut-off point can be drawn between reliable detection and non-detection. At each concentration level, it may be necessary to measure approximately ten replicates. The cut-off point depends on the number of false negative results that can be tolerated. It can be seen from Table 4.7 that for the given example the positive identification of the analyte is not reliable below 100 xg g-1. 

Measure the concentration of analyte in several identical aliquots (portions). The purpose of replicate measurements (repeated measurements) is to assess the variability (uncertainty) in the analysis and to guard against a gross error in the analysis of a single aliquot. The uncertainty of a measurement is as important as the measurement itself, because it tells us how reliable the measurement is. If necessary, use different analytical methods on similar samples to make sure that all methods give the same result and that the choice of analytical method is not biasing the result. You may also wish to construct and analyze several different bulk samples to see what variations arise from your sampling procedure. 

Analysis of the shape of error surfaces. To conclude this section, we consider a more quantitative approach to error estimation. The first step is to estimate the accuracy of the individual data points this can either be done by analysis of the variability of replicate measurements, or from the variation of the fitted result. From that, one can assess the shape of the error surface in the region of the minimum. The procedure is straightforward the square root of the error, defined as the SSD, is taken as a measure of the quality of the fit. A maximum allowed error is defined which depends on the reliability of the individual points, for example, 30% more than with the best fit, if the points are scattered by about 30%. Then each variable (not the SSD as before) is minimised and also maximised. A further condition is imposed that the sum of errors squared (SSD) should not increase by more than the fraction defined above. This method allows good estimates to be made of the different accuracy of the component variables, and also enables accuracy to be estimated reliably even in complex analyses. Finally, it reveals whether parameters are correlated. This is an important matter since it happens often, and in some extreme cases where parameters are tightly correlated it leads to situations where individual constants are effectively not defined at all, merely their products or quotients. Correlations can also occur between global and local parameters. 

Most chemical analyses are performed on replicate samples whose masses or volumes have been determined by careful measurements with an analytical balance or with a precise volumetric device. Replication improves the quality of the results and provides a measure of reliability. Quantitative measurements on replicates are usually averaged, and veirious statistical tests are performed on the results to establish reliability. 

Because one analysis gives no information about the variability of results, chemists usually carry two to five portions (replicates) of a sample through an entire analytical procedure. Individual results from a set of measurements are seldom the same (see Figure 5-1), so we usually consider the best estimate to be the central value for the set. We justify the extra effort required to analyze several samples in two ways. First, the central value of a set should be more reliable than any of the individual results. Usually, the mean or the median is used as the central value for a set of replicate measurements. Second, an analysis of the variation in data allows us to estimate the uncertainty associated with the central result. 

Automated methods are more reliable and much more precise than the average manual method dependence on the technique of the individual technologist is eliminated The relative precision, or repeatability, measured by the consistency of the results of repeated analyses performed on the same sample, ranges between 1% and 5% on automated analyzers. The accuracy of an assay, defined as the closeness of the result or of the mean of replicate measurements to the true or expected value (4), is also of importance in clinical medicine. 

Always replicate the treatments. Without replication, measurements of variability may not be reliable. 

A TV video camera in conjunction with a TV monitor and video cassette recorder (Figure 1) has an enormous advantage over the conventional 35 mm photography. Firstly, since an entire experiment can be stored on a TV cassette and thus becomes part of a permanent Phycomyces library, we can rerun any given experiment, many lasting several hours, at any future date. Secondly, the reliability of measurements is enhanced since replicates may be taken and the taped experiment may be stopped or rewound to check results. 

In this speaker s opinion the precision attained in measuring real samples [ed. as opposed to measurement of standards only or standards dissolved in substitute background matrices] is the only reliable basis for decisions on detection. In large measurement programs, the use of duplicate-sample control charts is the most feasible way to establish the precision parameters needed and to defend limits of detection. Otherwise, a sufficient number of replicate measurements must be made on the samples tested for this purpose. Without documented demonstration of precision, the data are meaningless... 

Replicate measurement of the analyte in its natural or endogenous matrix is necessary to confirm the reliability of the method. The lack of available analyte-free natural matrices for preparing calibration curves has led to the common use of substituted matrices (e.g., PBS-BSA with 0.05% Tween 20) that are optimized for the method reagents and commercial quality control (QC) materials, but these artificial matrices are not representative of the biological sample for which the method will be used. For this reason, precision testing using biological samples is an important component of validation. 

When summarizing the results of a set of n replicate measurements JC (/ =1,2,..., ) of a single experimental measured quantity x, it is necessary to deduce both some typical value representing all the measured values and also an indication of the spread of values obtained (i.e., how reliable is the quoted typical value). The most commonly used quantity describing the typical value is the mean (arithmetical average) x defined as ... 

Replicate measurements improve reliability. If s = 2.0%, 3 measurements give a 95% confidence interval of 5.0% ... 

C. A reliable assay of ATP (adenosine triphosphate) in a certain type of cell gives a value of lll.o iJimol/lOO mL, with a standard deviation of 2.g in four replicate measurements. You have developed a new assay, which gave the following values in replicate analyses 117, 119, 111, 115, 120 jjimol/lOO mL. 

This is a better way, because by doing several replicate measurements, very reliable values can be obtained for both E and the slope, and calculation eliminates errors in reading off the concentration for a given potential. 

Because of these complications, regardless of the very high within-run precision attainable via TIMS or ICP-MS, the true precision of the runs (as opposed to the internal or within run precision provided by the TIMS or ICP-MS operating software) can only be reliably established by replicate analyses of natural samples. One useful approach is to establish the external variance of a measurement technique by subtracting the internal variance from the total (= run-to-run) variance from replicate analyses, e.g.. 

Once the reliability of a replicate set of measurements has been established the mean of the set may be computed as a measure of the true mean. Unless an infinite number of measurements is made this true mean will always remain unknown. However, the t-factor may be used to calculate a confidence interval about the experimental mean, within which there is a known (90%) confidence of finding the true mean. The limits of this confidence interval are given by ... 

It is evident that the mean of n results is 4n times more reliable than any one of the individual results. Therefore, there exists a diminishing return from accumulating more and more replicate meaurements. In other words, the mean of 9 results is 3 times as reliable as 1 result in measuring central tendency (i.e., the value about which the individual results tend to cluster) the mean of 16 results is 4 times as reliable etc. 

Vertzoni et al. (30) recently clarified the applicability of the similarity factor, the difference factor, and the Rescigno index in the comparison of cumulative data sets. Although all these indices should be used with caution (because inclusion of too many data points in the plateau region will lead to the outcome that the profiles are more similar and because the cutoff time per percentage dissolved is empirically chosen and not based on theory), all can be useful for comparing two cumulative data sets. When the measurement error is low, i.e., the data have low variability, mean profiles can be used and any one of these indices could be used. Selection depends on the nature of the difference one wishes to estimate and the existence of a reference data set. When data are more variable, index evaluation must be done on a confidence interval basis and selection of the appropriate index, depends on the number of the replications per data set in addition to the type of difference one wishes to estimate. When a large number of replications per data set are available (e.g., 12), construction of nonparametric or bootstrap confidence intervals of the similarity factor appears to be the most reliable of the three methods, provided that the plateau level is 100. With a restricted number of replications per data set (e.g., three), any of the three indices can be used, provided either non-parametric or bootstrap confidence intervals are determined (30). 

Diastase number was measured using a buffered solution of starch according to AOAC [10], HMF was determined colorimetrically after dilution with distilled water and addition of p-toluidine solution. Absorbance of the solution was determined at 550 run using a spectrophotometer (HP 8450A UV/VIS). Three replicate analyzes were performed from each sample for data reliability. 

In view of the conflict between the reliability and the cost of adding more hardware, it is sensible to attempt to use the dissimilar measured values together to cross check each other, rather than replicating each hardware individually. This is the concept of analytical i.e. functional) redundancy which uses redundant analytical (or functional) relationships between various measured variables of the monitored process e.g., inputs/outputs, out-puts/outputs and inputs/inputs). Figure 3 illustrates the hardware and analytical redundancy concepts. 

---