Paired-comparison method

The paired comparisons method (NUREG/CR-3688) is a structured expert judgment method in which human errors are compared in pairs. By combining the judgments of the group of experts, the errors arc arranged in order of likelihood of occurrence, of the human errors considered, they can be used as "anchor points list. Documentation requirements are given in Table 4.5-8. 

The paired comparison method involves comparing all possible pairs of jobs under study. A simple way to do paired comparison is to set up a matrix, as shown in Figure 2. 

Shin, W. S., and Alien, D. B. (1994), An Interactive Paired Comparison Method for Bicriterion Integer Mathematical Programming, Naval Research Logistics, Vol. 41, pp. 423-434. 

In the directional paired comparison method the experimenter wants to determine whether the two samples differ in a specified dimension, such as saltiness, hardness, crispness samples are simultaneously presented to the assessors in two possible combinations (AB, BA), randomized across panelists, with an equal number of panelists receiving either sample A or sample B first. The assessors have to choose the sample within the pair that presents the higher intensity of a specified characteristic (one-sided test). 

Like the difference paired comparison method, the A-non-A test has four possible serving sequences (AA, BB, AB, BA) that are randomized across panelists, with each sequence appearing an equal number of times. As in paired comparison, the null hypothesis is of no distinction between the samples and the alternative hypothesis is Pa >0.5. 

Paired-comparison method is a two-product test both products are coded, and the subject s task is to indicate which one has more of a specific characteristic, such as sweetness. There are two orders of presentation - AB and BA. As long as one could specify the attribute and the subjects recognized it, the method is quite useful, because of its simplicity in terms of instructions and implementation. The complexity of technology in today s environment has reduced the popularity of this method. 

The classic approach to image quality assessment involves the presentation to a group of test subjects visual test material for evaluation and rating. The test material may include side-by-side display comparisons, or a variety of perception-threshold presentations. One common visual comparison technique is called the pair-comparison method. A number of observers are asked to view a specified number of images at two or more distances. At each distance, the subjects are asked to rank the order of the images in terms of overall quality, clearness, and personal preference. 

The most rigorous method to evaluate the contribution of belts to fatality reduction was developed by Evans (1986). With this method, known as the "double pair comparison" method, Evans first identified all fatal crashes involving cars with multiple occupants (where the driver was not necessarily the one who was killed). He then compared the fatality likelihood for drivers with and without belts relative to the likelihood of fatality of the other occupant with and without belts. The beauty of this approach is that there is no need for exposure measure, and hence no need to be concerned about exposure bias. Using the U.S. national Fatal Analysis Reporting System which documents every fatal crash in the U.S., Evans estimated that the... 

The results of such multiple paired comparison tests are usually analyzed with Friedman s rank sum test [4] or with more sophisticated methods, e.g. the one using the Bradley-Terry model [5]. A good introduction to the theory and applications of paired comparison tests is David [6]. Since Friedman s rank sum test is based on less restrictive, ordering assumptions it is a robust alternative to two-way analysis of variance which rests upon the normality assumption. For each panellist (and presentation) the three products are scored, i.e. a product gets a score 1,2 or 3, when it is preferred twice, once or not at all, respectively. The rank scores are summed for each product i. One then tests the hypothesis that this result could be obtained under the null hypothesis that there is no difference between the three products and that the ranks were assigned randomly. Friedman s test statistic for this reads... 

R. A. Bradley, Science, statistics, and paired comparisons. Biometrics, 32 (1976) 213-232. H.A. David, The Method of Paired Comparisons. Charles Griffin, London, 1963. 

Methods for the instrumental measurement of whiteness are well established but visual comparison remains important, even in well-equipped laboratories. Some degree of quantification is achieved by the method of paired comparisons, in which a panel of observers is presented with pairs of FBA-treated samples and asked to decide, without undue delay, which is the brighter. The total of positive scores can be used as a measure of whiteness and the results presented graphically as shown in Figure 11.7. Although time-... 

The failure to find non additive genetic effects in the large twin registry sample remains a puzzle and will be resolved only when we are able to test this sample with instruments comparable to those used in the other studies. As an aside it is worth mentioning that the two instruments used to derive the factor scales make use of quite different methods of measurement (paired comparisons vs. a Like, Indifferent, Dislike format) and utilize different content. Each instrument, however, yielded the same results. 

The fact that the EP wants to replace old TEC methods with more selective, efficient, and sensitive separation methods provides the chance for the introduction of more CE methods. The continuous development of analytical methods is reflected in the national and international pharmacopoeias. This might be demonstrated for atropine sulfate. Whereas the Deutsches Arzneibuch, 7th Edition (DAB 7) only limits the tropic acid by extraction and titration with NaOH and phenolphthalein indication, the 4th edition of the EP looked for foreign alkaloids and decomposition products by means of TEC with a potassium iodobismuthate for detection. By intensity comparison of the obtained spots, it was possible to limit these impurities to 0.5%. The EP 5 utilizes an ion-pair HPLC method that is able to limit most of the impurities to less than 0.2%. To make the method more robust, an HPLC method using a polar embedded was applied, which might be the next step for the EP. However, recently the same authors have reported on a MEEKC method being as robust and precise as the latter HPLC method (see Eigure 6) but far more sensitive and, therefore, a future perspective for the EP. 

The method of paired comparisons has the rater comparing each employee against every other employee, one by one, on either specific aspects of performance or overall performance. During each comparison, the rater places a checkmark by the name of the employee who was considered to have performed better. The employee with the most checkmarks is considered to be the best performer. While more precise and perhaps seemingly less ambiguous to employees, paired comparisons still may fail to generate substantive feedback and are more burdensome to implement that other relative systems. 

The use of 8 as a probe in the solvatochromic comparison method (paired with 7 or 5 for non-HBA solvents) was re-examined by Laurence, Nicolet and Helbert18. They used a much wider variety of EPD/HBA bases (68 in all) than those used in the construction of the ftT scale3,17 (25 solvents) (equation 8). The coefficients for the expression... 

Therefore, we make resistivity measurements by first generating calibration curves on known-reslstlvlty samples of the same type, orientation, and surface finish as the test specimens to be profiled. Calibration curves are generated for a particular pair of probes at a particular time, using known-reslstlvlty samples of the highest quality available. This calibration procedure Is a particularly noteworthy characteristic of the spreading resistance technique. It means that spreading resistance Is a comparison method, and that Its ultimate accuracy Is therefore limited only by the calibration material available. Fortunately, It s now possible to obtain complete sets of calibration samples from the National Bureau of Standards. 

Structure comparison methods are a way to compare three-dimensional structures. They are important for at least two reasons. First, they allow for inferring a similarity or distance measure to be used for the construction of structural classifications of proteins. Second, they can be used to assess the success of prediction procedures by measuring the deviation from a given standard-of-truth, usually given via the experimentally determined native protein structure. Formally, the problem of structure superposition is given as two sets of points in 3D space each connected as a linear chain. The objective is to provide a maximum number of point pairs, one from each of the two sets such that an optimal translation and rotation of one of the point sets (structural superposition) minimizes the rms (root mean square deviation) between the matched points. Obviously, there are two contrary criteria to be optimized the rms to be minimized and the number of matched residues to be maximized. Clearly, a smaller number of residue pairs can be superposed with a smaller rms and, clearly, a larger number of equivalent residues with a certain rms is more indicative of significant overall structural similarity. 

This may be trivial if sequence identity between the target sequence and a known 3D stmcture is high (say > 30%), as then simple pair-wise sequence comparison methods (FASTA, SSEARCH ) will easily identify the relation-ship. Where sequence identity is lower and a superfamily or even fold relationship must be identified, recognition of the stmctural similarity between two sequences may be very difficult. Sequence-only methods, such as PSI-BLAST, hidden Markov models and intermediate sequence search, use information from multiple sequence alignments to represent the characteristics shared by related sequences (sequence profiles), and use this to search for stmctural homologues. These profiles can then be augmented by secondary stmcture prediction. ... 

Analytical methods can be sub-divided into difference tests and descriptive analysis. Difference tests such as triangle and paired comparison tests are designed to identify differences between samples. In a triangle test, the assessor is given three samples, two of which are the same, and asked to identify which sample is different. In a paired comparison test, the assessor is asked to identify whether there is a difference in a particular sensory characteristic between a pair of samples. Descriptive analysis, in which the sensory characteristics of a sample are described and scored on a scale, is probably the most important analytical method. 

Variability Of Blank Responses. In order to limit the discussion, let us focus on water analyses as representative of environmental analyses. In the United Kingdom, the Standing Committee of Analysts of the Department of the Environment issues analytical methods In a series of booklets. Included among these are the Methods for the Examination of Waters and Associated Materials. 10-14) Several of these methods have been evaluated by Individual laboratories to determine the limit of detection based on the variability of the blank and using paired comparisons for blank correction. Published values for the limit of detection for several of these methods are... 

Yoshihara measured Tb in terbium oxide Tb O, and in terbium chloride solution by the sum peak pair method , and pointed out that the difference of the sum peak intensity ratio between two states could be enlarged by using comparison of a sum peak pair. This method has an advantage that the dependence of the sum peak pair ratio on the soun -detector distance is not as noticeable as that in the ordinary sum peak method. 

---