Paired Comparison Analysis

The next task is to prioritize and select a design by filtering through a number of design concepts to the point where just one is pursued for further development. Two techniques are very helpful in this regard Paired Comparison Analysis and Pugh Matrix. You can also use both of these techniques... 

Use Paired Comparison Analysis when you need to compare either more upstream innovation ideas or more downstream design concepts. This technique is especially helpful when you don t have objective data regarding how different ideas could meet your customers outcome expectations (see Technique 2), or when you re uncertain about how different design concepts could meet customer performance and perception expectations (see Technique 30). 

Part of this technique s advantage is that it s easier to use than such other comparative techniques as the Pugh Matrix (Technique 36). Therefore, Paired Comparison Analysis enables you to quickly but comprehensively assess the relative worth of all your options before committing to any specific one. 

Classically, completing a Paired Comparison Analysis calls for the use of brainstorming techniques to come up with great alternatives. We recommend using any or all of the ideation techniques in this book to move significantly beyond classic brainstorming. If simple brainstorming were sufficient, more teams would come up with more innovations more often than not. But this is not the case. 

You should not eliminate any alternatives at this point. Paired Comparison Analysis is a powerful tool for determining differences between alternatives, even if they seem very similar. 

Based on these figures, the choir theme is the advertising idea seen as most likely to deliver on the market-share goal for the client s new product. Be careful though—the Paired Comparison Analysis technique is used for determining which alternative is best based on only one criterion the operational definition. 

If you are interested in the original, classic works on Paired Comparison Analysis, see ... 

The results of such multiple paired comparison tests are usually analyzed with Friedman s rank sum test [4] or with more sophisticated methods, e.g. the one using the Bradley-Terry model [5]. A good introduction to the theory and applications of paired comparison tests is David [6]. Since Friedman s rank sum test is based on less restrictive, ordering assumptions it is a robust alternative to two-way analysis of variance which rests upon the normality assumption. For each panellist (and presentation) the three products are scored, i.e. a product gets a score 1,2 or 3, when it is preferred twice, once or not at all, respectively. The rank scores are summed for each product i. One then tests the hypothesis that this result could be obtained under the null hypothesis that there is no difference between the three products and that the ranks were assigned randomly. Friedman s test statistic for this reads... 

Sensory Analysis. A paired comparison test was run to determine if the difference in oil droplet size in the emulsion changed the perceived intensity of the orange flavor. The coarsest emulsion (3.87 pM) and the Microfluidized sample (0.90 pM) from the third set of spray dried samples were compared. The solutions were prepared using 200 ppm flavor in a 10% (w/v) sucrose solution with 0.30% of a 50% citric acid solution added. The amount of each powder required to attain 200 ppm orange oil was calculated on the basis of percent oil in each powder (determined by Clevenger analysis). A pair of samples at approximately 10 C was given to each of 24 untrained panelists. The samples were coded with random numbers. Half the panelists were asked to taste the coarsest sample first while while the other half tasted the Microfluidized sample first. This was done to determine whether or not adaptation was a factor. The panelists were asked to indicate which sample had the most intense orange flavor. 

Lewis DE, Moore CM, Leikin JB, KoUe, A. Meconium analysis for cocaine a validation study and comparison with paired urine analysis. J Anal Toxicol 1995 19 148-50. 

ISO 5495 Sensory Analysis - Methodology - Paired Comparison Test (1983)... 

Analytical methods can be sub-divided into difference tests and descriptive analysis. Difference tests such as triangle and paired comparison tests are designed to identify differences between samples. In a triangle test, the assessor is given three samples, two of which are the same, and asked to identify which sample is different. In a paired comparison test, the assessor is asked to identify whether there is a difference in a particular sensory characteristic between a pair of samples. Descriptive analysis, in which the sensory characteristics of a sample are described and scored on a scale, is probably the most important analytical method. 

Making paired comparisons. Even though we have just done an incorrect statistical analysis (because the samples were not independent),... 

Bioassay responses were calculated as LD q s, ED q s or ET q s using the trimmed Spearman-Karber method (14). Duncan s multiple range test and analysis of variance were used for the repellency trials, and the paired comparison t-test was used to analyze the food preference experiments. 

If the product cannot be disassembled and reassembled, the technique to use is paired comparisons. The concept is to select pairs of good and bad units and compare them, using whatever visual, mechanical, electrical, chemical, etc., comparisons are possible, recording whatever differences are noticed. Do this for several pairs, continuing until a pattern of differences becomes evident. In many cases, a half-dozen paired comparisons is enough to detect repeatable differences. The units chosen for this test should be selected at random to establish statistical confidence in the results. If the number of differences detected is more than four, then use of variables search is indicated. For four or fewer, a full factorial analysis can be done. 

Miller, R.W., 2002. Subjective property characterization by Quad analysis an efficient method for conducting paired comparisons. Text. Res. J. 72, 1041-1051. 

Incubations were performed in duplicate in each experiment. All statistical evaluations were based on paired comparisons. FFA were evaluated by the procedure of Dole and Meinertz and glycerol by the enzymatic method of Wieland (3, 4). The triglycerides content of the flasks, based on analysis of the free fatty acids after saponification was used for quantitative evaluation of the cells, present in each flask. 

The most rigorous method to evaluate the contribution of belts to fatality reduction was developed by Evans (1986). With this method, known as the "double pair comparison" method, Evans first identified all fatal crashes involving cars with multiple occupants (where the driver was not necessarily the one who was killed). He then compared the fatality likelihood for drivers with and without belts relative to the likelihood of fatality of the other occupant with and without belts. The beauty of this approach is that there is no need for exposure measure, and hence no need to be concerned about exposure bias. Using the U.S. national Fatal Analysis Reporting System which documents every fatal crash in the U.S., Evans estimated that the... 

The second component of analysis for determining the cognitive load effects of nanoscience and materials science contexts on exam items for the two different chemistiy courses, involved a paired item analysis. Two nanoscience items in the sub-categories of spectroscopy and band theory, as well as two materials items in the sub-categories of intermolecular forces (IMF) and Lewis stmctures, were inserted into the practice exam to serve as paired items in these same sub-categories. The first comparison we analyzed was both groups of students differential performance on non-nanoscience and materials exam items versus nanoscience and materials items (Figures 4 5). 

---