Rank order tests

Feltovich, N. Nonparametric tests of differences in medians comparison of the wilcoxon-mann-whitney and robust rank-order tests. Exp. Econ. 6, 273-297 (2003)... 

Feltovich, N. Critical values for the robust rank-order test. Commun. Stat. Simul. Comput. 34(3), 525-547 (2005)... 

In paired comparison tests two different samples are presented and one asks which of the two samples has most of the sensory property of interest, e.g. which of two products has the sweetest taste (Fig. 38.3). The pairs are presented in random order to each assessor and preferably tested twice, reversing the presentation order on the second tasting session. Fairly large numbers (>30) of test subjects are required. If there are more than two samples to be tested, one may compare all possible pairs ( round robin ). Since the number of possible pairs grows rapidly with the number of different products this is only practical for sets of three to six products. By combining the information of all paired comparisons for all panellists one may determine a rank order of the products and determine significant differences. For example, in a paired comparison one compares three food products (A) the usual freeze-dried form, (B) a new freeze-dried product, (C) the new product, not freeze-dried. Each of the three pairs are tested twice by 13 panellists in two different presentation orders, A-B, B-A, A-C, C-A, B-C, C-B. The results are given in Table 38.3. 

The results of such multiple paired comparison tests are usually analyzed with Friedman s rank sum test [4] or with more sophisticated methods, e.g. the one using the Bradley-Terry model [5]. A good introduction to the theory and applications of paired comparison tests is David [6]. Since Friedman s rank sum test is based on less restrictive, ordering assumptions it is a robust alternative to two-way analysis of variance which rests upon the normality assumption. For each panellist (and presentation) the three products are scored, i.e. a product gets a score 1,2 or 3, when it is preferred twice, once or not at all, respectively. The rank scores are summed for each product i. One then tests the hypothesis that this result could be obtained under the null hypothesis that there is no difference between the three products and that the ranks were assigned randomly. Friedman s test statistic for this reads... 

In non-metric MDS the analysis takes into account the measurement level of the raw data (nominal, ordinal, interval or ratio scale see Section 2.1.2). This is most relevant for sensory testing where often the scale of scores is not well-defined and the differences derived may not represent Euclidean distances. For this reason one may rank-order the distances and analyze the rank numbers with, for example, the popular method and algorithm for non-metric MDS that is due to Kruskal [7]. Here one defines a non-linear loss function, called STRESS, which is to be minimized ... 

Frequency domain performance has been analyzed with goodness-of-fit tests such as the Chi-square, Kolmogorov-Smirnov, and Wilcoxon Rank Sum tests. The studies by Young and Alward (14) and Hartigan et. al. (J 3) demonstrate the use of these tests for pesticide runoff and large-scale river basin modeling efforts, respectively, in conjunction with the paired-data tests. James and Burges ( 1 6 ) discuss the use of the above statistics and some additional tests in both the calibration and verification phases of model validation. They also discuss methods of data analysis for detection of errors this last topic needs additional research in order to consider uncertainties in the data which provide both the model input and the output to which model predictions are compared. 

A further difficulty with small scale tests is that the relative fire performance and even the rank order of materials can change with different fire environments. Small scale tests can rarely reflect real life fire situations and examples already exist where reliance on small scale tests has let to hazardous full scale situations. 

Except for the 18 inhibitors of calcein accumulation, the model produced a reasonable rank ordering of the tested molecules... 

Twenty-three companies produced significant economic value and outperformed their industry competition primarily through organic or internal growth. The companies are presented in a rank order based on performance on the tests. 

Measurement of tensile or shear stress is the most commonly used in vitro method to determine bioadhesion. All in vitro measurements provide a rank order of bioadhesive strength for a series of candidate polymers. Measurement of tensile strength involves quantitating the force required to break the adhesive bond between the test polymer... 

On comparing the activities of the five compounds for which numerical estimates are available in all three assays (synephrine, octopamine, phenylethanolamine, norepinephrine and tyramine) the rank orders of potency in the three systems are Crayfish, 1,2,3t4,5 Cockroach, 2,1,3,4,5 Locust 1,2,3t5,4. This indicates a basic similarity in the responses of these preparations. In each case it was found that ring hydroxylation of the phenylethylamine nucleus was not essential for activity, although p-hydroxylation does yield the best activity. This is particularly evident in the crayfish study where a-MAMBA (a-methylaminomethyl benzyl alcohol), the analog of synephrine which lacks ring substitution, was one of the most active compounds tested, and 3-phenylethanolamine, the corresponding analog of OA, is almost as active as OA. The base compound for this series, phenylethylamine, also shows appreciable activity, but only in the crayfish assay. 

FIGURE 5.6 Bayesian confidence limits of the fraction affected percentiles (5th, 50th, and 95th) of posterior normal cdfs for cadmium. Data plotted cumulatively at (i - 0.5)/n, with i rank order, and n the number of species tested. 

This approach makes the assumption that any effects on V ax are independent of substrate, i.e. the rank order of rates of metabolism is the same for a particular cDNA-expressed enzyme and the same enzyme present in human tissue preparations, and any factor which affects V ax for one substrate also does so equally for other substrates. The validity of this assumption has not been rigorously tested but for most enzymes an appropriate set of test compounds is available. 

Test can be made at a number of extensions and compounds can be compared in terms of fatigue life at the same strain or at the same strain energy. In the latter case, absolute comparisons can be made on compounds of different modulus. When comparing different rubbers, it is necessary to test at a number of strains or to define the severity of conditions which will occur in service, because with the number of variables (G, K, W, n and C0) the ranking order may vary with the maximum strain employed. 

Semiquantitative risk assessments may consist of rank-order statements or ratio statements. The more quantitative assessment uses ratios of mutagenicity. It would be useful to know, for example, that chemical A is consistently 10 times as mutagenic as chemical B. Unfortunately, chemical activity among different tests is often inconsistent , and such ratios are likely to vary greatly. 

---