Consistency testing base rate variability test

MAMBAC offers two standard internal consistency tests, the nose count test and the base rate variability test. MAMBAC allows for two analyses per pair of indicators, which means that two indicators produce two plots, three indicators produce six plots, four indicators produce 12 plots, five indicators produce 20 plots, etc. Waller s MAMBAC does not do this automatically, and the investigator has to run each analysis independently. Ruscio s software, on the other hand, is fully automatic. 

Nose count and base rate variability consistency tests are possible with MAXSLOPE, although it is not yet clear how these tests behave when the underlying distributions are of the difficult kind. Luckily, MAXSLOPE puts less emphasis on internal consistency testing and stresses external consistency testing instead. MAXSLOPE is different from other taxometric algorithms and thus can provide a strong test of external consistency for other procedures. 

As a result of this protocol, four indicators were dropped because in each case, they did not pass the first consistency test, that is, failed to discriminate adequately at all levels of the scale. Next, Tyrka et al. (1995) calculated the taxon base rate for each indicator using a hybrid of MAXCOV and Latent Class Analysis estimation procedures (for details see Golden, 1982) and adjusted the estimate for the true- and false-positive rates computed earlier. The average taxon base rate was. 49. The authors did not report a variability statistic, but a simple computation shows that SD of base rate estimates was. 04. 

Unfortunately, no consistency tests were reported in this study. Furthermore, instead of reporting the 28 individual base rate estimates, the authors reported an overall base rate estimate for each item, averaging over analyses in which the item was an output variable. This can be somewhat confusing because MAXCOV base rate estimates are associated with the input variable, not the output variable. The reason for this is MAXCOV base rate estimation is tied with locating the hitmax on the input indicator so that each base rate estimate is attached to a certain input indicator. This,... 

Lenzenweger (1999) performed MAXCOV using the three scales as taxon indicators. He used an interval size of. 50 SD and MRIN of 15. Two of the plots showed clear peaks, and one produced a cusp. The one incomplete peak was probably a consequence of the interval size being set too high, which allowed for only seven intervals on the input variable. A lower interval size would have produced a finer gradation and probably allowed the cusp to turn into a full peak. However, this may not have been possible due to the modest size of the sample. The base rates estimates were. 11,. 22, and. 13. The author did not report a base rate consistency test, but one can easily calculate the SD of the three estimates to be. 06, which is somewhat high but probably acceptable. 

---