Interrater-reliability

Ralafut MA, Schriger DL, Saver XL, Starkman S. Detection of early CT signs of >1/3 middle cerebral artery infarctions interrater reliability and sensitivity of CT interpretation by physicians involved in acute stroke care. Stroke 2000 31 1667-1671. 

A variety of decision rules were also provided to further assist clinicians in the use of the DSM-III and, ultimately, to increase interrater reliability. These decision rules pertain to specific criteria as well as to differential diagnoses and are presented to give clinicians more guidance and allow for less leeway in making diagnostic decisions. The diagnostic process is effectively... 

By many practical measures, the DSM-III (American Psychiatric Association, 1980) has been an immensely successful taxonomy. The energy that was focused on improving interrater reliability appears to have been successful. Compared with earlier versions, the DSM-III resulted in significantly higher interrater agreement for most psychiatric conditions. Conver-... 

The reason for the double-edged nature of reliability is that reliability is a necessary but not sufficient criterion for validity. It is necessary to make consistent assessments to have a valid system. However, it is possible that these reliable assessments measure something that is largely irrelevant to what is actually of interest. The quest for increasing interrater reliability has resulted in disassembling disorders into only those components that can be reliably assessed. However, some have suggested that some of the more essential behaviors (i.e., valid indicators of the diagnostic entity) that are less reliably assessed have been excluded. 

Rudy EB, Estok P. Professional and lay interrater reliability of urinary luteinizing hormone surges measured by OvuQuick test. J Obstet Gynecol Neonatal Nurs 1992 21 407-11. 

Interrater reliability and equivalent-forms reliability are two other approaches to reliability assessment that are not used as commonly in QOL research. More in-depth discussions of these and the other reliability assessment methods are found elsewhere. ... 

Reliability is the extent to which the score on the scale reflects the hypothetical true score and how much interference occurs from outside influences. Reliability is reported by the correlation coefficient, which represents a chance correlation (zero) or perfect correlation (one). Rating scales with reliability correlation coefficients of less than 0.7 are usually considered unreliable for clinical studies. Interrater reliability—agreement in rating scores among clinicians— is important to achieve when multiple clinicians rate the same patient or popnlation. Interrater reliability is established by having all raters independently rate individual patients at the same time to determine the correlation of their scores. Other types of reliabflity include... 

Brief Psychiatric Rating Scale (BPRS) Clinician-rated 18 items, 7-point severity scale score >38 indicates moderate severity The anchored BPRS provides descriptions of each severity rating to increase the interrater reliability. The BPRS has four clusters of symptoms thinking disturbance, anxious depression, withdrawal-retardation, and host i 1 ity-susp ic iou sness... 

Scharf J, Brockmann MA, Daffertshofer M et al (2006) Improvement of sensitivity and interrater reliability to detect acute stroke by dynamic perfusion computed tomography and computed tomography angiography. J Comput Assist Tomogr 30 105-110... 

James, L. R., Demaree, R. G., and Wolf, G. (1984), Estimating Within-group Interrater Reliability with and without Response Bias, Journal of Applied Psychology, Vol. 69, No. 1, pp. 85-98. 

Herrington, D. G Nakhleh, M. B. What Defines Effective Chemistry Laboratory Instruction Teaching Assistant and Student Perspectives. J. Chem. Ed 2003, 80, 1197-1205 (77). The article includes die entire survey that included one free response question. The findings from die qualitative analysis of the free response data are very carefiilly explained. An interrater reliability study is included. 

Danker-Hopfe, H., Kuntz, D., Gruber, G., Klosch, G., Lorenzo, J.L. et al.. Interrater reliability between scorers from eight European sleep laboratories in subjects with different sleep disorders, /. Sleep Res. 13, pp. 63-69,2004. 

Chiarello C.M., Savidge R. (1993) Interrater reliability of the Cybex EDI-320 and fluid goniometer in normals and patients with low back pain. Arch Phys MedRehabl. 74(l) 32-7. 

The BERS development process followed rigorous content validity procedures (Epstein, 1999). In addition, the bers has been normed using national samples of children with emotional distiubance (JV= 861) and without emotional disturbance [N= 2176) (Epstein Sharma, 1998), demonstrated convergent (Harniss et al., 1999) and discriminant validity (Reid et al., 2000), and shown shortterm and long-term test-retest and interrater reliability (Epstein et al., 1998 Epstein, Hertzog, Reid, 2001 Friedman, Leone, Friedman, 1999). Based on... 

---