Test-retest reliability

Devereux Elementary School Behavior Rating Scale. The Devereux Elementary School Behavior Rating Scale is a widely used test incorporating 47 items that have a high test-retest reliability. It uses a checklist format and is easy to use (requires 10 min). There are 11 factor scores and three item scores. 

Silverman, W.K., and Rabian, B. (1995) Test-retest reliability of the DSM-III-R anxiety childhood disorders symptoms using the Anxiety Disorders Interview Schedule for Children. J Anxiety Dis 9 1-12. 

Chambers, W.J., Puig-Antich, J., Hirsch M., Paez P., Ambrosini, P.J., Tabrizi M.A., and Davies, M. (1985) The assessment of affective disorders in children and adolescents by semistructured interview. Test—retest reliability of the Schedule for Affective Disorders and Schizophrenia for School-Age Children, Present Episode Version. Arch Gen Psychiatry 42 696—702. 

Finally, a cognitive performance assay used repeatedly during sleep deprivation should have high test-retest reliability it should be demonstrated to be sensitive to a large proportion of the performance phenomena associated with sleep loss and it should have the capacity to reflect aspects of real world performance (i.e., ecological validity). 

The FAQLQ-PF and -TF were developed and validated in four stages (1) item generation using focus groups with both children and parents, expert opinion, and literature review (2) item reduction, using clinical impact and factor analysis (3) internal and test-retest reliability and construct validity were evaluated and (4) cross-cultural and content validity was examined by administering the questionnaire in a US sample (FAQLQ-PF, only). 

Reliability refers to the extent to which measures give consistent or accurate results. The purpose of evaluating the reliability of a QOL instrument is to estimate how much of the variation in a score is real as opposed to random. The two reliability assessment methods discussed most often in the HRQOL literature are internal consistency and test-retest reliability. Internal consistency is an assessment of the performance of items within a scale. It is a function of the number of items and their covariation. Internal consistency is commonly measured using Cronbach s alpha coefficient. Alpha coefficients above 0.90 are recommended for making comparisons between individuals and above 0.70 for comparisons between groups. ... 

Test-retest reliability refers to the relationship between scores obtained from the same instrument on two or more separate occasions when all pertinent conditions remain relatively unchanged. It is usually evaluated using the intraclass correlation coefficient (ICC). However, QOL is not assumed to be constant over the course of time. In fact, most clinical studies attempt to assess how QOL changes. Test-retest reliability estimates may have limited value in evaluating measures that are designed to assess a dynamic process. 

Burt et al. (1998) developed the considerate and responsible co-worker CARE scale. The CARE scale measures an employee s attitudes toward safety and particularly their attitude toward the safety of their co-workers. As such, the items tend to use the word should. An employee who holds a positive and caring attitude toward safety should tend to agree with the items presented in the scale. A model examining a number of variables which are associated with CARE development is presented in Burt et al. (2008). Burt et al. (1998) reported a coefficients alpha of 0.91, and a test-retest reliability of 0.62 for the original 21 item Care scale (also see Burt 2001). Burt, Chmiel, and Hayes found an acceptable co-efficient alpha... 

Weir, J.P., 2005. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. 

Range 1-16 mo (Case-Smith and Bigsby, 2001) can be administered in a relatively brief period of time satisfactory inter-rater and test-retest reliability... 

Like all questionnaires that purport to reflect stable characteristics, it is important to determine just how stable or how consistent is a person s score over time. Ozkan et aL (2006) administered the DBQ to 622 drivers and found that over a three-year period the test-retest reliability of the scale scores was only 0.61. This is a fairly low level of reliability and it means that the DBQ can vary significantly over time, as one would expect given the findings that violations tend to diminish with age while lapses tend to increase with age. With a shorter -three-month - test-retest interval the reliability appears to be slightly higher (Parker et aL, 1995), and with a one week interval the test-retest reliability was higher still (Li et al., 2004 obtained r>.80 in China ). The utility of the DBQ, and its close relationship to scales of driver violence and vengeance (Li et aL, 2004) has also prompted the recent development of a Motorcycle Rider Behavior Questionnaire (Elliott et aL, 2007), in order to better understand the causes of motorcycle crashes (see Chapter 16). 

RL Doty, DA McKeown, WW Lee, P Shaman. A study of the test-retest reliability of ten olfactory tests. Chem Senses 20 645-656, 1995. 

Epstein, M. H., Hertzog, M. A., Reid, R. (2001). The Behavioral and Emotional Rating Scale Long term test-retest reliability. Behavioral Disorders, 26, 314-320. 

---