Content validity

Parrott, A.C., Performance tests in human psychopharmacology (2) content validity, criterion validity, and face validity, Hum. Psychopharmacol. Clin. Exp., 6, 91, 1991. 

The FAQLQ-PF and -TF were developed and validated in four stages (1) item generation using focus groups with both children and parents, expert opinion, and literature review (2) item reduction, using clinical impact and factor analysis (3) internal and test-retest reliability and construct validity were evaluated and (4) cross-cultural and content validity was examined by administering the questionnaire in a US sample (FAQLQ-PF, only). 

Presently, validity is represented as a process whereby we determine the degree of confidence we can place on inferences we make about people based on their scores from that scale. Some different types of validity, a discussion of which is beyond the scope of this chapter, are called content validity, criterion validity, and construct validity. 

Content validity, which is tested infrequently statistically, refers to how adequately the questions/items capture the relevant aspects of the domain or concept being measured. 

Validity, in contrast, is the ability of a scale to measure what it was designed to measure. Content validity measures the extent to which the scale assesses appropriate aspects of the iUness. Concurrent validity is a measure of the correlation of the rating scale with an external measure such as diagnosis or clinical change. Construct validity is the extent to which the test appears to measure... 

Content validity is perhaps the simplest but least convincing measure. If each of the items of our measurement device displays the correct content, then validity is established. Theoretically, if we could list all of the possible measures of a phenomenon, content validity would describe how well our measurement device samples these possible measures. In practice it is assessed by having experts in the field judge each item for how well its content represents the phenomenon studied. Thus, the heat balance equation would be judged by most thermal physiologists to have a content that well represents the thermal load on an operator. Not all aspects are as easily validated ... 

Chemical education researchers are typically asked to demonstrate that the test instruments used in their studies are valid and reliable. Validity is the extent to which a question measures what it purports to measure. There are several forms of validity (face validity, content validity, construct validity, etc.), and different forms become relevant for different test instruments. Reliability, on the other hand, is the extent to which the question is stable over time, i.e., if the instrument were administered to similar groups of students at two different times. 

Measurements can be reliable, but useless, without validity. Most validity studies have compared the results of one instrument to another instrument or to known quantities. This is the classical type of validity testing, which is an effort to determine whether the measurement reflects the variable being measured. In the absence of a gold standard this type of testing is of limited value. In addition to traditional studies of the validity of measurements, the issue of the validity of the inferences based on the measurements is becoming increasingly important (Rothstein and Echternach, 1993). That is, can the measurements be used to make inferences about human performance in real-life situations Unfortunately, measurements that have not demonstrated more than content validity are frequently... 

The data are obtained from the Dutch agri-food industry in 2002 and 2005 by means of a questionnaire survey. In order to increase the content validity, the questionnaires of both years were pre-tested with experts from business practice. If necessary, the questions were changed in order to increase a uniform interpretation of the used concepts. Only companies with more than 50 employees were included in this study. Their addresses and the number of employees were obtained from the Dutch Chambers of Commerce. In 2002, a questionnaire was sent to 355 companies of which 106 were returned. This equals a response rate of 29.9%. In 2005, 418 questionnaires were sent of which 100 questionnaires were returned, equaling a response rate of 23.9%. All the returned questionnaires could be used for analysis. In total, 23 companies were included in both years. In order to enable independent comparison between the sample of 2002 and 2005, they are included in the sample of 2002 (N=106) only and excluded from the sample of2005 (N=77). 

Content validity Good derived from literature Good as for initial instrument Doubtful limited eoverage only... 

Validity refers to whether the smvey instrument measures what it claims to measure. There are three basic types of validity for a measurement scale, each with particular experimental and statistical methodologies for evaluation. These are content validity (do relevant experts agree that the survey appears to measure what it is supposed to measme ), criterion validity (can scores from the survey be used to predict individual behavior or performance ), and construct validity (are the relationships found with the survey consistent with relevant theory and research ). 

In the weight example, the scale looks like it measures the correct weight of a person standing on it (content validity), but if results were compared with readings of other scales or with results of another estimate of weight, the numbers would not correspond. Construct validity would be questionable. Plus, this weight scale could not predict other... 

To achieve the content validity for IT resoimces, previous literature was reviewed (e.g., Ross et al. 1996 Weill et al. 1996 Armstrong and Sambamurthy 1999 Bharadwaj 2000 Byrd and Turner 2000 Dehning and Richardson 2002 Melville et al. 2004 Peppard and Ward 2004 Ranganathan et al. 2004 Bhatt and Grover 2005 Piccoli and Ives 2005 Ravichandran and Lertwongsatien 2005 Ray et al. 2005). Based on the definitions presented in Table 3.1, 14 items were developed to measure IT resoimces as the bundles of FT assets and capabilities that can be used to support lOS use in supply chain collaboration. These initial items were developed with two scales in mind. 

Questionnaires were sent to 5,717 executives in these companies, who had been identified as appropriate key respondents. The questionnaire had been pre-tested with ten executives from relevant companies and six academic experts. This step allowed for the assessment and evaluation of the presentation and content validity of items, ensuring that executives understand the instmctions, questions, aixi response scales of the questionnaire as they were intended. To reduce autocorrelation effects, questions forming one constmct were placed in separate sections of the questionnaires. The survey followed the guidelines prescribed in Dillman (1978). 

---