Questionnaires multiple testing

Emphasize the potential hazard of multiple testing with questionnaire data... 

Endless pairs of questions could be cross-tabulated and questionnaires are a heaven-sent opportunity for multiple testing. Because of this, it is particularly important that the primary analysis is planned in advance. Any amount of secondary analysis can ultimately be performed, so long as it is identified as such. 

The final chapters (18 and 19) cover some real-world problems that students probably will not run into until their final year, when carrying out their own independent research projects. The issues considered are multiple testing (all too common in student projects ) and questionnaire design and analysis. While the latter does not introduce many fundamentally new concepts, the use of questionnaires has increased so much, it seemed useful to bring together all the relevant points in a single resource. 

Lykken DT 1957 A study of anxiety in the sociopathic personality. J Abnorm Soc Psych 55 6—10 Lykken DT, Katzenmeyer C 1973 Manual for the activity preference questionnaire (APQ). In Psychiatric research reports. University of Minnesota Press, Minneapolis, MN McKeen Cattell J 1890 Mental tests and measurements. Mind 15 373—380 Messick S 1992 Multiple intelligences or multilevel intelligence Selective emphasis on distinctive properties of hierarchy On Gardner s Frames of mind and Sternberg s Beyond IQ in the context of theory and research on the structure of human abilities. Psych Inquir 3 365-384... 

Quizzes are games with questions and answers aimed at testing the knowledge of health professionals. These questionnaires can be in several forms multiple choice questionnaires (MCQs), open and short question questionnaires, and yes/no questionnaires. 

On return of the multiple blood test results, the test group was reviewed in order to see how many patients required further follow-up, blood tests, or referral to hospital. In addition, for every patient in the test group a questionnaire was completed by the general practitioners 3-6 months after the blood test. On this questionnaire any results marked by the laboratory staff as abnormal were placed in one of three categories (1) an expected abnormal result, (2) an unexpected abnormal result leading to a new or additional diagnosis, or (3) an unexpected abnormal result, unexplained at the time of completing the questionnaire. 

In the questionnaire sent at the end of the SMS study, people were asked to indicate in what kind of situations they would like to receive a warning message from the authorities. This was a multiple choice question with an option for additions. Table 1 shows what percentage of people would like to receive information in different types of emergency situations. It is striking that not many people choose smell inconvenience , when the location of this test (the city of Vlaardingen, which is located close to a big chemical industry area) is a place where this type of announcements occurs quite often. 

Students prior general knowledge of scientific topics was examined by a multiple-choice questionnaire (11 questions). The questions were selected fi om flic published form of the international standards test TIMMS (TIMMS, 1999) which was designed for 8 grade students internationally. Internal reliability was measured using the Kuder-Richardson formula 20, a =0.82. 

---