Sample sizes and return rates

You wrote it, so of course you understand it. However, rest assured, somewhere within your masterpiece lurks a question or an instruction that the public will manage to misconstrue. The only way to find it is to try the questionnaire out on a reasonable number of individuals similar to those who will ultimately act as your respondents. 

As with any form of sample-based research, it is important to know in advance how much data (completed questionnaires) will be required. We can calculate a necessary sample size using the approaches described earlier in this book. We just need to identify the primary question, determine what statistical analysis will be used to answer it and then perform a sample size calculation in the normal way. 

There is one very large hazard waiting to get you. The danger is that you calculate 100 completed questionnaires are needed, post out 500 and get 110 returned. You have got back more than enough, so what is the problem  

The 110 out of 500 (22 per cent) who did return the questionnaire are a self-selected minority. We already know that they differ in one way — they did return the questionnaire, while the other 78 per cent did not. However, is this decision to complete the form just random or is it linked to other personal characteristics It would be a brave (foolhardy ) researcher who was prepared to assume that returning or not returning the questionnaire was a purely random choice. The reality is that there are all sorts of ways in which the returners may differ from the non-returners. With a return rate as low as 22 per cent, we could be dealing with a highly unrepresentative minority and any conclusions will be biased towards whatever type of person happens to be most likely to return our questionnaire. 

One obvious likelihood is that those with a strong interest in, or opinion concerning, the relevant subject will be more likely to respond. Try asking this in a postal questionnaire  

---