The Impact of Splitting a Long Online Questionnaire on Data Quality

Long self-administered questionnaires may suffer from lower response rates, higher drop-outs, and lower quality responses. A shorter questionnaire reduces the burden of respondents. Using this as a starting point, we test the following method: split the long questionnaire into sub-questionnaires; invite everyone to answer the first sub-questionnaire; when respondents complete the first sub-questionnaire, invite them to answer the second sub-questionnaire, and so on. We present evidence that after splitting a long questionnaire into two shorter parts, the response rates of these sub-questionnaires are significantly higher than the response rate of the original, undivided, long questionnaire. However, the cumulative response rate of both parts is lower than the response rate of the long undivided questionnaire. Finally, we show that the respondents of the survey using the original, long questionnaire: i) provide more non-substantive answers ("neither/nor") to the Likert-type scale items and ii) give shorter answers to the open-ended questions of the survey than the respondents of the split survey. On the other hand, there is no significant difference between the long and the split questionnaire on the other indicators of response quality we have tested: item-nonresponse, speeding and straight-lining. This paper presents some first insights on splitting a long questionnaire into shorter parts. For now, the results are not promising to suggest with confidence to split the long questionnaire for the purpose of obtaining high data quality. Further research is needed to find the optimal interval time between the sub-questionnaires or the optimal length of the sub-questionnaires in which the overall response rate is maximized.