Do Open-ended Survey Questions on Migration Motives Create Coder Variability Problems?

Contemporary research on migration has benefited from adopting a variety of methodological approaches and different sources of information to provide answers to the ever-recurring question of why people migrate. Yet, when it comes to central methods used for researching migration motives, progress appears to have been slow. This paper focuses on surveys to research migration motives using self-administered postal questionnaires. It addresses a key validity question, namely the issue of whether the usage of open-ended questions creates coder variability problems. An experimental research design was used where five coders independently coded 500 randomly selected responses from a large survey on migration motives. Krippendorff's alpha was calculated to test the level of agreement between the coders. The results advance our knowledge in two important ways: firstly, it is shown that coder variability is not a major problem (Krippendorff's alpha = 0.82). Secondly, it identifies those types of responses that nevertheless appear problematic to code. The implications of these findings for survey research on migration motives are discussed, and it is argued that open-ended questions have some distinct advantages compared with the more commonly used closed-ended questions. Copyright (C) 2009 John Wiley & Sons, Ltd.