Understanding Sample Surveys: Selective Learning about Social Science Research Methods

We investigate differences in what students learn about survey methodology in a class on public opinion presented in two critically different ways: with the inclusion or exclusion of an original research project using a random-digit-dial telephone survey. Using a quasi-experimental design and data obtained from pretests and posttests in two public opinion courses, we test the hypothesis that students who participate in an original survey research project will have a stronger understanding of survey research methods than students who do not. To better assess the effect of the active learning element of the course, we estimate average treatment effects on the students who participated in the original survey project using nearest neighbor matching (Abadie et al. 2004) with student scores on a pretest. We find evidence of modest improvement in learning of survey methods in the course featuring the original survey research project; however, the major finding here is that a course featuring this kind of opportunity appeals to a different kind of student than a course that allows participants to stay closer to the classroom and library instead of the social science research laboratory. This discovery may have important implications for our understanding of the effects of active learning opportunities in other types of elective courses.