Are Survey Experiments Externally Valid?

被引:263
作者
Barabas, Jason [1 ]
Jerit, Jennifer [1 ]
机构
[1] Florida State Univ, Dept Polit Sci, Tallahassee, FL 32306 USA
基金
美国国家科学基金会;
关键词
STATISTICAL POWER; VOTER TURNOUT; INFORMATION; KNOWLEDGE; DELIBERATION; QUESTIONS; FIELD;
D O I
10.1017/S0003055410000092
中图分类号
D0 [政治学、政治理论];
学科分类号
0302 ; 030201 ;
摘要
Researchers use survey experiments to establish causal effects in descriptively representative samples, but concerns remain regarding the strength of the stimuli and the lack of realism in experimental settings. We explore these issues by comparing three national survey experiments on Medicare and immigration with contemporaneous natural experiments on the same topics. The survey experiments reveal that providing information increases political knowledge and alters attitudes. In contrast, two real-world government announcements had no discernable effects, except among people who were exposed to the same facts publicized in the mass media. Even among this exposed subsample, treatment effects were smaller and sometimes pointed in the opposite direction. Methodologically, our results suggest the need for caution when extrapolating from survey experiments. Substantively, we find that many citizens are able to recall factual information appearing in the news but may not adjust their beliefs and opinions in response to this information.
引用
收藏
页码:226 / 242
页数:17
相关论文
共 85 条
[1]   Who evaluates a presidential candidate by using non-policy campaign messages? [J].
Abrajano, MA .
POLITICAL RESEARCH QUARTERLY, 2005, 58 (01) :55-67
[2]  
Achen CH., 1982, INTERPRETING USING R
[3]  
[Anonymous], 2010, 1987 News that Matters: Television and American Opinion
[4]  
ARCENEAUX K, 2007, 2007 ANN M MIDW POL
[5]   Educating the Least Informed: Group Endorsements in a Grassroots Campaign [J].
Arceneaux, Kevin ;
Kolodny, Robin .
AMERICAN JOURNAL OF POLITICAL SCIENCE, 2009, 53 (04) :755-770
[6]   How deliberation affects policy opinions [J].
Barabas, J .
AMERICAN POLITICAL SCIENCE REVIEW, 2004, 98 (04) :687-701
[7]   Estimating the Causal Effects of Media Coverage on Policy-Specific Knowledge [J].
Barabas, Jason ;
Jerit, Jennifer .
AMERICAN JOURNAL OF POLITICAL SCIENCE, 2009, 53 (01) :73-89
[8]  
Bennett W.L., 2006, NEWS POLITICS ILLUSI, V7th
[9]   Do people behave in experiments as in the field? evidence from donations [J].
Benz, Matthias ;
Meier, Stephan .
EXPERIMENTAL ECONOMICS, 2008, 11 (03) :268-281
[10]   Assuming the costs of war: Events, elites, and american public support for military conflict [J].
Berinsky, Adam J. .
JOURNAL OF POLITICS, 2007, 69 (04) :975-997