How Do Lotteries and Study Results Influence Response Behavior in Online Panels?

Two incentive experiments on response behavior were conducted in a nonprofit online panel. Experiment 1 examined effects of a lottery and of the lottery's splitting into multiple prizes. Two cash lotteries and a no-incentive control group were compared. One lottery was announced to be paid out in one lump sum, whereas the other lottery was split into multiple smaller prizes. Dependent variables were two facets of response quantity, namely response and retention, and two facets of response quality, namely nondifferentiation and item nonresponse. Moreover, panelists' characteristics were tested whether they moderated the lottery effects. The lottery and its splitting did not significantly affect response behavior; however, in terms of effect sizes, splitting the lottery mildly decreased response quantity. Experiment 2 was in part aimed at replication. In addition, it examined the effect of offering study results. Dependent variables were response, retention, and nondifferentiation. The cash lottery mildly enhanced response quantity and quality, whereas splitting the lottery tended to decrease response quantity. Offering study results had no impact on response behavior, both as a stand-alone incentive and in combination with a lottery. The two experiments revealed moderating effects, but these were not stable across both studies. A share of invitees in Experiment 1 was reinvited in Experiment 2, thus enabling analysis of whether their lottery condition in Experiment 1 influenced their response behavior 5 months later in Experiment 2. No such longitudinal effects were found.