MicroMobile: Leveraging Mobile Advertising for Large-Scale Experimentation

Mobile systems researchers struggle with conducting experiments with real users: either the scale of the study lacks sufficient scale and diversity, or a great effort must be used to recruit and manage subjects. In this paper, we describe MicroMobile, a system for deploying short data-gathering experiments to an extremely diverse set of users via mobile advertising. We conduct experiments in three mediums: interactive advertisements, mobile browsers, and native applications on both major mobile operating systems. We use MicroMobile to demonstrate how researchers can use mobile advertising to recruit users, for as little as $1.50 per completed experiment. Across almost 500 completed experiments, we found that interactive ads have the highest participation rate (and thus lowest cost), which was 2x the participation rate of browser experiments and more than 6x native app experiments. Users were also highly diverse, spanning age, income, and ethnicity. While native apps are the most powerful platform, they constitute the most expensive targets. However, as mobile browsers add sensor APIs, browser-based experimentation has increasing applicability.