Choosing the right model for policy decision-making: the case of smallpox epidemiology

Policymakers increasingly draw on scientific methods, including simulation modeling, to justify their decisions. For these purposes, scientists and policymakers face an extensive choice of modeling strategies. Discussing the example of smallpox epidemiology, this paper distinguishes three types of strategies: Massive Simulation Models (MSMs), Simulation Models (ASMs) and Macro Equation Models (MEMs). By analyzing some of the main smallpox epidemic models proposed in the last 20 years, it discusses how to justify strategy choice with reference to the core characteristics of these respective strategies. First, I argue that MEMs often suffice for policy purposes, and need to be replaced only if they are insufficiently robust. Such robustness results, however, only requires the use of ASMs, not MSMs. Second, I argue that although MSMs have larger potentials than ASMs in various dimensions, they are also more likely to fail-and that in many cases, this probability of failing outweighs their higher potential. In particular, these dimensions include the representation of the relevant target, the accurate measurement of the relevant parameters, the number of parameters included, the number of mechanisms modeled simultaneously, and the ways of dealing with structural uncertainty. While this in no way excludes the prospect that some MSMs provide good justifications for policy decisions, my arguments caution against a general preference for MSMs over ASMs for policy decision purposes in general and vaccination problems in particular.