How well can method substitute for data? Five experiments in poverty analysis

No one doubts that good data are essential to sound policymaking. Alas, data are invariably faulty. Methodological solutions to data inadequacies have often been proposed and implemented, but they have been tested only rarely. Yet the methods that are used may well determine the direction of policy. For example, the particular survey method used-and the way nonsurvey data are interpreted-may be critical in assessing whether a country's strategy for reducing poverty is working. This article shows how counterfactual experiments can help test the reliability of various methods of dealing with common data problems. Well-designed methods-and they need not be very complicated-can help get around the problem, although it appears that substituting method for data is a long way from being perfect.