Sequential sampling, magnitude estimation, and the wisdom of crowds

Sir Francis Galton (Galton, 1907) conjectured the psychological process of magnitude estimation caused the curious distribution of judgments he observed at Plymouth in 1906. However, after he published Vox Populi, researchers narrowed their attention to the first moment of judgment distributions and its often remarkable alignment with the truth, while it became customary to explain this wisdom of crowds effect using ideas of statistics more than psychology, and without considering possible interactions with other distribution moments. Recently, however, an exploration of the cognitive foundation of judgment distributions was published (Nash, 2014). The study not only formalized a possible link between signal detection, evidence accumulation, and the shape of judgment distributions, but also in so doing, conjectured that magnitude estimation by independent individuals causes a systematic error in the wisdom of crowds indicated by judgment distribution skewness. The present study reports findings from an experiment on magnitude estimation and supports these predictions. The study moreover demonstrates that systematic errors by groups of people can be corrected using information about the judgment distribution these people together form, before errors might cause damage to decision making. In concluding, we revisit Galton's data from the West of England Fat Stock and Poultry Exhibition in light of what we have discovered. (C) 2017 The Author(s). Published by Elsevier Inc.