A Bayesian Hierarchical Model of Crowd Wisdom Based on Predicting Opinions of Others

In many domains, it is necessary to combine opinions or forecasts from multiple individuals. However, the average or modal judgment is often incorrect, shared information across respondents can result in correlated errors, and weighting judgments by confidence does not guarantee accuracy. We develop a Bayesian hierarchical model of crowd wisdom that incorporates predictions about others to address these aggregation challenges. The proposed model can be applied to single questions, and it can also estimate respondent expertise given multiple questions. Unlike existing Bayesian hierarchical models for aggregation, the model does not link the correct answer to consensus or privilege majority opinion. The model extends the "surprisingly popular algorithm" to enable statistical inference and in doing so, overcomes several of its limitations. We assess performance on empirical data and compare the results with other aggregation methods, including leading Bayesian hierarchical models.