Rating crowdsourced annotations: evaluating contributions of variable quality and completeness

被引:10
作者
Foody, Giles M. [1 ]
机构
[1] Univ Nottingham, Sch Geog, Nottingham NG7 2RD, England
关键词
volunteered geographic information; accuracy; error; LAND-COVER CHANGE; GEOGRAPHIC INFORMATION; DISASTER RESPONSE; DAMAGE ASSESSMENT; WEB; 2.0; ACCURACY; EARTHQUAKE;
D O I
10.1080/17538947.2013.839008
中图分类号
P9 [自然地理学];
学科分类号
0705 ; 070501 ;
摘要
Crowdsourcing has become a popular means to acquire data about the Earth and its environment inexpensively, but the data-sets obtained are typically imperfect and of unknown quality. Two common imperfections with crowdsourced data are the contributions from cheats or spammers and missing cases. The effect of the latter two imperfections on a method to evaluate the accuracy of crowdsourced data via a latent class model was explored. Using simulated and real data-sets, it was shown that the method is able to derive useful information on the accuracy of crowdsourced data even when the degree of imperfection was very high. The practical potential of this ability to obtain accuracy information within the geospatial sciences and the realm of Digital Earth applications was indicated with reference to an evaluation of building damage maps produced by multiple bodies after the 2010 earthquake in Haiti. Critically, the method allowed data-sets to be ranked in approximately the correct order of accuracy and this could help ensure that the most appropriate data-sets are used.
引用
收藏
页码:650 / 670
页数:21
相关论文
共 61 条
[1]   A Low-Cost Mobile Mapping System (LCMMS) for field data acquisition: a potential use to validate aerial/satellite building damage assessment [J].
Ajmar, Andrea ;
Balbo, Simone ;
Boccardo, Piero ;
Tonolo, Fabio Giulio ;
Piras, Marco ;
Princic, Jan .
INTERNATIONAL JOURNAL OF DIGITAL EARTH, 2013, 6 :103-123
[2]   Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS) [J].
Allouche, Omri ;
Tsoar, Asaf ;
Kadmon, Ronen .
JOURNAL OF APPLIED ECOLOGY, 2006, 43 (06) :1223-1232
[3]  
Antoniou V., 2010, Geomatica, V64, P99, DOI DOI 10.HTTP://DX.D0I.0RG/10.5623/GE0MAT-2010-0009
[4]  
Booth E., 2010, The Haiti Earthquake of 12 January 2010: A Field Report by EEFIT
[5]   THE MYTH OF AMATEUR CROWDS A critical discourse analysis of crowdsourcing coverage [J].
Brabham, Daren C. .
INFORMATION COMMUNICATION & SOCIETY, 2012, 15 (03) :394-410
[6]   First extensive and cost-effective quality check of Crisis Maps: presentation of assessment parameters and results [J].
Carrion, D. ;
Corbane, C. ;
Broglia, M. ;
Pesaresi, M. .
INTERNATIONAL JOURNAL OF DIGITAL EARTH, 2013, 6 :50-66
[7]   Digital Earth 2020: towards the vision for the next decade [J].
Craglia, Max ;
de Bie, Kees ;
Jackson, Davina ;
Pesaresi, Martino ;
Remetey-Fueloepp, Gabor ;
Wang, Changlin ;
Annoni, Alessandro ;
Bian, Ling ;
Campbell, Fred ;
Ehlers, Manfred ;
van Genderen, John ;
Goodchild, Michael ;
Guo, Huadong ;
Lewis, Anthony ;
Simpson, Richard ;
Skidmore, Andrew ;
Woodgate, Peter .
INTERNATIONAL JOURNAL OF DIGITAL EARTH, 2012, 5 (01) :4-21
[8]  
CSDGSND, 2010, UND CHANG PLAN STRAT
[9]   USGS Remote Sensing Coordination for the 2010 Haiti Earthquake [J].
Duda, Kenneth A. ;
Jones, Brenda K. .
PHOTOGRAMMETRIC ENGINEERING AND REMOTE SENSING, 2011, 77 (09) :899-907
[10]   Towards an integrated crowdsourcing definition [J].
Estelles-Arolas, Enrique ;
Gonzalez-Ladron-de-Guevara, Fernando .
JOURNAL OF INFORMATION SCIENCE, 2012, 38 (02) :189-200