Human Beyond the Machine: Challenges and Opportunities of Microtask Crowdsourcing

被引:28
作者
Gadiraju, Ujwal [1 ]
Demartini, Gianluca [2 ]
Kawase, Ricardo [1 ]
Dietze, Stefan [1 ]
机构
[1] Leibniz Univ Hannover, Res Ctr L3S, Hannover, Germany
[2] Univ Sheffield, Informat Sch, Data Sci, Sheffield S10 2TN, S Yorkshire, England
关键词
collective intelligence; crowdsourcing; data centric task; data science; microtask; MTurk; worker behavior;
D O I
10.1109/MIS.2015.66
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In the 21st century, where automated systems and artificial intelligence are replacing arduous manual labor by supporting data-intensive tasks, many problems still require human intelligence. Over the last decade, by tapping into human intelligence through microtasks, crowdsourcing has found remarkable applications in a wide range of domains. In this article, the authors discuss the growth of crowdsourcing systems since the term was coined by columnist Jeff Howe in 2006. They shed light on the evolution of crowdsourced microtasks in recent times. Next, they discuss a main challenge that hinders the quality of crowdsourced results: the prevalence of malicious behavior. They reflect on crowdsourcing's advantages and disadvantages. Finally, they leave the reader with interesting avenues for future research. © 2001-2011 IEEE.
引用
收藏
页码:81 / 85
页数:5
相关论文
共 19 条
[1]  
[Anonymous], 2010, Proceedings of the ACM SIGKDD Workshop on Human Computation, nod, DOI DOI 10.1145/1837885.1837906
[2]  
[Anonymous], 2014, 2 AAAI C HUMAN COMPU
[3]   Measuring Crowdsourcing Effort with Error-Time Curves [J].
Cheng, Justin ;
Teevan, Jaime ;
Bernstein, Michael S. .
CHI 2015: PROCEEDINGS OF THE 33RD ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, 2015, :1365-1374
[4]   The Dynamics of Micro-Task Crowdsourcing The Case of Amazon MTurk [J].
Difallah, Djellel Eddine ;
Catasta, Michele ;
Demartini, Gianluca ;
Ipeirotis, Panagiotis G. ;
Cudre-Mauroux, Philippe .
PROCEEDINGS OF THE 24TH INTERNATIONAL CONFERENCE ON WORLD WIDE WEB (WWW 2015), 2015, :238-247
[5]   Increasing cheat robustness of crowdsourcing tasks [J].
Eickhoff, Carsten ;
de Vries, Arjen P. .
INFORMATION RETRIEVAL, 2013, 16 (02) :121-137
[6]   Understanding Malicious Behavior in Crowdsourcing Platforms: The Case of Online Surveys [J].
Gadiraju, Ujwal ;
Kawase, Ricardo ;
Dietze, Stefan ;
Demartini, Gianluca .
CHI 2015: PROCEEDINGS OF THE 33RD ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, 2015, :1631-1640
[7]  
Gadiraju Ujwal, 2014, P 25 ACM C HYPERTEXT, P218
[8]  
Irani Lilly C., 2013, P SIGCHI C HUM FACT, P611, DOI [DOI 10.1145/2470654.2470742, 10.1145/2470654.2470742]
[9]  
Kazai G., 2011, P 20 ACM INT C INF K
[10]  
Kittur A., 2013, P 2013 C COMP SUPP C, P1301, DOI DOI 10.1145/2441776.2441923