MEDIC: a multi-task learning dataset for disaster image classification

被引:26
作者
Alam, Firoj [1 ]
Alam, Tanvirul [2 ]
Hasan, Md Arid [3 ,4 ]
Hasnat, Abul [5 ]
Imran, Muhammad [1 ]
Ofli, Ferda [1 ]
机构
[1] HBKU, Qatar Comp Res Inst, Doha, Qatar
[2] Rochester Inst Technol, Rochester, NY 14623 USA
[3] Cognit Insight Ltd, Dhaka, Bangladesh
[4] Daffodil Int Univ, Dhaka, Bangladesh
[5] BLACKBIRDAI, Rochester, NY USA
关键词
Multi-task learning; Social media images; Image classification; Natural disasters; Crisis informatics; Deep learning; Dataset; SOCIAL MEDIA;
D O I
10.1007/s00521-022-07717-0
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Recent research in disaster informatics demonstrates a practical and important use case of artificial intelligence to save human lives and suffering during natural disasters based on social media contents (text and images). While notable progress has been made using texts, research on exploiting the images remains relatively under-explored. To advance image-based approaches, we propose MEDIC (https://crisisnlp.qcri.org/meclic/index.html), which is the largest social media image classification dataset for humanitarian response consisting of 71,198 images to address four different tasks in a multi-task learning setup. This is the first dataset of its kind: social media images, disaster response, and multi-task learning research. An important property of this dataset is its high potential to facilitate research on multi-task learning, which recently receives much interest from the machine learning community and has shown remarkable results in terms of memory, inference speed, performance, and generalization capability. Therefore, the proposed dataset is an important resource for advancing image-based disaster management and multi-task machine learning research. We experiment with different deep learning architectures and report promising results, which are above the majority baselines for all tasks. Along with the dataset, we also release all relevant scripts (https://github.com/firojalam/medic).
引用
收藏
页码:2609 / 2632
页数:24
相关论文
共 101 条
[1]   JORD - A System for Collecting Information and Monitoring Natural Disasters by Linking Social Media with Satellite Imagery [J].
Ahmad, Kashif ;
Riegler, Michael ;
Pogorelov, Konstantin ;
Conci, Nicola ;
Halvorsen, Pal ;
De Natale, Francesco .
PROCEEDINGS OF THE 15TH INTERNATIONAL WORKSHOP ON CONTENT-BASED MULTIMEDIA INDEXING (CBMI), 2017,
[2]  
Ahmad S, 2017, P MEDIAEVAL
[3]  
Alam F., 2021, P INT AAAI C WEB SOC, V15, P933, DOI DOI 10.1609/ICWSM.V15I1.18116
[4]  
Alam F., 2021, ARXIV
[5]  
Alam F, 2019, P INT C INFORM SYSTE
[6]  
Alam F., 2021, P 15 INT AAAI C WEB, V15, P923
[7]   Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response [J].
Alam, Firoj ;
Ofli, Ferda ;
Imran, Muhammad ;
Alam, Tanvirul ;
Qazi, Umair .
2020 IEEE/ACM INTERNATIONAL CONFERENCE ON ADVANCES IN SOCIAL NETWORKS ANALYSIS AND MINING (ASONAM), 2020, :151-158
[8]  
Alam F, 2018, PROCEEDINGS OF THE 56TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL), VOL 1, P1077
[9]   Processing Social Media Images by Combining Human and Machine Computing during Crises [J].
Alam, Firoj ;
Ofli, Ferda ;
Imran, Muhammad .
INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 2018, 34 (04) :311-327
[10]  
Alam Firoj, 2017, P 2017 IEEEACM INT C, P1