Inferring Climate Change Stances from Multimodal Tweets

被引:0
作者
Bai, Nan [1 ]
Torres, Ricardo da Silva [1 ]
Fensel, Anna [1 ]
Metze, Tamara [2 ]
Dewulf, Art [1 ]
机构
[1] Wageningen Univ & Res, Wageningen, Netherlands
[2] Delft Univ Technol, Delft, Netherlands
来源
PROCEEDINGS OF THE 47TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, SIGIR 2024 | 2024年
关键词
Multimodal Embeddings; Transfer Learning; User-Generated Content; Climate Change Claims; Sea-Level Rise; Public Policy;
D O I
10.1145/3626772.3657950
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Climate change is a heated discussion topic in public arenas such as social media. Both texts and visuals play key roles in the debate, as they can complement, contradict, or reinforce each other in nuanced ways. It is therefore urgently needed to study the messages as multimodal objects to better understand the polarized debate about climate change impacts and policies. Multimodal representation models such as CLIP are known to be able to transfer knowledge across domains and modalities, enabling the investigation of textual and visual semantics together. Yet they are not directly able to distinguish the nuances between supporting and sceptic climate change stances. This paper explores a simple but effective strategy combining modality fusion and domain-knowledge enhancing to prepare CLIP-based models with knowledge of climate change stances. A multimodal Dutch Twitter dataset is collected and experimented with the proposed strategy, which increased the macro-average F1 score across stances from 51% to 86%. The outcomes can be applied in both data science and public policy studies, to better analyse how the combined use of texts and visuals generates meanings during debates, in the context of climate change and beyond.
引用
收藏
页码:2467 / 2471
页数:5
相关论文
共 45 条
[1]  
[Anonymous], 2021, PMLR
[2]   Heri-Graphs: A Dataset Creation Framework for Multi-Modal Machine Learning on Graphs of Heritage Values and Attributes with Social Media [J].
Bai, Nan ;
Nourian, Pirouz ;
Luo, Renqian ;
Roders, Ana Pereira .
ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2022, 11 (09)
[3]   Multimodal Machine Learning: A Survey and Taxonomy [J].
Baltrusaitis, Tadas ;
Ahuja, Chaitanya ;
Morency, Louis-Philippe .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2019, 41 (02) :423-443
[4]  
Bhatia S, 2021, 2021 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES (NAACL-HLT 2021), P2167
[5]  
Card Dallas, 2015, 53 ANN M ASS COMP LI, V2, P438
[6]  
Chen J., 2023, arXiv
[7]   Finding local community structure in networks [J].
Clauset, A .
PHYSICAL REVIEW E, 2005, 72 (02)
[8]   Computer-assisted classification of contrarian claims about climate change [J].
Coan, Travis G. ;
Boussalis, Constantine ;
Cook, John ;
Nanko, Mirjam O. .
SCIENTIFIC REPORTS, 2021, 11 (01)
[9]   Topic modeling and sentiment analysis of global climate change tweets [J].
Dahal, Biraj ;
Kumar, Sathish A. P. ;
Li, Zhenlong .
SOCIAL NETWORK ANALYSIS AND MINING, 2019, 9 (01)
[10]   Issue Framing in Conversations for Change: Discursive Interaction Strategies for "Doing Differences" [J].
Dewulf, Art ;
Bouwen, Rene .
JOURNAL OF APPLIED BEHAVIORAL SCIENCE, 2012, 48 (02) :168-193