Causal Inference with Knowledge Distilling and Curriculum Learning for Unbiased VQA

被引:28
作者
Pan, Yonghua [1 ]
Li, Zechao [1 ]
Zhang, Liyan [2 ]
Tang, Jinhui [1 ]
机构
[1] Nanjing Univ Sci & Technol, 200 Xiaolingwei St, Nanjing 210094, Jiangsu, Peoples R China
[2] Nanjing Univ Aeronaut & Astronaut, 29 Yudao St, Nanjing 210016, Jiangsu, Peoples R China
基金
中国国家自然科学基金;
关键词
Visual question answering; neural networks; knowledge distillation; causal inference;
D O I
10.1145/3487042
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Recently, many Visual Question Answering (VQA) models rely on the correlations between questions and answers yet neglect those between the visual information and the textual information. They would perform badly if the handled data distribute differently from the training data (i.e., out-of-distribution (OOD) data). Towards this end, we propose a two-stage unbiased VQA approach that addresses the unbiased issue from a causal perspective. In the causal inference stage, we mark the spurious correlation on the causal graph, explore the counterfactual causality, and devise a causal target based on the inherent correlations between the conventional and counterfactual VQA models. In the distillation stage, we introduce the causal target into the training process and leverages distilling as well as curriculum learning to capture the unbiased model. Since Causal Inference with Knowledge Distilling and Curriculum Learning (CKCL) reinforces the contribution of the visual information and eliminates the impact of the spurious correlation by distilling the knowledge in causal inference to the VQA model, it contributes to the good performance on both the standard data and out-of-distribution data. The extensive experimental results on VQA-CP v2 dataset demonstrate the superior performance of the proposed method compared to the state-of-the-art (SotA) methods.
引用
收藏
页数:23
相关论文
共 50 条
[41]   Invariant Feature Learning Based on Causal Inference from Heterogeneous Environments [J].
Su, Hang ;
Wang, Wei .
MATHEMATICS, 2024, 12 (05)
[42]   Learning from each other: causal inference and American political development [J].
Jenkins, Jeffery A. ;
McCarty, Nolan ;
Stewart, Charles, III .
PUBLIC CHOICE, 2020, 185 (3-4) :245-251
[43]   Exploring the Use of Q-Learning in Causal Inference for Adaptive Interventions [J].
Zhou, Sha ;
Jiang, YanHua ;
Jin, ZhiWei ;
Qian, ZhenZhen ;
Ji, MengMeng ;
Liu, Chi ;
Li, HongYi ;
Xuan, GuoWei ;
Shuai, YuXing ;
Chen, XinLin .
CAUSAL INFERENCE, PCIC 2024, 2025, 2200 :86-94
[44]   Hyperparameter Tuning for Causal Inference with Double Machine Learning: A Simulation Study [J].
Bach, Philipp ;
Schacht, Oliver ;
Chernozhukov, Victor ;
Klaassen, Sven ;
Spindler, Martin .
CAUSAL LEARNING AND REASONING, VOL 236, 2024, 236 :1065-1117
[45]   Timed Process Interventions: Causal Inference vs. Reinforcement Learning [J].
Weytjens, Hans ;
Verbeke, Wouter ;
De Weerdt, Jochen .
BUSINESS PROCESS MANAGEMENT WORKSHOPS, BPM 2023, 2024, 492 :245-258
[46]   Causal inference and Bayesian network structure learning from nominal data [J].
Guiming Luo ;
Boxu Zhao ;
Shiyuan Du .
Applied Intelligence, 2019, 49 :253-264
[47]   A theoretical analysis based on causal inference and single-instance learning [J].
Chao Wang ;
Xuantao Lu ;
Wei Wang .
Applied Intelligence, 2022, 52 :13902-13915
[48]   Valuing Training Data via Causal Inference for In-Context Learning [J].
Zhou, Xiaoling ;
Ye, Wei ;
Lee, Zhemg ;
Zou, Lei ;
Zhang, Shikun .
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2025, 37 (06) :3824-3840
[49]   An efficient catalyst screening strategy combining machine learning and causal inference [J].
Song, Chenyu ;
Shi, Yintao ;
Li, Meng ;
Wu, Lin ;
Xiong, Xiaorong ;
Liu, Jianyun ;
Xia, Dongsheng .
JOURNAL OF ENVIRONMENTAL MANAGEMENT, 2025, 377
[50]   2nd Workshop on Causal Inference and Machine Learning in Practice [J].
Lee, Jeong-Yoon ;
Wu, Yifeng ;
Harinen, Totte ;
Pan, Jing ;
Lo, Paul ;
Zhao, Zhenyu ;
Chen, Huigang ;
Zheng, Zeyu ;
Vanchinathan, Hasta ;
Wang, Yingfei ;
Stevenson, Roland .
PROCEEDINGS OF THE 30TH ACM SIGKDD CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, KDD 2024, 2024, :6726-6726