Effective image tampering localization with multi-scale ConvNeXt feature fusion

被引:23
作者
Zhu, Haochen [1 ,2 ]
Cao, Gang [1 ,2 ]
Zhao, Mo [1 ]
Tian, Huawei [3 ]
Lin, Weiguo [1 ]
机构
[1] Commun Univ China, Sch Comp & Cyber Sci, Beijing 100024, Peoples R China
[2] Commun Univ China, State Key Lab Media Convergence & Commun, Beijing 100024, Peoples R China
[3] Peoples Publ Secur Univ China, Beijing 100038, Peoples R China
基金
中国国家自然科学基金;
关键词
Image forensics; Tampering localization; Encoder and decoder; ConvNeXt; Multi-scale feature fusion;
D O I
10.1016/j.jvcir.2023.103981
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
With the widespread use of powerful image editing tools, image tampering becomes easy and realistic. Existing image forensic methods still face challenges of low generalization performance and robustness. In this letter, we propose an effective image tampering localization scheme based on ConvNeXt encoder and multi-scale Feature Fusion (ConvNeXtFF). Stacked ConvNeXt blocks are utilized as an encoder to capture hierarchical multi-scale features, which are then fused in decoder for locating tampered pixels accurately. Combined loss function and effective data augmentation strategies are adopted to further improve the model performance. Extensive experimental results show that both localization accuracy and robustness of the ConvNeXtFF scheme outperform other state-of-the-art ones. The source code is available at https://github.com/multimediaFor/ConvNeXtFF.
引用
收藏
页数:7
相关论文
共 33 条
[1]   Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries [J].
Bappy, Jawadul H. ;
Simons, Cody ;
Nataraj, Lakshmanan ;
Manjunath, B. S. ;
Roy-Chowdhury, Amit K. .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019, 28 (07) :3286-3300
[2]   Exploiting Spatial Structure for Localizing Manipulated Image Regions [J].
Bappy, Jawadul H. ;
Roy-Chowdhury, Amit K. ;
Bunk, Jason ;
Nataraj, Lakshmanan ;
Manjunath, B. S. .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :4980-4989
[3]   The Lovasz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks [J].
Berman, Maxim ;
Triki, Amal Rannen ;
Blaschko, Matthew B. .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :4413-4421
[4]   Image Manipulation Detection by Multi-View Multi-Scale Supervision [J].
Chen, Xinru ;
Dong, Chengbo ;
Ji, Jiaqi ;
Cao, Juan ;
Li, Xirong .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :14165-14173
[5]   Noiseprint: A CNN-Based Camera Model Fingerprint [J].
Cozzolino, Davide ;
Verdoliva, Luisa .
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, 2020, 15 (01) :144-159
[6]   Exposing Digital Image Forgeries by Illumination Color Classification [J].
de Carvalho, Tiago Jose ;
Riess, Christian ;
Angelopoulou, Elli ;
Pedrini, Helio ;
Rocha, Anderson de Rezende .
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, 2013, 8 (07) :1182-1194
[7]  
Deng J, 2009, PROC CVPR IEEE, P248, DOI 10.1109/CVPRW.2009.5206848
[8]  
github, 2020, MMSegmentation: OpenMMLab Semantic Segmentation Toolbox and Benchmark
[9]  
Gloe T, 2010, J DIGIT FORENSIC PRA, P1584, DOI [10.1080/15567281.2010.531500, DOI 10.1080/15567281.2010.531500]
[10]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778