Fighting Deepfake by Exposing the Convolutional Traces on Images

被引:37
|
作者
Guarnera, Luca [1 ,2 ]
Giudice, Oliver [1 ]
Battiato, Sebastiano [1 ,2 ]
机构
[1] Univ Catania, Dept Math & Comp Sci, I-95124 Catania, Italy
[2] iCTLab Srl, I-95124 Catania, Italy
关键词
Deepfake detection; generative adversarial networks; multimedia forensics; image forensics;
D O I
10.1109/ACCESS.2020.3023037
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Advances in Artificial Intelligence and Image Processing are changing the way people interacts with digital images and video. Widespread mobile apps like FACEAPP make use of the most advanced Generative Adversarial Networks (GAN) to produce extreme transformations on human face photos such gender swap, aging, etc. The results are utterly realistic and extremely easy to be exploited even for non-experienced users. This kind of media object took the name of Deepfake and raised a new challenge in the multimedia forensics field: the Deepfake detection challenge. Indeed, discriminating a Deepfake from a real image could be a difficult task even for human eyes but recent works are trying to apply the same technology used for generating images for discriminating them with preliminary good results but with many limitations: employed Convolutional Neural Networks are not so robust, demonstrate to be specific to the context and tend to extract semantics from images. In this paper, a new approach aimed to extract a Deepfake fingerprint from images is proposed. The method is based on the Expectation-Maximization algorithm trained to detect and extract a fingerprint that represents the Convolutional Traces (CT) left by GANs during image generation. The CT demonstrates to have high discriminative power achieving better results than state-of-the-art in the Deepfake detection task also proving to be robust to different attacks. Achieving an overall classification accuracy of over 98%, considering Deepfakes from 10 different GAN architectures not only involved in images of faces, the CT demonstrates to be reliable and without any dependence on image semantic. Finally, tests carried out on Deepfakes generated by FACEAPP achieving 93% of accuracy in the fake detection task, demonstrated the effectiveness of the proposed technique on a real-case scenario.
引用
收藏
页码:165085 / 165098
页数:14
相关论文
共 50 条
  • [31] Exposing deepfake using fusion of deep-learned and hand-crafted features
    Amr Megahed
    Qi Han
    Sondos Fadl
    Multimedia Tools and Applications, 2024, 83 : 26797 - 26817
  • [32] Watching the BiG artifacts: Exposing DeepFake videos via Bi-granularity artifacts
    Chen, Han
    Li, Yuezun
    Lin, Dongdong
    Li, Bin
    Wu, Junqiang
    PATTERN RECOGNITION, 2023, 135
  • [33] Exposing digital forgeries in scientific images
    Farid, Hany
    Proc. Multimedia Secur. Workshop, (29-36):
  • [34] Deepfake Histologic Images for Enhancing Digital Pathology
    Falahkheirkhah, Kianoush
    Tiwari, Saumya
    Yeh, Kevin
    Gupta, Sounak
    Herrera-Hernandez, Loren
    McCarthy, Michael R.
    Jimenez, Rafael E.
    Cheville, John C.
    Bhargava, Rohit
    LABORATORY INVESTIGATION, 2023, 103 (01)
  • [35] Exposing deepfake using fusion of deep-learned and hand-crafted features
    Megahed, Amr
    Han, Qi
    Fadl, Sondos
    MULTIMEDIA TOOLS AND APPLICATIONS, 2024, 83 (09) : 26797 - 26817
  • [36] Recurrent Convolutional Structures for Audio Spoof and Video Deepfake Detection
    Chintha, Akash
    Thai, Bao
    Sohrawardi, Saniat Javid
    Bhatt, Kartavya
    Hickerson, Andrea
    Wright, Matthew
    Ptucha, Raymond
    IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2020, 14 (05) : 1024 - 1037
  • [37] Speech Audio Deepfake Detection via Convolutional Neural Networks
    Valente, Lucas P.
    de Souza, Marcelo M. S.
    da Rocha, Alan M.
    IEEE CONFERENCE ON EVOLVING AND ADAPTIVE INTELLIGENT SYSTEMS 2024, IEEE EAIS 2024, 2024, : 382 - 387
  • [38] Detecting deepfake videos based on spatiotemporal attention and convolutional LSTM
    Chen, Beijing
    Li, Tianmu
    Ding, Weiping
    INFORMATION SCIENCES, 2022, 601 : 58 - 70
  • [39] Improved Deepfake Video Detection Using Convolutional Vision Transformer
    Deressa, Deressa Wodajo
    Lambert, Peter
    Van Wallendael, Glenn
    Atnafu, Solomon
    Mareen, Hannes
    2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024, 2024, : 492 - 497
  • [40] Detection Enhancement for Various Deepfake Types Based on Residual Noise and Manipulation Traces
    Kang, Jihyeon
    Ji, Sang-Keun
    Lee, Sangyeong
    Jang, Daehee
    Hou, Jong-Uk
    IEEE ACCESS, 2022, 10 : 69031 - 69040