Weakly-supervised cross-contrastive learning network for image manipulation detection and localization

With the significant reduction in the cost of image manipulation due to advancements in image editing tools, it is crucial to investigate methods for detecting image manipulation. Currently, mainstream methods are based on various types of deep learning models, which have achieved some success. However, these models largely rely on pixel-level ground truth annotations for supervision, leading to an increase in image-level false positives due to limited real images. Obtaining GT annotations is time-consuming and labor-intensive, and the supervised model has a high demand for tampering mask. To address these limitations, we propose a Weakly-Supervised CrossContrastive Learning (WSCCL) network that can detect and locate image manipulation based solely on imagelevel labels ('real'/'tampered'). Specifically, we first leverage a dual-stream encoder-decoder architecture to extract visual and noise features separately and generate corresponding prediction distribution maps. We then adopt an adaptive approach to fuse prediction distribution maps, obtaining weakly-supervised pseudo-label. We design the Cross-Contrastive Learning Module(CCLM) using different aggregation methods for different layer features in the encoder, and apply cross-contrastive learning for the fusion features and the predicted features maps generated by the decoder. Finally, WSCCL compares the similarity between the reconstructed image obtained from the decoder and the predicted distribution map to make the pseudo-label closer to the real GT. Furthermore, extensive experiments confirm that our approach based on weakly supervised learning is comparable to supervised learning, both at the image-level and pixel-level. WSCCL exhibits strong adaptability to various types of manipulation and high resistance to attacks. This study demonstrates that our weakly supervised learning method can compete fully with supervised learning, regardless of the level of manipulation or annotation.