Toward Generic and Controllable Attacks Against Object Detection

Existing adversarial attacks against object detectors (ODs) have two inherent limitations. First, ODs have complex meta-structure designs, hence most advanced attacks for ODs concentrate on attacking specific detector-intrinsic structures [e.g., RPN and nonmaximal suppression (NMS)], which makes it hard for them to work on other new detectors. Second, most works against ODs make adversarial examples (AEs) by adding image-level perturbations into original images, which brings redundant perturbations in semantically meaningless areas (e.g., backgrounds). This article proposes a generic white-box attack on mainstream ODs with controllable perturbations. For a generic attack, LGP treats ODs as black boxes and only attacks their outputs, thereby eliminating the limitations of detector-intrinsic structures. Regarding controllability, we establish an object-wise constraint to induce the attachment of perturbations to foregrounds. Experimentally, the proposed LGP successfully attacked 16 state-of-the-art ODs on MS-COCO and DOTA datasets, with promising imperceptibility and transferability obtained. Code is publicly released in https://github.com/liguopeng0923/LGP.git.