Object-Aware Transfer-Based Black-Box Adversarial Attack on Object Detector

Deep neural networks have been demonstrated to be vulnerable to adversarial noise from attacks. Compared with white-box attacks, black-box attacks fool deep neural networks to yield erroneous predictions without knowing the model parameters. Black-box attacks include query-based attacks and transfer-based attacks; the former rely on querying the model while the latter just rely on the transferability of adversarial examples, thus challenging. Existing transfer-based black-box adversarial attack methods focus on the image classification task. Especially, we empirically verify that those methods struggle to balance the attack on objects with different classes and sizes, and thus they perform poorly in the attack on object detectors. In this work, we propose an Object-Aware mechanism to address this issue. It includes Object-Wise Gradient (OWG) calculation to balance the attack on multiple objects and a Domain-Division Map (DDM) to weigh the attack in size. Incorporating our method with seminal baselines (e.g., I-FGSM, MI-FGSM), we achieve superior attack performance on multiple object detectors (e.g., Faster R-CNN, DETR, SSD), which justifies the effectiveness and generality of our method.