QESAR: Query Effective Decision-Based Attack on Skeletal Action Recognition

Generating high-quality adversarial examples in a black-box setting often requires larger query volumes and adds more noticeable perturbations in skeleton action recognition tasks. We propose a query effective decision-based attack on skeletal action recognition (QESAR) to address this. We use the current gradient direction and successful historical information to reduce query volumes to update the sampling distribution, enabling a quick estimation of the next gradient direction. To ensure the imperceptibility of adversarial examples, we propose a hierarchical joint perturbation method to estimate the direction of perturbation accurately. Additionally, we design an objective function that satisfies joint angle and bone length constraints to minimize the magnitude of perturbation, ensuring that the generated adversarial examples do not exhibit noticeable distortions. Finally, in experiments, we find that QESAR can generate adversarial examples that satisfy skeletal action constraints with lower query volume. On the HDM05 and NTU datasets targeted at the ST-GCN and SGN models, QESAR achieves a 100% attack success rate while reducing the query volume by hundreds to thousands. Specifically, in the untargeted attack scenario, the QESAR scheme outperforms in terms of four metrics on the HDM05 dataset against the SGN model: average joint position deviation, average joint position acceleration deviation, average joint angle deviation, reducing them by 0.0078, 0.0679, and 6.9311, respectively.