Hierarchical Keypoints Feature Alignment for Domain Adaptive Pose Estimation

Unsupervised Domain Adaptation (UDA) is a popular research topic in computer vision. One of the most common strategies in this field is to reduce domain shifts through global feature alignment. Unfortunately, we observe its failure on adaptive human pose estimation. From our analysis, we find out two major reasons: the extreme imbalance between keypoints and non-keypoints regions and the high diversity of human skeleton structures. To address these problems, we propose a simple yet effective approach named PoseDA. Our idea is to let the alignment focus on the feature of keypoint regions and enforce the model to learn domain-invariant representations for keypoint prediction. The key component of our PoseDA is a Hierarchical Feature Selection of Keypoints Regions (HFS) module, which consists of Coarse Feature Selection of Keypoints Regions (CFS) and Reliable Feature Selection of Keypoints Regions (RFS). By using HFS, we obtain reliable and compact keypoints features, allowing our model to achieve more effective feature alignment. Under three UDA scenarios, i.e. JTA -> MPII, SURREAL -> LSP, SURREAL -> Human3.6, our PoseDA establishes new state-of-the-art performance. In particular, our PoseDA outperforms the previous best UDA methods by over 7% w.r.t. PCKh on JTA -> MPII.