Spatial Attention-Based Capsule Networks With Guaranteed Group Equivariance

Some capsule networks (CapsNets) reported lately aim to enforce capsule poses and descriptors to be equivariant and invariant respectively by adding extra loss functions as regularization but without providing rigorous proof. To address this problem, a group equivariant spatial attention mechanism (GSA) is proposed to rigidly guarantee the equivariance with mathematical proof while enhancing the spatial information in capsule poses. In addition, to alleviate the computation burden associated with the conventional routing algorithm, group poolings are developed to generate the descriptors and poses of capsules, which contribute greatly to preserving the invariance and equivariance of CapsNets. With the proposed components of GSA and group poolings, a new attentive CapsNet, namely spatial attentive group equivariant CapsNets (SAGE-CapsNets), is constructed in this paper. To validate the invariance and equivariance of SAGE-CapsNets, we conduct experiments involving classification, semantic segmentation, and visualization. The results obtained from these experiments provide empirical evidence of the effectiveness of our proposed approach.