RepMobile: A MobileNet-Like Network With Structural Reparameterization for Sensor-Based Human Activity Recognition

How to design and deploy efficient neural network backbones always plays a crucial role in sensor-based human activity recognition (HAR) on mobile devices, which have targeted at reducing floating-point operations (FLOPs) and parameter count, hence offering a faster activity inference. However, since the most commonly used efficiency indicators such as FLOPs and parameter count are indirect metrics for activity inference, they could not directly translate into practical inference latency in real-world. This article is the first attempt to mitigate such discrepancy between theoretic computational efficiency and practical inference latency. Our main goal is to identify and analyze key performance bottlenecks including activations and branching, which might potentially incur higher inference latency for on-device activity inference due to the degree of parallelism and memory access (MAC) cost. To this end, we provide an in-depth analysis about how the indirect metrics and different structural design choices correlate with practical inference latency, which has been rarely explored in ubiquitous HAR scenario. In particular, based on the above analyses, motivated by an idea of structural reparameterization, we present an efficient architecture called RepMobile built on MobileNet by decoupling training-time and inference-time architectures, where a linearly overparameterized model is used to improve accuracy at training time, and a reparameterization is performed for faster activity inference. We validate the effectiveness and efficiency of our proposed method on several public HAR benchmarks, including PAMAP2, UNIMIB-SHAR, WISDM, and USC-HAD, as well as a resource-restrained mobile device, indicating a better tradeoff between accuracy and on-device latency.