A multimodal classification method: Cow behavior pattern classification with improved EdgeNeXt using an inertial measurement unit

In this study, a dynamic cow behavior pattern classification model using an improved EdgeNeXt network with inertial measurement unit (IMU) multimodal data was developed. This model was trained using three modalities of the IMU data: acceleration, angular velocity, and magnetic field. To augment the spatial connection among the three modalities, the nine-axis IMU data were extracted into matrices and then transformed into images. The improved EdgeNeXt model was trained to classify cow behavior patterns using two batch sizes: 128 and 256. Two EdgeNeXt models and the Swin Transformer, MobileNetV2, and ConvNeXt models were also trained for comparison. The results of the improved EdgeNeXt classification model were superior to those of the other five algorithms. The best overall classification accuracy of the improved EdgeNeXt model was 95.85 % with a batch size of 256. In this model, the classification accuracy of particular behavior patterns was 95.2 % (feeding), 95.6 % (lying), 96.8 % (ruminating-lying), 97.6 % (rub scratching (legs)), 95.5 % (social licking), and 94.4 % (rub scratching (neck)). The limitation of this study is that the EdgeNeXt model relies more on global feature extraction, resulting in some misclassifications between behavior patterns and similar movements. The continuous operation and data transformation of the IMU limit the battery life. In the future, the lightweight EdgeNeXt cow behavior classification model will be applied to edge computing devices in livestock farms to improve computational efficiency. Moreover, multimodal data, such as video and acoustics, will be added to train the cow behavior pattern classification model to extend the scope of livestock monitoring.