Counting piglet suckling events using deep learning-based action density estimation

Analysis of piglet suckling behaviour is important for the evaluation of piglet nutrient ingestion, health, welfare, and affinity with the sow. In this study, an action density estimation network (ADEN) was proposed for counting the events of piglet suckling followed by automated analysis of suckling behaviour. ADEN is a two-stream network primarily composed of 1) a network stream that processes video images with a higher frame rate (faster stream) and 2) a network stream that processes video images with a lower frame rate (slower stream). Each stream consists of a ResNet-50 with five convolutional stages. A multi-stage attention connection (MSAC), composed of four Spatial-Temporal-Channel (STC) multi-attention structures, is proposed to bridge Faster Stream and Slower Stream and capture discriminative features. The output attention features from each faster stream stage are laterally fused into the corresponding slower stream stage in a concatenating manner. Following this, the features from the last convolutional stages in the Slow stream and Fast stream are fused using concatenation and are then decoded by using three convolutional layers. The last convolutional layer outputs a heatmap on the action density of piglet suckling behaviour. Finally, the number of suckling events is predicted by integrating all the pixel values in the heatmap. Experimental and comparative tests were conducted to validate the effectiveness of the proposed ADEN with a training dataset and a test dataset from 14 pig pens. The 507 video clips (126,750 images for 7 h) from the 1-9th pens were selected as training datasets. The 143 video clips (35,750 images for 2 h) from the 10-13th pens were selected as short-term test datasets. One untrimmed video (162,000 images for 9 h) from the 14th pen was used to ultimately evaluate the action density estimation performance of the ADEN. ADEN was compared with seven approaches and its superiority was demonstrated with an r = 0.938, an RMSE = 1.080, and a MAE = 0.967 in short video clips and r = 0.982, MAE = 0.161, and RMSE = 0.563 in the untrimmed long video. The ADEN proved it feasible to predict the number of suckling events by using action density estimation.