Convolutional Neural Network-Based Automated System for Dog Tracking and Emotion Recognition in Video Surveillance

被引:4
作者
Chen, Huan-Yu [1 ]
Lin, Chuen-Horng [1 ]
Lai, Jyun-Wei [1 ]
Chan, Yung-Kuan [2 ]
机构
[1] Natl Taichung Univ Sci & Technol, Dept Comp Sci & Informat Engn, 129,Sec 3,Sanmin Rd, Taichung 404, Taiwan
[2] Natl Chung Hsing Univ, Dept Management Informat Syst, 145 Xingda Rd, Taichung 402, Taiwan
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 07期
关键词
convolutional neural networks; dog detection; dog tracking; dog emotion recognition; long short-term memory;
D O I
10.3390/app13074596
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
This paper proposes a multi-convolutional neural network (CNN)-based system for the detection, tracking, and recognition of the emotions of dogs in surveillance videos. This system detects dogs in each frame of a video, tracks the dogs in the video, and recognizes the dogs' emotions. The system uses a YOLOv3 model for dog detection. The dogs are tracked in real time with a deep association metric model (DeepDogTrack), which uses a Kalman filter combined with a CNN for processing. Thereafter, the dogs' emotional behaviors are categorized into three types-angry (or aggressive), happy (or excited), and neutral (or general) behaviors-on the basis of manual judgments made by veterinary experts and custom dog breeders. The system extracts sub-images from videos of dogs, determines whether the images are sufficient to recognize the dogs' emotions, and uses the long short-term deep features of dog memory networks model (LDFDMN) to identify the dog's emotions. The dog detection experiments were conducted using two image datasets to verify the model's effectiveness, and the detection accuracy rates were 97.59% and 94.62%, respectively. Detection errors occurred when the dog's facial features were obscured, when the dog was of a special breed, when the dog's body was covered, or when the dog region was incomplete. The dog-tracking experiments were conducted using three video datasets, each containing one or more dogs. The highest tracking accuracy rate (93.02%) was achieved when only one dog was in the video, and the highest tracking rate achieved for a video containing multiple dogs was 86.45%. Tracking errors occurred when the region covered by a dog's body increased as the dog entered or left the screen, resulting in tracking loss. The dog emotion recognition experiments were conducted using two video datasets. The emotion recognition accuracy rates were 81.73% and 76.02%, respectively. Recognition errors occurred when the background of the image was removed, resulting in the dog region being unclear and the incorrect emotion being recognized. Of the three emotions, anger was the most prominently represented; therefore, the recognition rates for angry emotions were higher than those for happy or neutral emotions. Emotion recognition errors occurred when the dog's movements were too subtle or too fast, the image was blurred, the shooting angle was suboptimal, or the video resolution was too low. Nevertheless, the current experiments revealed that the proposed system can correctly recognize the emotions of dogs in videos. The accuracy of the proposed system can be dramatically increased by using more images and videos for training the detection, tracking, and emotional recognition models. The system can then be applied in real-world situations to assist in the early identification of dogs that may exhibit aggressive behavior.
引用
收藏
页数:29
相关论文
共 58 条
[1]   A Framework for Studying Emotions across Species [J].
Anderson, David J. ;
Adolphs, Ralph .
CELL, 2014, 157 (01) :187-200
[2]  
[Anonymous], 1998, 4 IEEE WORKSH APPL C
[3]  
Bazzani L, 2012, PROC CVPR IEEE, P1886, DOI 10.1109/CVPR.2012.6247888
[4]  
Bewley A, 2016, IEEE IMAGE PROC, P3464, DOI 10.1109/ICIP.2016.7533003
[5]  
Boneh-Shitrit T., 2022, DEEP LEARNING MODELS
[6]   Emotions in goats: mapping physiological, behavioural and vocal profiles [J].
Briefer, Elodie F. ;
Tettamanti, Federico ;
McElligott, Alan G. .
ANIMAL BEHAVIOUR, 2015, 99 :131-143
[7]   Going Deeper than Tracking: A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions [J].
Broome, Sofia ;
Feighelstein, Marcelo ;
Zamansky, Anna ;
Lencioni, Gabriel Carreira ;
Andersen, Pia Haubro ;
Pessanha, Francisca ;
Mahmoud, Marwa ;
Kjellstrom, Hedvig ;
Salah, Albert Ali .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2023, 131 (02) :572-590
[8]   Mean shift: A robust approach toward feature space analysis [J].
Comaniciu, D ;
Meer, P .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2002, 24 (05) :603-619
[9]  
Coren Stanley., MODERNDOG
[10]   Histograms of oriented gradients for human detection [J].
Dalal, N ;
Triggs, B .
2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 1, PROCEEDINGS, 2005, :886-893