Realtime Multispectral Pedestrian Detection With Visible and Far-Infrared Under Ambient Temperature Changing

被引:0
|
作者
Okuda, Masato [1 ]
Yoshida, Kota [2 ]
Fujino, Takeshi [2 ]
机构
[1] Ritsumeikan Univ, Grad Sch Sci & Engn, Kusatsu, Shiga 5258577, Japan
[2] Ritsumeikan Univ, Dept Sci & Engn, Kusatsu, Shiga 5258577, Japan
来源
IEEE OPEN JOURNAL OF INTELLIGENT TRANSPORTATION SYSTEMS | 2024年 / 5卷
关键词
Finite impulse response filters; Cameras; Pedestrians; Accuracy; Image edge detection; Feature extraction; Deep learning; YOLO; Training; Synchronization; Object detection; pedestrian detection; deep learning; far-infrared; sensor fusion; VEHICLE;
D O I
10.1109/OJITS.2024.3507917
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In recent intelligent transportation systems (ITS), it is important to recognize pedestrians and avoid collisions. Various sensors are used to detect pedestrians, and some research on pedestrian detection uses a visible light (RGB) camera and a far-infrared (FIR) camera. FIR cameras are significantly affected by ambient temperatures such as summer and winter. However, few studies have focused on this property when evaluating pedestrian detection accuracy. Therefore, this paper investigates the effect of temperature change in real-time multispectral pedestrian detection. We created an original dataset with three subsets, Hot, Intermediate, and Cold, and evaluated temperature effects by changing the subsets during training and testing. We first evaluated YOLOv8s-4ch, which simply extended the input layer of YOLOv8 from 3 channels of RGB to 4 channels of RGB-FIR. To further improve detection performance, we built a new model called YOLOv8s-2stream. This model has two backbones for RGB and FIR, and fuses their feature maps in each resolution. We found that the model trained on a specific temperature subset dropped the test accuracy in other subsets. On the other hand, when training using a Mix set covering all temperature sets (Hot, Inter., Cold), the model achieved the highest accuracy through all conditions. Moreover, our YOLOv8s-2stream has improved by 3.9 points of accuracy (AP@0.5:0.95) compared to YOLOv8s-4ch, and achieved 73 FPS inference speed on Jetson.
引用
收藏
页码:797 / 809
页数:13
相关论文
共 50 条
  • [21] Pedestrian detection in far infrared images
    Olmeda, Daniel
    Premebida, Cristiano
    Nunes, Urbano
    Maria Armingol, Jose
    de la Escalera, Arturo
    INTEGRATED COMPUTER-AIDED ENGINEERING, 2013, 20 (04) : 347 - 360
  • [22] Detection of extrasolar far-infrared lasers
    Colgan, SWJ
    Erickson, EF
    Haas, MR
    Hollenbach, DJ
    Smith, HA
    Strelnitski, VS
    SEARCH FOR EXTRATERRESTRIAL INTELLIGENCE (SETI) IN THE OPTICAL SPECTRUM II, 1996, 2704 : 52 - 52
  • [23] Quantum structures for far-infrared detection
    Perera, A.G.U.
    Matsik, S.G.
    International Journal of High Speed Electronics and Systems, 2002, 12 (03) : 821 - 872
  • [24] Progress in far-infrared detection technology
    Zhou, YD
    Becker, CR
    Ashokan, R
    Selamet, Y
    Chang, Y
    Boreiko, RT
    Betz, AL
    Sivananthan, S
    MATERIALS FOR INFRARED DETECTORS II, 2002, 4795 : 121 - 128
  • [25] DETECTION OF INTERSTELLAR CH IN THE FAR-INFRARED
    STACEY, GJ
    LUGTEN, JB
    GENZEL, R
    ASTROPHYSICAL JOURNAL, 1987, 313 (02): : 859 - 866
  • [26] NIOBIUM MICROBOLOMETERS FOR FAR-INFRARED DETECTION
    MACDONALD, ME
    GROSSMAN, EN
    IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES, 1995, 43 (04) : 893 - 896
  • [27] HgCdTe for far-infrared heterodyne detection
    Zhou, YD
    Zhao, J
    Boreiko, R
    Chang, Y
    Selamet, Y
    Ashokan, R
    Becker, CR
    Betz, A
    Sivananthan, S
    MATERIALS FOR INFRARED DETECTORS III, 2003, 5209 : 99 - 106
  • [28] FAR-INFRARED LMR DETECTION OF HYDROXYMETHYL
    RADFORD, HE
    EVENSON, KM
    JENNINGS, DA
    CHEMICAL PHYSICS LETTERS, 1981, 78 (03) : 589 - 591
  • [29] DETECTION OF FAR-INFRARED ASTRONOMICAL SOURCES
    FURNISS, I
    JENNINGS, RE
    MOORWOOD, AF
    ASTROPHYSICAL JOURNAL, 1972, 176 (03): : L105 - &
  • [30] Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System
    Tumas, Paulius
    Serackis, Arturas
    Nowosielski, Adam
    ELECTRONICS, 2021, 10 (08)