TinyPillarNet: Tiny Pillar-Based Network for 3D Point Cloud Object Detection at Edge

被引:8
作者
Li, Yishi [1 ,2 ]
Zhang, Yuhao [1 ,2 ]
Lai, Rui [1 ,2 ]
机构
[1] Xidian Univ, Sch Microelect, Xian 710071, Peoples R China
[2] Xidian Univ, Chongqing Innovat Res Inst Integrated Circuits, Chongqing 400031, Peoples R China
关键词
3D object detection; point cloud; tiny machine learning (TinyML); FPGA; SYSTEM;
D O I
10.1109/TCSVT.2023.3297620
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Limited by huge computational cost, high inference latency and large memory consumption, existing 3D point cloud object detection methods are hard to be deployed on Internet of Things (IoT) edge devices. To handle this challenge, we present an extremely tiny framework termed TinyPillarNet. This framework leverages innovative pillar encoder to represent point cloud as immensely tiny pseudo-maps for extremely shrinking the input 3D sensing data. Moreover, a compact dual-stream feature extraction network is put forward to respectively extract intrinsic feature and distributional saliency map, which jointly boosts the detection precision with the lowest hardware cost. Extended experiments on KITTI benchmark demonstrated that our TinyPillarNet yields applicable precision with a record tiny weight size of 1.69 MB at a high inference speed of 1.67 times faster than the current record. Furthermore, the specially designed prototype verification system achieves a superior energy efficiency, which outperforms the similar deep learning based point cloud processing solutions on FPGA with a big margin.
引用
收藏
页码:1772 / 1785
页数:14
相关论文
共 71 条
[51]   An Ultra-Low Power TinyML System for Real-Time Visual Processing at Edge [J].
Xu, Kunran ;
Zhang, Huawei ;
Li, Yishi ;
Zhang, Yuhao ;
Lai, Rui ;
Liu, Yi .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, 2023, 70 (07) :2640-2644
[52]   SECOND: Sparsely Embedded Convolutional Detection [J].
Yan, Yan ;
Mao, Yuxing ;
Li, Bo .
SENSORS, 2018, 18 (10)
[53]  
Yang B, 2018, PR MACH LEARN RES, V87
[54]   PIXOR: Real-time 3D Object Detection from Point Clouds [J].
Yang, Bin ;
Luo, Wenjie ;
Urtasun, Raquel .
2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, :7652-7660
[55]   Nested Architecture Search for Point Cloud Semantic Segmentation [J].
Yang, Fan ;
Li, Xin ;
Shen, Jianbing .
IEEE TRANSACTIONS ON IMAGE PROCESSING, 2023, 32 :2889-2900
[56]   3DSSD: Point-based 3D Single Stage Object Detector [J].
Yang, Zetong ;
Sun, Yanan ;
Liu, Shu ;
Jia, Jiaya .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2020), 2020, :11037-11045
[57]   STD: Sparse-to-Dense 3D Object Detector for Point Cloud [J].
Yang, Zetong ;
Sun, Yanan ;
Liu, Shu ;
Shen, Xiaoyong ;
Jia, Jiaya .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :1951-1960
[58]   Semi-supervised 3D Object Detection with Proficient Teachers [J].
Yin, Junbo ;
Fang, Jin ;
Zhou, Dingfu ;
Zhang, Liangjun ;
Xu, Cheng-Zhong ;
Shen, Jianbing ;
Wang, Wenguan .
COMPUTER VISION, ECCV 2022, PT XXXVIII, 2022, 13698 :727-743
[59]   ProposalContrast: Unsupervised Pre-training for LiDAR-Based 3D Object Detection [J].
Yin, Junbo ;
Zhou, Dingfu ;
Zhang, Liangjun ;
Fang, Jin ;
Xu, Cheng-Zhong ;
Shen, Jianbing ;
Wang, Wenguan .
COMPUTER VISION, ECCV 2022, PT XXXIX, 2022, 13699 :17-33
[60]   Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection From Point Clouds [J].
Yin, Junbo ;
Shen, Jianbing ;
Gao, Xin ;
Crandall, David J. ;
Yang, Ruigang .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (08) :9822-9835