In-Sensor Visual Perception and Inference

被引:4
作者
Liu, Yanan [1 ]
Fan, Rui [2 ,3 ]
Guo, Jianglong [4 ]
Ni, Hepeng [5 ]
Bhutta, Muhammad Usman Maqboo [6 ]
机构
[1] Shanghai Univ, Sch Microelect, Shanghai, Peoples R China
[2] Tongji Univ, Coll Elect & Informat Engn, State Key Lab Intelligent Autonomous Syst, Shanghai, Peoples R China
[3] Tongji Univ, Frontiers Sci Ctr Intelligent Autonomous Syst, Shanghai, Peoples R China
[4] Harbin Inst Technol, Sch Sci, Shenzhen, Peoples R China
[5] Shandong Jianzhu Univ, Sch Mech & Elect Engn, Jinan, Peoples R China
[6] Chinese Univ Hong Kong, Robot Lab, Hong Kong, Peoples R China
来源
INTELLIGENT COMPUTING | 2023年 / 2卷
基金
中国国家自然科学基金;
关键词
SIMD PROCESSOR ARRAY; MACHINE; VISION;
D O I
10.34133/icomputing.0043
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Conventional machine vision systems have separate perception, memory, and processing architectures, which may exacerbate the increasing need for ultrahigh image processing rates and ultralow power consumption. In contrast, in-sensor visual computing performs signal processing at the pixel level using the collected analog signals directly, without sending data to other processors. Therefore, the in-sensor computing paradigm may hold the key to realizing extremely efficient and low power visual signal processing by integrating sensing, storage, and computation onto focal planes using either novel circuit designs or new materials. The focal-plane sensor-processor (FPSP), which is a typical in-sensor visual computing device, is a vision chip that has been developed for nearly 2 decades in domains such as image processing, computer vision, robotics, and neural networks. In contrast to conventional computer vision systems, the FPSP gives vision systems in-sensor image processing capabilities, thus decreasing system complexity, reducing power consumption, and enhancing information processing efficiency and security. Although many studies on in-sensor computing using the FPSP have been conducted since its invention, no thorough and systematic summary of these studies exists. This review explains the use of image processing algorithms, neural networks, and applications of in-sensor computing in the fields of machine vision and robotics. The objective is to assist future developers, researchers, and users of unconventional visual sensors in understanding in-sensor computing and associated applications.
引用
收藏
页数:16
相关论文
共 105 条
[1]  
Aistorm, Ai in sensor
[2]  
Alonso-Montes C, 2005, 2005 9 INT WORKSH CE
[3]  
[Anonymous], 2011, Focal-plane sensor-processor chips
[4]  
Barr David R. W., 2012, INT WORKSHOP CELLUL
[5]  
Bose L, 2020, EUR C COMP VIS
[6]   Pixel Processor Arrays For Low Latency Gaze Estimation [J].
Bose, Laurie ;
Chen, Jianing ;
Carey, Stephen J. ;
Dudek, Piotr .
2022 IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES ABSTRACTS AND WORKSHOPS (VRW 2022), 2022, :970-971
[7]   A Camera That CNNs: Towards Embedded Neural Networks on Pixel Processor Arrays [J].
Bose, Laurie ;
Chen, Jianing ;
Carey, Stephen J. ;
Dudek, Piotr ;
Mayol-Cuevas, Walterio .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :1335-1344
[8]   Live Demonstration: Digit Recognition On Pixel Processor Arrays [J].
Bose, Laurie ;
Chen, Jianing ;
Carey, Stephen J. ;
Dudek, Piotr ;
Mayol-Cuevas, Walterio .
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW 2019), 2019, :1705-1706
[9]   Visual Odometry for Pixel Processor Arrays [J].
Bose, Laurie ;
Chen, Jianing ;
Carey, Stephen J. ;
Dudek, Piotr ;
Mayol-Cuevas, Walterio .
2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, :4614-4622
[10]  
Carey SJ, 2014, INT WORK CELL NANO