Hardware-Compliant Compressive Image Sensor Architecture Based on Random Modulations and Permutations for Embedded Inference

This work presents a compact CMOS Image Sensor (CIS) architecture enabling embedded object recognition facilitated by a dedicated end-of-column Compressive Sensing (CS), reducing on-chip memory needs. Our sensing scheme is based on a combination of random modulations and permutations leading to an implementation with very limited hardware impacts. It is designed to meet both theoretical (i.e., stable embedding, measurements incoherence) and practical requirements (i.e., silicon footprint, power consumption). The only additional hardware compared to a standard CIS architecture using first order incremental Sigma-Delta ( Sigma Delta) Analog to Digital Converter (ADC) are a pseudo-random data mixing circuit, an in-Sigma Delta +/- 1 modulator and a small Digital Signal Processor (DSP). On the algorithmic side, three variants are presented to perform the inference on compressed measurements with a tunable complexity (i.e., one-vs.-all SVM, hierarchical SVM and small ANN with 1-D maxpooling). An object recognition accuracy of similar or equal to 98.8% is reached on the COIL database (COIL, 100 classes) using our dedicated Neural Network classifier. We stress that the signal-independent dimensionality reduction performed by our dedicated CS scheme (1/480 in 480x640 VGA resolution case) allows to dramatically reduce memory requirements mainly related to the remotely learned coefficients used for the inference stage.