SAE-RNN Deep Learning for RGB-D Based Object Recognition

RGB-D image is a multimodal data. Previous works have proved that using color and depth images together can dramatically increase the RGB-D based object recognition accuracy, but most of them either simply take all modalities as input, ignoring information about specific modalities, or train a first layer representation for each modality separately and concatenate them ignoring correlated modality information. In this paper, we use a variant of the sparse auto-encoder (SAE) which can specify how mode-sparse or mode-dense the features should be. A new deep learning network combining the variant SAE with the recursive neural networks (RNNs) was proposed. Through it, we got very discriminating features and obtained state of the art performance on a standard RGB-D object dataset.