LHRNet: Lateral hierarchically refining network for salient object detection

Deep convolutional neural networks (CNNs) have shown outstanding performance in salient object detection. However, there exist two conundrums under-explored. 1) High-level features are beneficial to locate salient objects while low-level features contain fine-grained details. How to combine these two types of features to promote accuracy is the first conundrum. 2) Previous CNN-based methods adopt a convolutional layer after extracting features to infer saliency maps. While encountering images that are different greatly from training dataset, adopting a convolutional layer as a classifier is not robust enough to detect all salient objects. In addition, limited receptive field and lack of spatial correlation will cause salient objects to be incomplete while blurring their boundaries. In this paper, a Lateral Hierarchically Refining Network (LHRNet) is put forward for accurate salient object detection. Firstly, LHRNet efficiently integrates multi-level features, which simultaneously incorporates coarse semantics and fine details. Then a coarse saliency prediction is made from low-resolution features by convolution. Finally, a series of nearest neighbor classifiers are learned to hierarchically restore the missing parts of salient objects while refining their boundaries, yielding a more reliable final prediction. Comprehensive experiments demonstrate that this network performs favorably against state-of-the-art approaches on six datasets.