Attention-based adaptive context network for anchor-free instance segmentation

It is crucial to obtain accurate and efficient instance segmentation masks in many modern applications such as automatic pilot and robotic manipulation. In this paper, we propose a straightforward and flexible two-stage framework for instance segmentation, which simultaneously generates box-level localization information in an image and instance-level segmentation information for each instance. We name this framework as Attention-based Adaptive Context Network for anchor-free Instance Segmentation (ContextMask), which extends the object detector FCOS (Fully Convolutional One-stage Object Detection) by adding a novel multi-scale adaptive context-guided mask (MACG-Mask) branch containing an adaptive context network and a MaskIoU branch. The adaptive context network is to combine the global context in predicted bounding boxes and the MaskIoU branch is to evaluate the quality of the predicted masks. With the development of deep convolutional neural networks, the network continues to deepen so that it is difficult to balance spatial information and semantic information well. To address the issue, we design a weighted FPN, which obtains feature maps with balance-well spatial and semantic information by concatenating and weighting feature maps of different resolutions. Besides, we also propose an attention-based head, which adds spatial attention and channel attention module to make each pixel have a unique weight to solve the problem of large-scale variant of objects. We verify ContextMask’s effectiveness on the fine-annotations Cityscapes and COCO dataset. ContextMask outperforms state-of-the-art methods and achieves 38.4%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$38.4\%$$\end{document}AP on the Cityscapes dataset and 39.0%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document}AP on the COCO dataset.