实例分割和全景分割

基本概念

  1. 语义分割：每个像素所属类别（包括stuff和object类）
  2. 实例分割：每个object类别和mask（只有object）
  3. 全景分割：每个像素所属类别和object类别的实例ID

Mask R-CNN

  1. Mask R-CNN结构
    img

  2. RPN网络（Region Proposal Network）:

    1. 对backbone提取出的feature map去预测哪些地方可能存在对象 ，通常使用9个先验框进行判断，然后通过平移来调整先验框
      img
    2. Feature Pyramid Network(特征金字塔)
      img
    3. Head Architecture
      img

全景分割：

UPSnet网络

img

语义分割头（Semantic segmentation head）

img
采用的是可变形卷积，对feature map进行下采样，再经过一个上采样将所有的feature map上采样到同一个维度然后进行堆叠融合，再经过一乘一卷积获得语义分割信息。

全景分割头(panoptic segmentation head)

image-20220407093333107

Panoptic-DeepLab

image-20220407094841043

实例分割和全景分割论文推荐：

实例分割（lnstance Segmentation）:

Dai et al., Instance-aware semantic segmentation via multi-task network cascades, In CVPR 2016
Li et al., Fully convolutional instance-aware semantic segmentation, In CVPR 2017
He et al., Mask R-CNN,In lCCV 2019
Chen et al., Masklab: Instance segmentation by refining object detection with semantic and direction features,IncVPR 2018
Chen et al.,Tensormask: A foundation for dense object segmentation, In CVPR 2019Bolya et al.，YOLACT: Real-time instance segmentation, In CVPR 2019
Xie et al,PolarMask: Single shot instance segmentation with polar representation, In CVPR 2020

全景分割(Panoptic Segmentation):

Li et al.,Attention-guided unified network for panoptic segmentation. In CVPR,2019
Kirillov et al.,Panoptic feature pyramid networks. In ’ CVPR,2019
Sofiiuk et al.,Adaptive instance selection network. In lCCV,2019
Xiong et al., Upsnet: A unified panoptic segmentation network. In CVPR, 2019
Yang et al., Deeperlab: Single-shot image parser, In arXiv 2019
Li et al.,Learning to fuse things and stuff, In arXiv 2018
Porzi et al., Seamless scene segmentation.In CVPR,2019
Gao et al.,Ssap: Single-shot instance segmentation with affinity pyramid. In lCCV, 2019