Skip to content

Latest commit

 

History

History
 
 

deeplabv3plus

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

DeepLabV3+

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Introduction

Official Repo

Code Snippet

Abstract

Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple effective fields-of-view, while the latter networks can capture sharper object boundaries by gradually recovering the spatial information. In this work, we propose to combine the advantages from both methods. Specifically, our proposed model, DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries. We further explore the Xception model and apply the depthwise separable convolution to both Atrous Spatial Pyramid Pooling and decoder modules, resulting in a faster and stronger encoder-decoder network. We demonstrate the effectiveness of the proposed model on PASCAL VOC 2012 and Cityscapes datasets, achieving the test set performance of 89.0% and 82.1% without any post-processing. Our paper is accompanied with a publicly available reference implementation of the proposed models in Tensorflow at this https URL.

Results and models

Cityscapes

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-50-D8 512x1024 40000 7.5 3.94 V100 79.61 81.01 config model | log
DeepLabV3+ R-101-D8 512x1024 40000 11 2.60 V100 80.21 81.82 config model | log
DeepLabV3+ R-50-D8 769x769 40000 8.5 1.72 V100 78.97 80.46 config model | log
DeepLabV3+ R-101-D8 769x769 40000 12.5 1.15 V100 79.46 80.50 config model | log
DeepLabV3+ R-18-D8 512x1024 80000 2.2 14.27 V100 76.89 78.76 config model | log
DeepLabV3+ R-50-D8 512x1024 80000 - - V100 80.09 81.13 config model | log
DeepLabV3+ R-101-D8 512x1024 80000 - - V100 80.97 82.03 config model | log
DeepLabV3+ (FP16) R-101-D8 512x1024 80000 6.35 7.87 V100 80.46 - config model | log
DeepLabV3+ R-18-D8 769x769 80000 2.5 5.74 V100 76.26 77.91 config model | log
DeepLabV3+ R-50-D8 769x769 80000 - - V100 79.83 81.48 config model | log
DeepLabV3+ R-101-D8 769x769 80000 - - V100 80.65 81.47 config[1] model | log
DeepLabV3+ R-101-D16-MG124 512x1024 40000 5.8 7.48 V100 79.09 80.36 config model | log
DeepLabV3+ R-101-D16-MG124 512x1024 80000 9.9 - V100 79.90 81.33 config model | log
DeepLabV3+ R-18b-D8 512x1024 80000 2.1 14.95 V100 75.87 77.52 config model | log
DeepLabV3+ R-50b-D8 512x1024 80000 7.4 3.94 V100 80.28 81.44 config model | log
DeepLabV3+ R-101b-D8 512x1024 80000 10.9 2.60 V100 80.16 81.41 config model | log
DeepLabV3+ R-18b-D8 769x769 80000 2.4 5.96 V100 76.36 78.24 config model | log
DeepLabV3+ R-50b-D8 769x769 80000 8.4 1.72 V100 79.41 80.56 config model | log
DeepLabV3+ R-101b-D8 769x769 80000 12.3 1.10 V100 79.88 81.46 config model | log

[1] The training of the model is sensitive to random seed, and the seed to train it is 1111.

ADE20K

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-50-D8 512x512 80000 10.6 21.01 V100 42.72 43.75 config model | log
DeepLabV3+ R-101-D8 512x512 80000 14.1 14.16 V100 44.60 46.06 config model | log
DeepLabV3+ R-50-D8 512x512 160000 - - V100 43.95 44.93 config model | log
DeepLabV3+ R-101-D8 512x512 160000 - - V100 45.47 46.35 config model | log

Pascal VOC 2012 + Aug

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-50-D8 512x512 20000 7.6 21 V100 75.93 77.50 config model | log
DeepLabV3+ R-101-D8 512x512 20000 11 13.88 V100 77.22 78.59 config model | log
DeepLabV3+ R-50-D8 512x512 40000 - - V100 76.81 77.57 config model | log
DeepLabV3+ R-101-D8 512x512 40000 - - V100 78.62 79.53 config model | log

Pascal Context

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-101-D8 480x480 40000 - 9.09 V100 47.30 48.47 config model | log
DeepLabV3+ R-101-D8 480x480 80000 - - V100 47.23 48.26 config model | log

Pascal Context 59

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-101-D8 480x480 40000 - - V100 52.86 54.54 config model | log
DeepLabV3+ R-101-D8 480x480 80000 - - V100 53.2 54.67 config model | log

LoveDA

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-18-D8 512x512 80000 1.93 25.57 V100 50.28 50.47 config model | log
DeepLabV3+ R-50-D8 512x512 80000 7.37 6.00 V100 50.99 50.65 config model | log
DeepLabV3+ R-101-D8 512x512 80000 10.84 4.33 V100 51.47 51.32 config model | log

Potsdam

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-18-D8 512x512 80000 1.91 81.68 V100 77.09 78.44 config model | log
DeepLabV3+ R-50-D8 512x512 80000 7.36 26.44 V100 78.33 79.27 config model | log
DeepLabV3+ R-101-D8 512x512 80000 10.83 17.56 V100 78.7 79.47 config model | log

Vaihingen

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-18-D8 512x512 80000 1.91 72.79 V100 72.50 74.13 config model | log
DeepLabV3+ R-50-D8 512x512 80000 7.36 26.91 V100 73.97 75.05 config model | log
DeepLabV3+ R-101-D8 512x512 80000 10.83 18.59 V100 73.06 74.14 config model | log

iSAID

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-18-D8 896x896 80000 6.19 24.81 V100 61.35 62.61 config model | log
DeepLabV3+ R-50-D8 896x896 80000 21.45 8.42 V100 67.06 68.02 config model | log

Mapillary Vistas v1.2

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
DeepLabV3+ R-50-D8 1280x1280 300000 24.04 17.92 A100 47.35 - config model | log

Note:

  • D-8/D-16 here corresponding to the output stride 8/16 setting for DeepLab series.
  • MG-124 stands for multi-grid dilation in the last stage of ResNet.
  • FP16 means Mixed Precision (FP16) is adopted in training.
  • 896x896 is the Crop Size of iSAID dataset, which is followed by the implementation of PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation

Citation

@inproceedings{deeplabv3plus2018,
  title={Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation},
  author={Liang-Chieh Chen and Yukun Zhu and George Papandreou and Florian Schroff and Hartwig Adam},
  booktitle={ECCV},
  year={2018}
}