Skip to content

Latest commit

 

History

History

nonlocal_net

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

NonLocal Net

Non-local Neural Networks

Introduction

Official Repo

Code Snippet

Abstract

Both convolutional and recurrent operations are building blocks that process one local neighborhood at a time. In this paper, we present non-local operations as a generic family of building blocks for capturing long-range dependencies. Inspired by the classical non-local means method in computer vision, our non-local operation computes the response at a position as a weighted sum of the features at all positions. This building block can be plugged into many computer vision architectures. On the task of video classification, even without any bells and whistles, our non-local models can compete or outperform current competition winners on both Kinetics and Charades datasets. In static image recognition, our non-local models improve object detection/segmentation and pose estimation on the COCO suite of tasks. Code is available at this https URL.

Results and models

Cityscapes

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
NonLocalNet R-50-D8 512x1024 40000 7.4 2.72 V100 78.24 - config model | log
NonLocalNet R-101-D8 512x1024 40000 10.9 1.95 V100 78.66 - config model | log
NonLocalNet R-50-D8 769x769 40000 8.9 1.52 V100 78.33 79.92 config model | log
NonLocalNet R-101-D8 769x769 40000 12.8 1.05 V100 78.57 80.29 config model | log
NonLocalNet R-50-D8 512x1024 80000 - - V100 78.01 - config model | log
NonLocalNet R-101-D8 512x1024 80000 - - V100 78.93 - config model | log
NonLocalNet R-50-D8 769x769 80000 - - V100 79.05 80.68 config model | log
NonLocalNet R-101-D8 769x769 80000 - - V100 79.40 80.85 config model | log

ADE20K

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
NonLocalNet R-50-D8 512x512 80000 9.1 21.37 V100 40.75 42.05 config model | log
NonLocalNet R-101-D8 512x512 80000 12.6 13.97 V100 42.90 44.27 config model | log
NonLocalNet R-50-D8 512x512 160000 - - V100 42.03 43.04 config model | log
NonLocalNet R-101-D8 512x512 160000 - - V100 44.63 45.79 config model | log

Pascal VOC 2012 + Aug

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) Device mIoU mIoU(ms+flip) config download
NonLocalNet R-50-D8 512x512 20000 6.4 21.21 V100 76.20 77.12 config model | log
NonLocalNet R-101-D8 512x512 20000 9.8 14.01 V100 78.15 78.86 config model | log
NonLocalNet R-50-D8 512x512 40000 - - V100 76.65 77.47 config model | log
NonLocalNet R-101-D8 512x512 40000 - - V100 78.27 79.12 config model | log

Citation

@inproceedings{wang2018non,
  title={Non-local neural networks},
  author={Wang, Xiaolong and Girshick, Ross and Gupta, Abhinav and He, Kaiming},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={7794--7803},
  year={2018}
}