UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Segmentation-Aware Convolutional Networks Using Local Attention Masks

Harley, AW; Derpanis, KG; Kokkinos, I; (2017) Segmentation-Aware Convolutional Networks Using Local Attention Masks. In: (Proceedings) 16th IEEE International Conference on Computer Vision (ICCV). (pp. pp. 5048-5057). IEEE Green open access

[img]
Preview
Text
1708.04607.pdf - Published version

Download (3MB) | Preview

Abstract

We introduce an approach to integrate segmentation information within a convolutional neural network (CNN). This counter-acts the tendency of CNNs to smooth information across regions and increases their spatial precision. To obtain segmentation information, we set up a CNN to provide an embedding space where region co-membership can be estimated based on Euclidean distance. We use these embeddings to compute a local attention mask relative to every neuron position. We incorporate such masks in CNNs and replace the convolution operation with a “segmentation-aware” variant that allows a neuron to selectively attend to inputs coming from its own region. We call the resulting network a segmentation-aware CNN because it adapts its filters at each image point according to local segmentation cues, while at the same time remaining fully-convolutional. We demonstrate the merit of our method on two widely different dense prediction tasks, that involve classification (semantic segmentation) and regression (optical flow). Our results show that in semantic segmentation we can replace DenseCRF inference with a cascade of segmentation-aware filters, and in optical flow we obtain clearly sharper responses than the ones obtained with comparable networks that do not use segmentation. In both cases segmentation-aware convolution yields systematic improvements over strong baselines.

Type: Proceedings paper
Title: Segmentation-Aware Convolutional Networks Using Local Attention Masks
Event: 16th IEEE International Conference on Computer Vision (ICCV)
Location: Venice, ITALY
Dates: 22 October 2017 - 29 October 2017
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/ICCV.2017.539
Publisher version: https://doi.org/10.1109/ICCV.2017.539
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Science & Technology, Technology, Computer Science, Artificial Intelligence, Engineering, Electrical & Electronic, Computer Science, Engineering
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10060976
Downloads since deposit
47Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item