UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Efficient Semantic Segmentation via Self-Attention and Self-Distillation

An, S; Liao, Q; Lu, Z; Xue, J-H; (2022) Efficient Semantic Segmentation via Self-Attention and Self-Distillation. IEEE Transactions on Intelligent Transportation Systems 10.1109/tits.2021.3139001. (In press). Green open access

[thumbnail of ShuminAn-T-ITS-accepted.pdf]
Preview
Text
ShuminAn-T-ITS-accepted.pdf - Accepted Version

Download (9MB) | Preview

Abstract

Lightweight models are pivotal in efficient semantic segmentation, but they often suffer from insufficient context information due to limited convolution and small receptive field. To address this problem, we propose a tailored approach to efficient semantic segmentation by leveraging two complementary distillation schemes for supplementing context information to small networks: 1) a self-attention distillation scheme, which transfers long-range context knowledge adaptively from large teacher networks to small student networks; and 2) a layer-wise context distillation scheme, which transfers structured context from deep layers to shallow layers within student networks for promoting semantic consistency of the shallow layers. Extensive experiments on the ADE20K, Cityscapes, and Camvid datasets well demonstrate the effectiveness of our proposal.

Type: Article
Title: Efficient Semantic Segmentation via Self-Attention and Self-Distillation
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/tits.2021.3139001
Publisher version: https://doi.org/10.1109/tits.2021.3139001
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Semantics, Context modeling, Knowledge engineering, Correlation, Convolution, Adaptation models, Transforms
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science
URI: https://discovery.ucl.ac.uk/id/eprint/10141780
Downloads since deposit
135Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item