An, S;
Liao, Q;
Lu, Z;
Xue, J-H;
(2022)
Efficient Semantic Segmentation via Self-Attention and Self-Distillation.
IEEE Transactions on Intelligent Transportation Systems
10.1109/tits.2021.3139001.
(In press).
Preview |
Text
ShuminAn-T-ITS-accepted.pdf - Accepted Version Download (9MB) | Preview |
Abstract
Lightweight models are pivotal in efficient semantic segmentation, but they often suffer from insufficient context information due to limited convolution and small receptive field. To address this problem, we propose a tailored approach to efficient semantic segmentation by leveraging two complementary distillation schemes for supplementing context information to small networks: 1) a self-attention distillation scheme, which transfers long-range context knowledge adaptively from large teacher networks to small student networks; and 2) a layer-wise context distillation scheme, which transfers structured context from deep layers to shallow layers within student networks for promoting semantic consistency of the shallow layers. Extensive experiments on the ADE20K, Cityscapes, and Camvid datasets well demonstrate the effectiveness of our proposal.
| Type: | Article |
|---|---|
| Title: | Efficient Semantic Segmentation via Self-Attention and Self-Distillation |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/tits.2021.3139001 |
| Publisher version: | https://doi.org/10.1109/tits.2021.3139001 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Semantics, Context modeling, Knowledge engineering, Correlation, Convolution, Adaptation models, Transforms |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10141780 |
Archive Staff Only
 |
View Item |
