UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

BSL-1K: Scaling Up Co-articulated Sign Language Recognition Using Mouthing Cues

Albanie, S; Varol, G; Momeni, L; Afouras, T; Chung, JS; Fox, N; Zisserman, A; (2020) BSL-1K: Scaling Up Co-articulated Sign Language Recognition Using Mouthing Cues. In: Vedaldi, A and Bischof, H and Brox, T and Frahm, JM, (eds.) Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI. (pp. pp. 35-53). Springer: Cham, Switzerland. Green open access

[thumbnail of BSL1k.pdf]
Preview
Text
BSL1k.pdf - Published Version

Download (15MB) | Preview

Abstract

Recent progress in fine-grained gesture and action classification, and machine translation, point to the possibility of automated sign language recognition becoming a reality. A key stumbling block in making progress towards this goal is a lack of appropriate training data, stemming from the high complexity of sign annotation and a limited supply of qualified annotators. In this work, we introduce a new scalable approach to data collection for sign recognition in continuous videos. We make use of weakly-aligned subtitles for broadcast footage together with a keyword spotting method to automatically localise sign-instances for a vocabulary of 1,000 signs in 1,000 h of video. We make the following contributions: (1) We show how to use mouthing cues from signers to obtain high-quality annotations from video data—the result is the BSL-1K dataset, a collection of British Sign Language (BSL) signs of unprecedented scale; (2) We show that we can use BSL-1K to train strong sign recognition models for co-articulated signs in BSL and that these models additionally form excellent pretraining for other sign languages and benchmarks—we exceed the state of the art on both the MSASL and WLASL benchmarks. Finally, (3) we propose new large-scale evaluation sets for the tasks of sign recognition and sign spotting and provide baselines which we hope will serve to stimulate research in this area.

Type: Proceedings paper
Title: BSL-1K: Scaling Up Co-articulated Sign Language Recognition Using Mouthing Cues
Event: ECCV 2020: 16th European Conference on Computer Vision
ISBN-13: 978-3-030-58620-1
Open access status: An open access version is available from UCL Discovery
DOI: 10.1007/978-3-030-58621-8_3
Publisher version: https://doi.org/10.1007/978-3-030-58621-8_3
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Sign language recognition, Visual keyword spotting
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > Linguistics
URI: https://discovery.ucl.ac.uk/id/eprint/10120166
Downloads since deposit
27Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item