UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Let's Agree to Disagree: Learning Highly Debatable Multirater Labelling

Sudre, CH; Anson, BG; Ingala, S; Lane, CD; Jimenez, D; Haider, L; Varsavsky, T; ... Cardoso, MJ; + view all (2019) Let's Agree to Disagree: Learning Highly Debatable Multirater Labelling. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019 , 11767 pp. 665-673. 10.1007/978-3-030-32251-9_73. Green open access

[thumbnail of 1909.01891v1.pdf]
Preview
Text
1909.01891v1.pdf - Accepted Version

Download (481kB) | Preview

Abstract

Classification and differentiation of small pathological objects may greatly vary among human raters due to differences in training, expertise and their consistency over time. In a radiological setting, objects commonly have high within-class appearance variability whilst sharing certain characteristics across different classes, making their distinction even more difficult. As an example, markers of cerebral small vessel disease, such as enlarged perivascular spaces (EPVS) and lacunes, can be very varied in their appearance while exhibiting high inter-class similarity, making this task highly challenging for human raters. In this work, we investigate joint models of individual rater behaviour and multi-rater consensus in a deep learning setting, and apply it to a brain lesion object-detection task. Results show that jointly modelling both individual and consensus estimates leads to significant improvements in performance when compared to directly predicting consensus labels, while also allowing the characterization of human-rater consistency.

Type: Article
Title: Let's Agree to Disagree: Learning Highly Debatable Multirater Labelling
Open access status: An open access version is available from UCL Discovery
DOI: 10.1007/978-3-030-32251-9_73
Publisher version: https://doi.org/10.1007/978-3-030-32251-9_73
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions.
Keywords: Deep learning, Noisy labels, Classification
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Brain Repair and Rehabilitation
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Neuroinflammation
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Medicine
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science > Population Science and Experimental Medicine
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science > Population Science and Experimental Medicine > MRC Unit for Lifelong Hlth and Ageing
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Med Phys and Biomedical Eng
URI: https://discovery.ucl.ac.uk/id/eprint/10108463
Downloads since deposit
315Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item