UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Why is the Winner the Best?

Eisenmann, M; Reinke, A; Weru, V; Tizabi, MD; Isensee, F; Adler, TJ; Ali, S; ... Maier-Hein, L; + view all (2023) Why is the Winner the Best? In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). (pp. pp. 19955-19966). IEEE: Vancouver, BC, Canada. Green open access

[thumbnail of Eisenmann_Why_Is_the_Winner_the_Best_CVPR_2023_paper.pdf]
Preview
Text
Eisenmann_Why_Is_the_Winner_the_Best_CVPR_2023_paper.pdf - Accepted Version

Download (5MB) | Preview

Abstract

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and post-processing (66%). The “typical” lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.

Type: Proceedings paper
Title: Why is the Winner the Best?
Event: IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023
Dates: 17 Jun 2023 - 24 Jun 2023
ISBN-13: 979-8-3503-0129-8
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/CVPR52729.2023.01911
Publisher version: https://doi.org/10.1109/CVPR52729.2023.01911
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Measurement; Statistical analysis; Microscopy; Pipelines; Medical services; Lead; Multitasking
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science > Population Science and Experimental Medicine
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Population Health Sciences > Institute of Cardiovascular Science > Population Science and Experimental Medicine > MRC Unit for Lifelong Hlth and Ageing
URI: https://discovery.ucl.ac.uk/id/eprint/10179660
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item