UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Recommendations for the development and use of imaging test sets to investigate the test performance of artificial intelligence in health screening

Chalkidou, A; Shokraneh, F; Kijauskaite, G; Taylor-Phillips, S; Halligan, S; Wilkinson, L; Glocker, B; ... Seedat, F; + view all (2022) Recommendations for the development and use of imaging test sets to investigate the test performance of artificial intelligence in health screening. The Lancet. Digital health , 4 (12) e899-e905. 10.1016/S2589-7500(22)00186-8. Green open access

[thumbnail of 1-s2.0-S2589750022001868-main.pdf]
Preview
Text
1-s2.0-S2589750022001868-main.pdf - Published Version

Download (173kB) | Preview

Abstract

Rigorous evaluation of artificial intelligence (AI) systems for image classification is essential before deployment into health-care settings, such as screening programmes, so that adoption is effective and safe. A key step in the evaluation process is the external validation of diagnostic performance using a test set of images. We conducted a rapid literature review on methods to develop test sets, published from 2012 to 2020, in English. Using thematic analysis, we mapped themes and coded the principles using the Population, Intervention, and Comparator or Reference standard, Outcome, and Study design framework. A group of screening and AI experts assessed the evidence-based principles for completeness and provided further considerations. From the final 15 principles recommended here, five affect population, one intervention, two comparator, one reference standard, and one both reference standard and comparator. Finally, four are appliable to outcome and one to study design. Principles from the literature were useful to address biases from AI; however, they did not account for screening specific biases, which we now incorporate. The principles set out here should be used to support the development and use of test sets for studies that assess the accuracy of AI within screening programmes, to ensure they are fit for purpose and minimise bias.

Type: Article
Title: Recommendations for the development and use of imaging test sets to investigate the test performance of artificial intelligence in health screening
Location: England
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/S2589-7500(22)00186-8
Publisher version: https://doi.org/10.1016/S2589-7500(22)00186-8
Language: English
Additional information: Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license.
Keywords: Artificial Intelligence, Diagnostic Imaging, Mass Screening
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Institute of Ophthalmology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Medicine
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Medicine > Department of Imaging
URI: https://discovery.ucl.ac.uk/id/eprint/10162169
Downloads since deposit
28Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item