Xiong, Zhuxin;
Wang, Xiaofei;
Zhou, Yukun;
Keane, Pearse A;
Tham, Yih Chung;
Wang, Ya Xing;
Wong, Tien Yin;
(2025)
How Generalizable Are Foundation Models When Applied to Different Demographic Groups and Settings?
NEJM AI
, 2
(1)
10.1056/aics2400497.
Preview |
Text
Keane_Manuscript_2.pdf Download (270kB) | Preview |
Abstract
RETFound is a retinal image–based foundational artificial intelligence (AI) model that can be fine-tuned to downstream tasks. However, its generalizability to Asian populations remains unclear. In this study, we fine-tuned RETFound on an Asian-specific dataset. We then evaluated the performance of RETFound versus a conventional Vision Transformer model (pretrained on ImageNet) in diagnosing glaucoma and coronary heart disease and predicting the 3-year risk of stroke in an Asian population. When fine-tuned on a “full” dataset, RETFound showed no significant improvement compared with a conventional Vision Transformer model (area under the curves [AUCs] of 0.863, 0.628, and 0.557 vs. 0.853, 0.621, and 0.543, respectively; all P≥0.2). Furthermore, in scenarios with limited training data (fine-tuned on ≤25% of the full dataset), RETFound showed a slight advantage (up to a maximum AUC increase of 0.03). However, these improvements were not statistically significant (all P≥0.2). These findings indicate the challenges foundational AI models face in adapting to diverse demographics, emphasizing the need for more diverse data in current foundation models and the importance of global collaboration on foundation model research.
| Type: | Article |
|---|---|
| Title: | How Generalizable Are Foundation Models When Applied to Different Demographic Groups and Settings? |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1056/aics2400497 |
| Publisher version: | https://doi.org/10.1056/aics2400497 |
| Language: | English |
| Additional information: | © 2023 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/) |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Institute of Ophthalmology |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10203510 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
