eprintid: 10146010
rev_number: 7
eprint_status: archive
userid: 699
dir: disk0/10/14/60/10
datestamp: 2022-03-30 09:03:57
lastmod: 2022-03-30 09:04:19
status_changed: 2022-03-30 09:03:57
type: article
metadata_visibility: show
sword_depositor: 699
creators_name: de Lange, Ann-Marie G
creators_name: Anatürk, Melis
creators_name: Rokicki, Jaroslav
creators_name: Han, Laura KM
creators_name: Franke, Katja
creators_name: Alnaes, Dag
creators_name: Ebmeier, Klaus P
creators_name: Draganski, Bogdan
creators_name: Kaufmann, Tobias
creators_name: Westlye, Lars T
creators_name: Hahn, Tim
creators_name: Cole, James H
title: Mind the gap: Performance metric evaluation in brain-age prediction
ispublished: inpress
divisions: C05
divisions: F48
divisions: B04
divisions: UCL
keywords: brain-age prediction, machine learning, neuroimaging, statistics
note: © 2022 The Authors. Human Brain Mapping published by Wiley Periodicals LLC. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium,
provided the original work is properly cited.
abstract: Estimating age based on neuroimaging-derived data has become a popular approach to developing markers for brain integrity and health. While a variety of machine-learning algorithms can provide accurate predictions of age based on brain characteristics, there is significant variation in model accuracy reported across studies. We predicted age in two population-based datasets, and assessed the effects of age range, sample size and age-bias correction on the model performance metrics Pearson's correlation coefficient (r), the coefficient of determination (R2 ), Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). The results showed that these metrics vary considerably depending on cohort age range; r and R2 values are lower when measured in samples with a narrower age range. RMSE and MAE are also lower in samples with a narrower age range due to smaller errors/brain age delta values when predictions are closer to the mean age of the group. Across subsets with different age ranges, performance metrics improve with increasing sample size. Performance metrics further vary depending on prediction variance as well as mean age difference between training and test sets, and age-bias corrected metrics indicate high accuracy-also for models showing poor initial performance. In conclusion, performance metrics used for evaluating age prediction models depend on cohort and study-specific data characteristics, and cannot be directly compared across different studies. Since age-bias corrected metrics generally indicate high accuracy, even for poorly performing models, inspection of uncorrected model results provides important information about underlying model attributes such as prediction variance.
date: 2022-03-21
date_type: published
publisher: Wiley
official_url: https://doi.org/10.1002/hbm.25837
oa_status: green
full_text_type: pub
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 1946283
doi: 10.1002/hbm.25837
medium: Print-Electronic
lyricists_name: Cole, James
lyricists_id: JCOLE07
actors_name: Barczynska, Patrycja
actors_id: PBARC91
actors_role: owner
funding_acknowledgements: 5R24AG061421-03 [Collaboratory on Research Definitions for Reserve and Resilience in Cognitive Aging and Dementia]; FR 3709/1-2 [Deutsche Forschungsgemeinschaft]; HA7070/2-2 [Deutsche Forschungsgemeinschaft]; HA7070/3 [Deutsche Forschungsgemeinschaft]; HA7070/4 [Deutsche Forschungsgemeinschaft]; ERA PerMed project "IMPLEMENT" [ERA-net Cofound]; [Fondation Leenaards]; 802998 [H2020 European Research Council]; 1117747 [HDH Wills 1965 Charitable Trust]; 2015073 [Helse Sør-Øst RHF]; 2019107 [Helse Sør-Øst RHF]; AMSP 07 [Interdisciplinary Center for Clinical Research of the Jena University hospital]; MzH 3/020/20 [Interdisciplinary Center for Clinical Research of the Medical Faculty of Münster]; G1001354 [Medical Research Council]; MR/R024790/2 [Medical Research Council]; 223273 [Norges Forskningsråd]; 249795 [Norges Forskningsråd]; 273345 [Norges Forskningsråd]; 276082 [Norges Forskningsråd]; 32003B_135679 [Swiss National Science Foundation]; 32003B_159780 [Swiss National Science Foundation]; 324730_192755 [Swiss National Science Foundation]; CRSK-3_190185 [Swiss National Science Foundation]; PZ00P3_193658 [Swiss National Science Foundation]
full_text_status: public
publication: Human Brain Mapping
event_location: United States
issn: 1065-9471
citation:        de Lange, Ann-Marie G;    Anatürk, Melis;    Rokicki, Jaroslav;    Han, Laura KM;    Franke, Katja;    Alnaes, Dag;    Ebmeier, Klaus P;                     ... Cole, James H; + view all <#>        de Lange, Ann-Marie G;  Anatürk, Melis;  Rokicki, Jaroslav;  Han, Laura KM;  Franke, Katja;  Alnaes, Dag;  Ebmeier, Klaus P;  Draganski, Bogdan;  Kaufmann, Tobias;  Westlye, Lars T;  Hahn, Tim;  Cole, James H;   - view fewer <#>    (2022)    Mind the gap: Performance metric evaluation in brain-age prediction.                   Human Brain Mapping        10.1002/hbm.25837 <https://doi.org/10.1002/hbm.25837>.    (In press).    Green open access   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10146010/1/Cole_%20Mind%20the%20gap%20Performance%20metric%20evaluation%20in%20brain%20age%20prediction.pdf