Nielsen, Morten;
Eugster, Anne;
Jensen, Mathias Fynbo;
Goel, Manisha;
Tiffeau-Mayer, Andreas;
Pelissier, Aurelien;
Valkiers, Sebastiaan;
... Barton, Justin; + view all
(2024)
Lessons learned from the IMMREP23 TCR-epitope prediction challenge.
ImmunoInformatics
, 16
, Article 100045. 10.1016/j.immuno.2024.100045.
Preview |
Text
Nielsen2024.pdf - Published Version Download (1MB) | Preview |
Abstract
Here, we present the findings from IMMREP23, the second benchmark competition focused on predicting the specificity of TCR-pMHC interactions. The interaction of T cell receptors (TCR) towards their pMHC target is a cornerstone of the cellular immune system. Over the last decade, substantial progress has been made within the field of TCR specificity prediction, providing proof of concept for predicting TCR-pMHC interactions in a narrow space of “seen” pMHC targets where substantial training data is available. However, a significant challenge persists in extending the predictive capability to novel “unseen” pMHC targets. Furthermore, the performance of proposed methods is often challenged when evaluated outside the initial publication and data sets. To address these issues, IMMREP23 challenge invited participants to predict, for a given test set of TCR-pMHC pairs, the likelihood that a pair would bind. A total of 53 teams participated, providing a total of 398 submissions. The benchmark confirms that current methods achieve reasonable performance in the "seen" pMHC setting. However, most participating methods had close to random performance on the subset of “unseen” peptides, underlining that this prediction challenge remains essentially unsolved. Finally, another key lesson from the benchmark is the critical issue of data leakage. Specifically, the data set construction procedure employed in IMMREP23 led to biases in the negative test data set. These biases were identified by several participating teams, and complicated the interpretation of the benchmark results. Based on these results, we put forward suggestions on how future competitions could avoid such data leakages and biases.
| Type: | Article |
|---|---|
| Title: | Lessons learned from the IMMREP23 TCR-epitope prediction challenge |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.immuno.2024.100045 |
| Publisher version: | https://doi.org/10.1016/j.immuno.2024.100045 |
| Language: | English |
| Additional information: | Copyright © 2024 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/bync-nd/4.0/). |
| Keywords: | TCR specificity; Benchmark; Prediction; Machine learning |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Medical Sciences > Div of Infection and Immunity |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10205429 |
Archive Staff Only
 |
View Item |
