%0 Journal Article
%@ 2405-4712
%A Nagano, Yuta
%A Pyo, Andrew GT
%A Milighetti, Martina
%A Henderson, James
%A Shawe-Taylor, John
%A Chain, Benny
%A Tiffeau-Mayer, Andreas
%D 2025
%F discovery:10205427
%I Elsevier BV
%J Cell Systems
%K Protein language models; contrastive learning; TCR repertoire; T cell specificity; TCR; T cell receptor; representation learning
%N 1
%T Contrastive learning of T cell receptor representations
%U https://discovery.ucl.ac.uk/id/eprint/10205427/
%V 16
%X Computational prediction of the interaction of T cell receptors (TCRs) and their ligands is a grand challenge in immunology. Despite advances in high-throughput assays, specificity-labeled TCR data remain sparse. In other domains, the pre-training of language models on unlabeled data has been successfully used to address data bottlenecks. However, it is unclear how to best pre-train protein language models for TCR specificity prediction. Here, we introduce a TCR language model called SCEPTR (simple contrastive embedding of the primary sequence of T cell receptors), which is capable of data-efficient transfer learning. Through our model, we introduce a pre-training strategy combining autocontrastive learning and masked-language modeling, which enables SCEPTR to achieve its state-of-the-art performance. In contrast, existing protein language models and a variant of SCEPTR pre-trained without autocontrastive learning are outperformed by sequence alignment-based methods. We anticipate that contrastive learning will be a useful paradigm to decode the rules of TCR specificity. A record of this paper’s transparent peer review process is included in the supplemental information.
%Z Copyright ©  2024 The Author(s). Published by Elsevier Inc.   This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).