Wang, Ruohan;
Falk, John Isak Texas;
Pontil, Massimiliano;
Ciliberto, Carlo;
(2024)
Robust Meta-Representation Learning via Global Label Inference and Classification.
IEEE Transactions on Pattern Analysis and Machine Intelligence
, 46
(4)
pp. 1996-2010.
10.1109/TPAMI.2023.3328184.
Preview |
Text
Robust_Meta-Representation_Learning_via_Global_Label_Inference_and_Classification.pdf Download (1MB) | Preview |
Abstract
Few-shot learning (FSL) is a central problem in meta-learning, where learners must efficiently learn from few labeled examples. Within FSL, feature pre-training has become a popular strategy to significantly improve generalization performance. However, the contribution of pre-training to generalization performance is often overlooked and understudied, with limited theoretical understanding. Further, pre-training requires a consistent set of global labels shared across training tasks, which may be unavailable in practice. In this work, we address the above issues by first showing the connection between pre-training and meta-learning. We discuss why pre-training yields more robust meta-representation and connect the theoretical analysis to existing works and empirical results. Second, we introduce Meta Label Learning (MeLa), a novel meta-learning algorithm that learns task relations by inferring global labels across tasks. This allows us to exploit pre-training for FSL even when global labels are unavailable or ill-defined. Lastly, we introduce an augmented pre-training procedure that further improves the learned meta-representation. Empirically, MeLa outperforms existing methods across a diverse range of benchmarks, in particular under a more challenging setting where the number of training tasks is limited and labels are task-specific.
| Type: | Article |
|---|---|
| Title: | Robust Meta-Representation Learning via Global Label Inference and Classification |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/TPAMI.2023.3328184 |
| Publisher version: | http://dx.doi.org/10.1109/tpami.2023.3328184 |
| Language: | English |
| Additional information: | Copyright © 2023 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ |
| Keywords: | Few-Shot Image Classification, Meta-Learning, Learning with Limited Labels, Representation Learning |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10185737 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
