Meunier, Dimitri;
Alquier, Pierre;
(2021)
Meta-Strategy for Learning Tuning Parameters with Guarantees.
Entropy
, 23
(10)
, Article 1257. 10.3390/e23101257.
Preview |
Text
entropy-23-01257-v2.pdf - Published Version Download (593kB) | Preview |
Abstract
Online learning methods, similar to the online gradient algorithm (OGA) and exponentially weighted aggregation (EWA), often depend on tuning parameters that are difficult to set in practice. We consider an online meta-learning scenario, and we propose a meta-strategy to learn these parameters from past tasks. Our strategy is based on the minimization of a regret bound. It allows us to learn the initialization and the step size in OGA with guarantees. It also allows us to learn the prior or the learning rate in EWA. We provide a regret analysis of the strategy. It allows to identify settings where meta-learning indeed improves on learning each task in isolation.
| Type: | Article |
|---|---|
| Title: | Meta-Strategy for Learning Tuning Parameters with Guarantees |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.3390/e23101257 |
| Publisher version: | https://doi.org/10.3390/e23101257 |
| Language: | English |
| Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
| Keywords: | Meta-learning; hyperparameters; priors; online learning; Bayesian inference; online optimization; gradient descent |
| UCL classification: | UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10150499 |
Archive Staff Only
 |
View Item |
