UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms

Aminian, G; Toni, L; Rodrigues, M; (2021) Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms. In: Proceedings of the 2020 IEEE Information Theory Workshop (ITW). IEEE: Riva del Garda, Italy. Green open access

[thumbnail of ITW_Jensen_Shannon_Upper_Bound-4.pdf]
Preview
Text
ITW_Jensen_Shannon_Upper_Bound-4.pdf - Accepted Version

Download (307kB) | Preview

Abstract

Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios. We show that our general bound can specialize in various previous bounds. We also show that our general bound can be specialized under some conditions to a new bound involving the Jensen-Shannon information between a random variable modelling the set of training samples and another random variable modelling the hypothesis. We also prove that our bound can be tighter than mutual information-based bounds under some conditions.

Type: Proceedings paper
Title: Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms
Event: 2020 IEEE Information Theory Workshop (ITW)
Dates: 11 April 2021 - 15 April 2021
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/ITW46852.2021.9457642
Publisher version: https://doi.org/10.1109/ITW46852.2021.9457642
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Generalization Error Bounds, Mutual Information, Jensen-Shannon Information
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng
URI: https://discovery.ucl.ac.uk/id/eprint/10118789
Downloads since deposit
72Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item