Ferianc, Martin; Bohdal, Ondrej; Hospedales, Tiimothy; Rodrigues, Miguel; (2023) Impact of Noise on Calibration and Generalisation of Neural Networks. In: Proceedings of the Second Workshop on Spurious Correlations, Invariance and Stability. (pp. pp. 1-7). ICML: San Diego, CA, USA.

Preview

Text SCIS_23.pdf - Accepted Version Download (196kB) | Preview

Abstract

Noise injection and data augmentation strategies have been effective for enhancing the generalisation and robustness of neural networks (NNs). Certain types of noise such as label smoothing and MixUp have also been shown to improve calibration. Since noise can be added in various stages of the NN’s training, it motivates the question of when and where the noise is the most effective. We study a variety of noise types to determine how much they improve calibration and generalisation, and under what conditions. More specifically we evaluate various noise-injection strategies in both in-distribution (ID) and out-of-distribution (OOD) scenarios. The findings highlight that activation noise was the most transferable and effective in improving generalisation, while input augmentation noise was prominent in improving calibration on OOD but not necessarily ID data.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as CSVJSONXML

Archive Staff Only

View Item