Sokolic, J; Gyries, R; Sapiro, G; Rodrigues, MRD; (2016) Lessons from the Rademacher complexity for deep learning. In: Bengio, Samy and Kingsbury, Brian, (eds.) Proceedings of the workshop track - International Conference on Learning Representations 2016: ICLR 2016. ICLR: San Juan, Puerto Rico.

Preview

Text LessonsFromTheRCforDL_Sokolic.pdf - Published Version Download (264kB) | Preview

Abstract

Understanding the generalization properties of deep learning models is critical for successful applications, especially in the regimes where the number of training samples is limited. We study the generalization properties of deep neural networks via the empirical Rademacher complexity and show that it is easier to control the complexity of convolutional networks compared to general fully connected networks. In particular, we justify the usage of small convolutional kernels in deep networks as they lead to a better generalization error. Moreover, we propose a representation based regularization method that allows to decrease the generalization error by controlling the coherence of the representation. Experiments on the MNIST dataset support these foundations.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item