Ferianc, M; Sankaran, A; Mastropietro, O; Saboori, E; Cappart, Q; (2022) On Causal Inference for Data-free Structured Pruning. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence: ITCI'22. AAAI: Vancouver, BC, Canada.

Preview

Text 2112.10229v1.pdf - Published Version Download (339kB) | Preview

Abstract

Neural networks (NNs) are making a large impact both on research and industry. Nevertheless, as NNs' accuracy increases, it is followed by an expansion in their size, required number of compute operations and energy consumption. Increase in resource consumption results in NNs' reduced adoption rate and real-world deployment impracticality. Therefore, NNs need to be compressed to make them available to a wider audience and at the same time decrease their runtime costs. In this work, we approach this challenge from a causal inference perspective, and we propose a scoring mechanism to facilitate structured pruning of NNs. The approach is based on measuring mutual information under a maximum entropy perturbation, sequentially propagated through the NN. We demonstrate the method's performance on two datasets and various NNs' sizes, and we show that our approach achieves competitive performance under challenging conditions.

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item