UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Efficient architecture search by network transformation

Cai, H; Chen, T; Zhang, W; Yu, Y; Wang, J; (2018) Efficient architecture search by network transformation. In: (Proceedings) 32nd AAAI Conference on Artificial Intelligence, AAAI 2018. (pp. pp. 2787-2794). AAAI: New Orleans, LA, USA. Green open access

[img]
Preview
Text
16755-77334-1-PB.pdf - Published version

Download (567kB) | Preview

Abstract

Techniques for automatically designing deep neural network architectures such as reinforcement learning based approaches have recently shown promising results. However, their success is based on vast computational resources (e.g. hundreds of GPUs), making them difficult to be widely used. A noticeable limitation is that they still design and train each network from scratch during the exploration of the architecture space, which is highly inefficient. In this paper, we propose a new framework toward efficient architecture search by exploring the architecture space based on the current network and reusing its weights. We employ a reinforcement learning agent as the meta-controller, whose action is to grow the network depth or layer width with function-preserving transformations. As such, the previously validated networks can be reused for further exploration, thus saves a large amount of computational cost. We apply our method to explore the architecture space of the plain convolutional neural networks (no skip-connections, branching etc.) on image benchmark datasets (CIFAR-10, SVHN) with restricted computational resources (5 GPUs). Our method can design highly competitive networks that outperform existing networks using the same design scheme. On CIFAR-10, our model without skip-connections achieves 4.23% test error rate, exceeding a vast majority of modern architectures and approaching DenseNet. Furthermore, by applying our method to explore the DenseNet architecture space, we are able to achieve more accurate networks with fewer parameters.

Type: Proceedings paper
Title: Efficient architecture search by network transformation
Event: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018
ISBN-13: 9781577358008
Open access status: An open access version is available from UCL Discovery
Publisher version: https://www.aaai.org/ocs/index.php/AAAI/AAAI18/pap...
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Automatic Architecture Search; Deep Neural Networks; Reinforcement Learning
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10070680
Downloads since deposit
16Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item