Lopes, Vasco; Carlucci, Fabio Maria; Esperanca, Pedro M; Singh, Marco; Yang, Antoine; Gabillon, Victor; Xu, Hang; ... Wang, Jun; + view all Lopes, Vasco; Carlucci, Fabio Maria; Esperanca, Pedro M; Singh, Marco; Yang, Antoine; Gabillon, Victor; Xu, Hang; Chen, Zewei; Wang, Jun; - view fewer (2024) Manas: multi-agent neural architecture search. Machine Learning , 113 (1) pp. 73-96. 10.1007/s10994-023-06379-w.

Preview

Text 1909.01051v4.pdf - Accepted Version Download (1MB) | Preview

Abstract

The Neural Architecture Search (NAS) problem is typically formulated as a graph search problem where the goal is to learn the optimal operations over edges in order to maximize a graph-level global objective. Due to the large architecture parameter space, efficiency is a key bottleneck preventing NAS from its practical use. In this work, we address the issue by framing NAS as a multi-agent problem where agents control a subset of the network and coordinate to reach optimal architectures. We provide two distinct lightweight implementations, with reduced memory requirements (1/8th of state-of-the-art), and performances above those of much more computationally expensive methods. Theoretically, we demonstrate vanishing regrets of the form O(T) , with T being the total number of rounds. Finally, we perform experiments on CIFAR-10 and ImageNet, and aware that random search and random sampling are (often ignored) effective baselines, we conducted additional experiments on 3 alternative datasets, with complexity constraints, and 2 network configurations, and achieve competitive results in comparison with the baselines and other methods.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item