Joksas, Dovydas;
Wang, Erwei;
Barmpatsalos, Nikolaos;
Ng, Wing H;
Kenyon, Anthony J;
Constantinides, George A;
Mehonic, Adnan;
(2022)
Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks.
Advanced Science
, Article 2105784. 10.1002/advs.202105784.
Preview |
Text
Advanced Science - 2022 - Joksas - Nonideality%E2%80%90Aware Training for Accurate and Robust Low%E2%80%90Power Memristive Neural Networks.pdf - Published Version Download (1MB) | Preview |
Abstract
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade-offs, such as those between power and reliability. In this work, we design nonideality-aware training of memristor-based neural networks capable of dealing with the most common device nonidealities. We demonstrate the feasibility of using high-resistance devices that exhibit high $I$-$V$ nonlinearity -- by analyzing experimental data and employing nonideality-aware training, we estimate that the energy efficiency of memristive vector-matrix multipliers is improved by three orders of magnitude ($0.715\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$ to $381\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$) while maintaining similar accuracy. We show that associating the parameters of neural networks with individual memristors allows to bias these devices towards less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. We demonstrate the universality and robustness of our approach when dealing with a wide range of nonidealities.
| Type: | Article |
|---|---|
| Title: | Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1002/advs.202105784 |
| Publisher version: | https://doi.org/10.1002/advs.202105784 |
| Language: | English |
| Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Chemical Engineering UCL > Provost and Vice Provost Offices > UCL BEAMS UCL UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Electronic and Electrical Eng |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10148024 |
Archive Staff Only
 |
View Item |
