Wilson, Paul;
Zanasi, Fabio;
Constantinides, George;
(2025)
Convergence for Discrete Parameter Update Schemes.
In:
Proceedings of the 17th Annual Workshop on Optimization for Machine Learning.
Preview |
Text
37_Convergence_for_Discrete_Pa.pdf - Accepted Version Download (377kB) | Preview |
Abstract
Modern deep learning models require immense computational resources, motivating research into low-precision training. Quantised training addresses this by representing training components in low-bit integers, but typically relies on discretising real-valued updates. We introduce an alternative approach where the update rule itself is discrete, avoiding the quantisation of continuous updates by design. We establish convergence guarantees for a general class of such discrete schemes, and present a multinomial update rule as a concrete example, supported by empirical evaluation. This perspective opens new avenues for efficient training, particularly for models with inherently discrete structure.
| Type: | Proceedings paper |
|---|---|
| Title: | Convergence for Discrete Parameter Update Schemes |
| Event: | OPT2025: 17th Annual Workshop on Optimization for Machine Learning |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://opt-ml.org/papers/2025/paper37.pdf |
| Language: | English |
| Additional information: | For the purpose of open access, the author(s) has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10221280 |
Archive Staff Only
 |
View Item |
