Vargaftik, Shay;
Ben-Basat, Ran;
Portnoy, Amit;
Mendelson, Gal;
Ben-Itzhak, Yaniv;
Mitzenmacher, Michael;
(2021)
DRIVE: one-bit distributed mean estimation.
In:
Advances in Neural Information Processing Systems 34 (NeurIPS 2021).
NeurIPS Proceedings
Preview |
PDF
drive_one_bit_distributed_mean-Supplementary Material.pdf - Published Version Download (965kB) | Preview |
Abstract
We consider the problem where n clients transmit d-dimensional real-valued vectors using dp1 ` op1qq bits each, in a manner that allows the receiver to approximately reconstruct their mean. Such compression problems naturally arise in distributed and federated learning. We provide novel mathematical results and derive computationally efficient algorithms that are more accurate than previous compression techniques. We evaluate our methods on a collection of distributed and federated learning tasks, using a variety of datasets, and show a consistent improvement over the state of the art.
| Type: | Proceedings paper |
|---|---|
| Title: | DRIVE: one-bit distributed mean estimation |
| Event: | Advances in Neural Information Processing Systems 34 (NeurIPS 2021) |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.neurips.cc/paper/2021/hash/039... |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10152070 |
Archive Staff Only
 |
View Item |
