Salimbeni, H;
Cheng, C-A;
Boots, B;
Deisenroth, M;
(2018)
Orthogonally Decoupled Variational Gaussian Processes.
In: Bengio, S and Wallach, H and Larochelle, H and Grauman, K and CesaBianchi, N and Garnett, R, (eds.)
Proceedings of the 32nd Conference on Neural Information Processing Systems (NIPS 2018).
Neural Information Processing Systems (NIPS): Montreal, QC, Canada.
Preview |
Text
Deisenroth_permitted VoR_8088-orthogonally-decoupled-variational-gaussian-processes.pdf - Published Version Download (431kB) | Preview |
Abstract
Gaussian processes (GPs) provide a powerful non-parametric framework for reasoning over functions. Despite appealing theory, its superlinear computational and memory complexities have presented a long-standing challenge. State-of-the-art sparse variational inference methods trade modeling accuracy against complexity. However, the complexities of these methods still scale superlinearly in the number of basis functions, implying that that sparse GP methods are able to learn from large datasets only when a small model is used. Recently, a decoupled approach was proposed that removes the unnecessary coupling between the complexities of modeling the mean and the covariance functions of a GP. It achieves a linear complexity in the number of mean parameters, so an expressive posterior mean function can be modeled. While promising, this approach suffers from optimization difficulties due to ill-conditioning and non-convexity. In this work, we propose an alternative decoupled parametrization. It adopts an orthogonal basis in the mean function to model the residues that cannot be learned by the standard coupled approach. Therefore, our method extends, rather than replaces, the coupled approach to achieve strictly better performance. This construction admits a straightforward natural gradient update rule, so the structure of the information manifold that is lost during decoupling can be leveraged to speed up learning. Empirically, our algorithm demonstrates significantly faster convergence in multiple experiments.
| Type: | Proceedings paper |
|---|---|
| Title: | Orthogonally Decoupled Variational Gaussian Processes |
| Event: | 32nd Conference on Neural Information Processing Systems (NIPS 2018), 3-8 December 2018, Montreal, QC, Canada |
| Location: | Montreal, CANADA |
| Dates: | 02 December 2018 - 08 December 2018 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://papers.nips.cc/paper/8088-orthogonally-dec... |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10083562 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
