Dance, Hugh;
Paige, Brooks;
(2022)
Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes.
In: Camps-Valls, G and Ruiz, FJR and Valera, I, (eds.)
Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) 2022.
(pp. pp. 1-27).
PMLR
Preview |
Text
dance22a.pdf - Published Version Download (1MB) | Preview |
Abstract
Variable selection in Gaussian processes (GPs) is typically undertaken by thresholding the inverse lengthscales of automatic relevance determination kernels, but in high-dimensional datasets this approach can be unreliable. A more probabilistically principled alternative is to use spike and slab priors and infer a posterior probability of variable inclusion. However, existing implementations in GPs are very costly to run in both high-dimensional and large-n datasets, or are only suitable for unsupervised settings with specific kernels. As such, we develop a fast and scalable variational inference algorithm for the spike and slab GP that is tractable with arbitrary differentiable kernels. We improve our algorithm’s ability to adapt to the sparsity of relevant variables by Bayesian model averaging over hyperparameters, and achieve substantial speed ups using zero temperature posterior restrictions, dropout pruning and nearest neighbour minibatching. In experiments our method consistently outperforms vanilla and sparse variational GPs whilst retaining similar runtimes (even when n=10^6) and performs competitively with a spike and slab GP using MCMC but runs up to 1000 times faster.
| Type: | Proceedings paper |
|---|---|
| Title: | Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes |
| Event: | International Conference on Artificial Intelligence and Statistics |
| Location: | Virtual Conference |
| Dates: | 28th-30th March 2022 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://proceedings.mlr.press/v151/dance22a.html |
| Language: | English |
| Additional information: | © The Authors 2023. Original content in this paper is licensed under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) Licence (https://creativecommons.org/licenses/by/4.0/) |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10171344 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
