Szabo, Z;
Sriperumbudur, B;
Poczos, B;
Gretton, A;
(2016)
Distribution Regression with Minimax-Optimal Guarantee.
Presented at: MASCOT-NUM 2016, Toulouse, France.
Preview |
Text
Zoltan_Szabo_invited_talk_MASCOT-NUM_25_03_2016.pdf Download (1MB) | Preview |
Abstract
We focus on the distribution regression problem (DRP): we regress from probability measures to Hilbert-space valued outputs, where the input distributions are only available through samples (this is the 'two-stage sampled' setting). Several important statistical and machine learning problems can be phrased within this framework including point estimation tasks without analytical solution (such as hyperparameter or entropy estimation) and multi-instance learning. However, due to the two-stage sampled nature of the problem, the theoretical analysis becomes quite challenging: to the best of our knowledge the only existing method with performance guarantees to solve the DRP task requires density estimation (which often performs poorly in practise) and the distributions to be defined on a compact Euclidean domain. We present a simple, analytically tractable alternative to solve the DRP task: we embed the distributions to a reproducing kernel Hilbert space and perform ridge regression from the embedded distributions to the outputs. Our main contribution is to prove that this scheme is consistent in the two-stage sampled setup under mild conditions (on separable topological domains enriched with kernels): we present an exact computational-statistical efficiency tradeoff analysis showing that the studied estimator is able to match the one-stage sampled minimax-optimal rate. This result answers a 17-year-old open question, by establishing the consistency of the classical set kernel [Haussler, 1999; Gaertner et. al, 2002] in regression. We also cover consistency for more recent kernels on distributions, including those due to [Christmann and Steinwart, 2010]. The practical efficiency of the studied technique is illustrated in supervised entropy learning and aerosol prediction using multispectral satellite images.
| Type: | Conference item (Presentation) |
|---|---|
| Title: | Distribution Regression with Minimax-Optimal Guarantee |
| Event: | MASCOT-NUM 2016 |
| Location: | Toulouse, France |
| Dates: | 23 - 25 March 2016 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | http://mascot2016.sciencesconf.org/ |
| Language: | English |
| Keywords: | Two-stage sampled distribution regression, kernel ridge regression, mean embedding, multi-instance learning, minimax optimality. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit |
| URI: | https://discovery.ucl.ac.uk/id/eprint/1474104 |
Archive Staff Only
 |
View Item |
