UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Kernel Exponential Family Estimation via Doubly Dual Embedding

Dai, B; Dai, H; Gretton, A; Song, L; Schuurmans, D; He, N; (2019) Kernel Exponential Family Estimation via Doubly Dual Embedding. In: Chaudhuri, K and Sugiyama, M, (eds.) Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics. (pp. pp. 2321-2330). Proceedings of Machine Learning Research: Naha, Okinawa, Japan. Green open access

[thumbnail of Gretton_dai19a.pdf]
Preview
Text
Gretton_dai19a.pdf - Published version

Download (1MB) | Preview

Abstract

We investigate penalized maximum log-likelihood estimation for exponential family distributions whose natural parameter resides in a reproducing kernel Hilbert space. Key to our approach is a novel technique, doubly dual embedding, that avoids computation of the partition function. This technique also allows the development of a flexible sampling strategy that amortizes the cost of Monte-Carlo sampling in the inference stage. The resulting estimator can be easily generalized to kernel conditional exponential families. We establish a connection between kernel exponential family estimation and MMD-GANs, revealing a new perspective for understanding GANs. Compared to the score matching based estimators, the proposed method improves both memory and time efficiency while enjoying stronger statistical properties, such as fully capturing smoothness in its statistical convergence rate while the score matching estimator appears to saturate. Finally, we show that the proposed estimator empirically outperforms state-of-the-art methods in both kernel exponential family estimation and its conditional extension.

Type: Proceedings paper
Title: Kernel Exponential Family Estimation via Doubly Dual Embedding
Event: 22nd International Conference on Artificial Intelligence and Statistics
Open access status: An open access version is available from UCL Discovery
Publisher version: http://proceedings.mlr.press/v89/dai19a.html
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10076529
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item