UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Fast Parallel Estimation of High Dimensional Information Theoretical Quantities with Low Dimensional Random Projection Ensembles

Szabo, Z; Lőrincz, A; (2009) Fast Parallel Estimation of High Dimensional Information Theoretical Quantities with Low Dimensional Random Projection Ensembles. In: Adali, T and Jutten, C and Romano, JMT and Barros, AK, (eds.) Independent Component Analysis and Signal Separation: 8th International Conference, ICA 2009, Paraty, Brazil, March 15-18, 2009. Proceedings. (146 - 153). Springer-Verlag Berlin Heidelberg Green open access

[thumbnail of szabo09fast.pdf]
Preview
PDF
szabo09fast.pdf
Available under License : See the attached licence file.

Download (329kB)
[thumbnail of szabo09fast_poster.pdf]
Preview
PDF
szabo09fast_poster.pdf
Available under License : See the attached licence file.

Download (303kB)

Abstract

The estimation of relevant information theoretical quantities, such as entropy, mutual information, and various divergences is computationally expensive in high dimensions. However, for this task, one may apply pairwise Euclidean distances of sample points, which suits random projection (RP) based low dimensional embeddings. The Johnson-Lindenstrauss (JL) lemma gives theoretical bound on the dimension of the low dimensional embedding. We adapt the RP technique for the estimation of information theoretical quantities. Intriguingly, we find that embeddings into extremely small dimensions, far below the bounds of the JL lemma, provide satisfactory estimates for the original task.We illustrate this in the Independent Subspace Analysis (ISA) task; we combine RP dimension reduction with a simple ensemble method. We gain considerable speed-up with the potential of real-time parallel estimation of high dimensional information theoretical quantities.

Type: Book chapter
Title: Fast Parallel Estimation of High Dimensional Information Theoretical Quantities with Low Dimensional Random Projection Ensembles
Event: International Conference on Independent Component Analysis and Signal Separation (ICA)
Location: Paraty, Brazil
Dates: 2009-03-15 - 2009-03-18
ISBN-13: 978-3-642-00598-5
Open access status: An open access version is available from UCL Discovery
DOI: 10.1007/978-3-642-00599-2_19
Publisher version: http://dx.doi.org/10.1007/978-3-642-00599-2_19
Language: English
Additional information: The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-00599-2_19
Keywords: independent subspace analysis, information theoretical estimations, pairwise distances, random projection
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
URI: https://discovery.ucl.ac.uk/id/eprint/1433164
Downloads since deposit
181Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item