UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Efficient Conditionally Invariant Representation Learning

Pogodin, Roman; Deka, Namrata; Li, Yazhe; Sutherland, Danica J; Veitch, Victor; Gretton, Arthur; (2023) Efficient Conditionally Invariant Representation Learning. In: Proceedings of the Eleventh International Conference on Learning Representations. (pp. p. 4723). International Conference on Learning Representations: Kigali, Rwanda. (In press). Green open access

[thumbnail of 4723_efficient_conditionally_invari.pdf]
Preview
Text
4723_efficient_conditionally_invari.pdf - Accepted Version

Download (559kB) | Preview

Abstract

We introduce the Conditional Independence Regression CovariancE (CIRCE), a measure of conditional independence for multivariate continuous-valued variables. CIRCE applies as a regularizer in settings where we wish to learn neural features φ(X) of data X to estimate a target Y , while being conditionally independent of a distractor Z given Y . Both Z and Y are assumed to be continuous-valued but relatively low dimensional, whereas X and its features may be complex and high dimensional. Relevant settings include domain-invariant learning, fairness, and causal learning. The procedure requires just a single ridge regression from Y to kernelized features of Z, which can be done in advance. It is then only necessary to enforce independence of φ(X) from residuals of this regression, which is possible with attractive estimation properties and consistency guarantees. By contrast, earlier measures of conditional feature dependence require multiple regressions for each step of feature learning, resulting in more severe bias and variance, and greater computational cost. When sufficiently rich features are used, we establish that CIRCE is zero if and only if φ(X) ⊥⊥ Z | Y . In experiments, we show superior performance to previous methods on challenging benchmarks, including learning conditionally invariant image features.

Type: Proceedings paper
Title: Efficient Conditionally Invariant Representation Learning
Event: The Eleventh International Conference on Learning Representations
Open access status: An open access version is available from UCL Discovery
Publisher version: https://iclr.cc/public/JournalToConference
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10166304
Downloads since deposit
18Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item