UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Probing the latent hierarchical structure of data via diffusion models

Sclocchi, Antonio; Favero, Alessandro; Levi, Noam Itzhak; Wyart, Matthieu; (2025) Probing the latent hierarchical structure of data via diffusion models. Journal of Statistical Mechanics: Theory and Experiment , 2025 (8) , Article 084005. 10.1088/1742-5468/aded6c. Green open access

[thumbnail of jstat_2025_8_084005.pdf]
Preview
Text
jstat_2025_8_084005.pdf - Published Version

Download (2MB) | Preview

Abstract

High-dimensional data must be highly structured to be learnable. Although the compositional and hierarchical nature of data is often put forward to explain learnability, quantitative measurements establishing these properties are scarce. Likewise, accessing the latent variables underlying such a data structure remains a challenge. In this work, we show that forward–backward experiments in diffusion-based models, where data is noised and then denoised to generate new samples, are a promising tool to probe the latent structure of data. We predict in simple hierarchical models that, in this process, changes in data occur by correlated chunks, with a length scale that diverges at a noise level where a phase transition is known to take place. Remarkably, we confirm this prediction in both text and image datasets using state-of-the-art diffusion models. Our results show how latent variable changes manifest in the data and establish how to measure these effects in real data using diffusion models.

Type: Article
Title: Probing the latent hierarchical structure of data via diffusion models
Open access status: An open access version is available from UCL Discovery
DOI: 10.1088/1742-5468/aded6c
Publisher version: https://doi.org/10.1088/1742-5468/aded6c
Language: English
Additional information: Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 license, https://creativecommons.org/licenses/by/4.0/. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Keywords: Correlation functions, deep learning, critical behavior of disordered systems, message-passing algorithms
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10212791
Downloads since deposit
1Download
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item