Ou, Z; Zhang, M; Zhang, A; Xiao, TZ; Li, Y; Barber, D; (2025) Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching. In: Proceedings of the International Conference on Representation Learning 2025 (ICLR 2025). (pp. pp. 36922-36947). ICLR

Preview

Text 6659_Improving_Probabilistic_D.pdf - Accepted Version Download (35MB) | Preview

Abstract

The probabilistic diffusion model has become highly effective across various domains. Typically, sampling from a diffusion model involves using a denoising distribution characterized by a Gaussian with a learned mean and either fixed or learned covariances. In this paper, we leverage the recently proposed covariance moment matching technique and introduce a novel method for learning the diagonal covariance. Unlike traditional data-driven diagonal covariance approximation approaches, our method involves directly regressing the optimal diagonal analytic covariance using a new, unbiased objective named Optimal Covariance Matching (OCM). This approach can significantly reduce the approximation error in covariance prediction. We demonstrate how our method can substantially enhance the sampling efficiency, recall rate and likelihood of commonly used diffusion models.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item