Huang, Kevin Han; Liu, Xing; Duncan, Andrew B; Gandy, Axel; (2023) A High-dimensional Convergence Theorem for U-statistics with Applications to Kernel-based Testing. In: Neu, Gergely and Rosasco, Lorenzo, (eds.) Proceedings of Thirty Sixth Conference on Learning Theory. (pp. pp. 3827-3918). Proceedings of Machine Learning Research (PMLR): Bangalore, India.

Preview

Text Huang_23a.pdf Download (3MB) | Preview

Abstract

We prove a convergence theorem for U-statistics of degree two, where the data dimension d is allowed to scale with sample size n . We find that the limiting distribution of a U-statistic undergoes a phase transition from the non-degenerate Gaussian limit to the degenerate limit, regardless of its degeneracy and depending only on a moment ratio. A surprising consequence is that a non-degenerate U-statistic in high dimensions can have a non-Gaussian limit with a larger variance and asymmetric distribution. Our bounds are valid for any finite n and d , independent of individual eigenvalues of the underlying function, and dimension-independent under a mild assumption. As an application, we apply our theory to two popular kernel-based distribution tests, MMD and KSD, whose high-dimensional performance has been challenging to study. In a simple empirical setting, our results correctly predict how the test power at a fixed threshold scales with d and the bandwidth.

Download activity - last month

Export as XMLJSONCSV

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item