Meunier, Dimitri; Shen, Zikai; Mollenhauer, Mattes; Gretton, Arthur; Li, Zhu; (2024) Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms. In: Globerson, A and Mackey, L and Belgrave, D and Fan, A and Paquet, U and Tomczak, J and Zhang, C, (eds.) Advances in Neural Information Processing Systems 37 (NeurIPS 2024). (pp. pp. 1-46). NeurIPS: Vancouver, BC, Canada.

Preview

Text NeurIPS-2024-optimal-rates-for-vector-valued-spectral-regularization-learning-algorithms-Paper-Conference.pdf - Published Version Download (623kB) | Preview

Abstract

We study theoretical properties of a broad class of regularized algorithms with vector-valued output. These spectral algorithms include kernel ridge regression, kernel principal component regression and various implementations of gradient descent. Our contributions are twofold. First, we rigorously confirm the so-called saturation effect for ridge regression with vector-valued output by deriving a novel lower bound on learning rates; this bound is shown to be suboptimal when the smoothness of the regression function exceeds a certain level.Second, we present an upper bound on the finite sample risk for general vector-valued spectral algorithms, applicable to both well-specified and misspecified scenarios (where the true regression function lies outside of the hypothesis space), and show that this bound is minimax optimal in various regimes. All of our results explicitly allow the case of infinite-dimensional output variables, proving consistency of recent practical applications.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item