Kayal, Aya; Vakili, Sattar; Toni, Laura; Bernacchia, Alberto; (2025) Near-Optimal Sample Complexity in Reward-Free Kernel-based Reinforcement Learning. In: Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz, (eds.) Proceedings of the 28th International Conference on Artificial Intelligence and Statistics. (pp. pp. 874-882). PMLR (Proceedings of Machine Learning Research): Mai Khao, Thailand.

Preview

Text Kayal_25a.pdf Download (1MB) | Preview

Abstract

Reinforcement Learning (RL) problems are being considered under increasingly more complex structures. While tabular and linear models have been thoroughly explored, the analytical study of RL under non-linear function approximation, especially kernel-based models, has recently gained traction for their strong representational capacity and theoretical tractability. In this context, we examine the question of statistical efficiency in kernel-based RL within the reward-free RL framework, specifically asking: how many samples are required to design a near-optimal policy? Existing work addresses this question under restrictive assumptions about the class of kernel functions. We first explore this question assuming a generative model, then relax this assumption at the cost of increasing the sample complexity by a factor of H, the episode length. We tackle this fundamental problem using a broad class of kernels and a simpler algorithm compared to prior work. Our approach derives new confidence intervals for kernel ridge regression, specific to our RL setting, that may be of broader applicability. We further validate our theoretical findings through simulations.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item