?url_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Adc&rft.title=Distributionally+Robust+Model-based+Reinforcement+Learning+with+Large+State+Spaces&rft.creator=Ramesh%2C+Shyam+Sundhar&rft.creator=Sessa%2C+Pier+Giuseppe&rft.creator=Hu%2C+Yifan&rft.creator=Krause%2C+Andreas&rft.creator=Bogunovic%2C+Ilija&rft.description=Three+major+challenges+in+reinforcement+learning+are+the+complex+dynamical+systems+with+large+state+spaces%2C+the+costly+data+acquisition+processes%2C+and+the+deviation+of+real-world+dynamics+from+the+training+environment+deployment.+To+overcome+these+issues%2C+we+study+distributionally+robust+Markov+decision+processes+with+continuous+state+spaces+under+the+widely+used+Kullback%E2%80%93Leibler%2C+chi-square%2C+and+total+variation+uncertainty+sets.+We+propose+a+model-based+approach+that+utilizes+Gaussian+Processes+and+the+maximum+variance+reduction+algorithm+to+efficiently+learn+multi-output+nominal+transition+dynamics%2C+leveraging+access+to+a+generative+model+(i.e.%2C+simulator).+We+further+demonstrate+the+statistical+sample+complexity+of+the+proposed+method+for+different+uncertainty+sets.+These+complexity+bounds+are+independent+of+the+number+of+states+and+extend+beyond+linear+dynamics%2C+ensuring+the+effectiveness+of+our+approach+in+identifying+near-optimal+distributionally-robust+policies.+The+proposed+method+can+be+further+combined+with+other+model-free+distributionally+robust+reinforcement+learning+methods+to+obtain+a+near-optimal+robust+policy.+Experimental+results+demonstrate+the+robustness+of+our+algorithm+to+distributional+shifts+and+its+superior+performance+in+terms+of+the+number+of+samples+needed.&rft.publisher=PMLR&rft.contributor=Dasgupta%2C+S&rft.contributor=Mandt%2C+S&rft.contributor=Li%2C+Y&rft.date=2024&rft.type=Proceedings+paper&rft.language=eng&rft.source=+++++In%3A+Dasgupta%2C+S+and+Mandt%2C+S+and+Li%2C+Y%2C+(eds.)+Proceedings+of+The+27th+International+Conference+on+Artificial+Intelligence+and+Statistics.++(pp.+pp.+1-42).++PMLR+(2024)+++++&rft.format=text&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10198782%2F1%2Fsundhar-ramesh24a.pdf&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10198782%2F&rft.rights=open