eprintid: 10197138 rev_number: 10 eprint_status: archive userid: 699 dir: disk0/10/19/71/38 datestamp: 2024-09-18 15:28:59 lastmod: 2024-09-18 15:28:59 status_changed: 2024-09-18 15:28:59 type: proceedings_section metadata_visibility: show sword_depositor: 699 creators_name: Triantafyllidis, E creators_name: Christianos, F creators_name: Li, Z title: Intrinsic Language-Guided Exploration for Complex Long-Horizon Robotic Manipulation Tasks ispublished: pub divisions: UCL divisions: B04 divisions: F48 note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. abstract: Current reinforcement learning algorithms struggle in sparse and complex environments, most notably in long-horizon manipulation tasks entailing a plethora of different sequences. In this work, we propose the Intrinsically Guided Exploration from Large Language Models (IGE-LLMs) framework. By leveraging LLMs as an assistive intrinsic reward, IGE-LLMs guides the exploratory process in reinforcement learning to address intricate long-horizon with sparse rewards robotic manipulation tasks. We evaluate our framework and related intrinsic learning methods in an environment challenged with exploration, and a complex robotic manipulation task challenged by both exploration and long-horizons. Results show IGE-LLMs (i) exhibit notably higher performance over related intrinsic methods and the direct use of LLMs in decision-making, (ii) can be combined and complement existing learning methods highlighting its modularity, (iii) are fairly insensitive to different intrinsic scaling parameters, and (iv) maintain robustness against increased levels of uncertainty and horizons. date: 2024-08-08 date_type: published publisher: Institute of Electrical and Electronics Engineers (IEEE) official_url: http://dx.doi.org/10.1109/icra57147.2024.10611483 oa_status: green full_text_type: other language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 2311968 doi: 10.1109/ICRA57147.2024.10611483 isbn_13: 979-8-3503-8457-4/24 lyricists_name: Li, Zhibin lyricists_id: ZLISX72 actors_name: Flynn, Bernadette actors_id: BFFLY94 actors_role: owner full_text_status: public pres_type: paper publication: Proceedings - IEEE International Conference on Robotics and Automation volume: 70 pagerange: 7493-7500 event_title: IEEE International Conference on Robotics and Automation (ICRA) 2024 event_location: Yokohama, Japan event_dates: 13th-17th May 2024 book_title: Proceedings - IEEE International Conference on Robotics and Automation citation: Triantafyllidis, E; Christianos, F; Li, Z; (2024) Intrinsic Language-Guided Exploration for Complex Long-Horizon Robotic Manipulation Tasks. In: Proceedings - IEEE International Conference on Robotics and Automation. (pp. pp. 7493-7500). Institute of Electrical and Electronics Engineers (IEEE) Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10197138/1/2309.16347v2.pdf