eprintid: 10178561 rev_number: 13 eprint_status: archive userid: 699 dir: disk0/10/17/85/61 datestamp: 2023-10-11 10:13:04 lastmod: 2024-03-08 14:01:14 status_changed: 2024-03-08 14:01:14 type: proceedings_section metadata_visibility: show sword_depositor: 699 creators_name: Long, Xiang creators_name: Beddow, Luke creators_name: Hadjivelichkov, Denis creators_name: Delfaki, Andromachi Maria creators_name: Wurdemann, Helge creators_name: Kanoulas, Dimitrios title: Reinforcement Learning-based Grasping via One-Shot Affordance Localization and Zero-Shot Contrastive Language–Image Learning ispublished: pub divisions: UCL divisions: B04 divisions: C05 divisions: F48 keywords: Location awareness, Affordances, Pipelines, Grasping, System integration, Robots, Videos note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. abstract: We present a novel robotic grasping system using a caging-style gripper, that combines one-shot affordance localization and zero-shot object identification. We demonstrate an integrated system requiring minimal prior knowledge, focusing on flexible few-shot object agnostic approaches. For grasping a novel target object, we use as input the color and depth of the scene, an image of an object affordance similar to the target object, and an up to three-word text prompt describing the target object. We demonstrate the system using real-world grasping of objects from the YCB benchmark set, with four distractor objects cluttering the scene. Overall, our pipeline has a success rate of the affordance localization of 96%, object identification of 62.5%, and grasping of 72%. Videos are on the project website: https://sites.google.com/view/ rl-affcorrs-grasp. date: 2024-01-08 date_type: published publisher: IEEE official_url: https://doi.org/10.1109/SII58957.2024.10417178 oa_status: green full_text_type: other language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 2095337 doi: 10.1109/SII58957.2024.10417178 lyricists_name: Kanoulas, Dimitrios lyricists_id: DKANO15 actors_name: Kanoulas, Dimitrios actors_id: DKANO15 actors_role: owner full_text_status: public pres_type: paper place_of_pub: Ha Long, Vietnam event_title: The 2024 16th IEEE/SICE International Symposium on System Integration event_location: Ha Long, Vietnam event_dates: 8 Jan 2024 - 11 Jan 2024 book_title: Proceedings of the 2024 IEEE/SICE International Symposium on System Integration (SII) citation: Long, Xiang; Beddow, Luke; Hadjivelichkov, Denis; Delfaki, Andromachi Maria; Wurdemann, Helge; Kanoulas, Dimitrios; (2024) Reinforcement Learning-based Grasping via One-Shot Affordance Localization and Zero-Shot Contrastive Language–Image Learning. In: Proceedings of the 2024 IEEE/SICE International Symposium on System Integration (SII). IEEE: Ha Long, Vietnam. Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10178561/1/sii_2024_xiang.pdf