Long, Xiang; Beddow, Luke; Hadjivelichkov, Denis; Delfaki, Andromachi Maria; Wurdemann, Helge; Kanoulas, Dimitrios; (2024) Reinforcement Learning-based Grasping via One-Shot Affordance Localization and Zero-Shot Contrastive Language–Image Learning. In: Proceedings of the 2024 IEEE/SICE International Symposium on System Integration (SII). IEEE: Ha Long, Vietnam.

Preview

Text sii_2024_xiang.pdf - Other Download (16MB) | Preview

Abstract

We present a novel robotic grasping system using a caging-style gripper, that combines one-shot affordance localization and zero-shot object identification. We demonstrate an integrated system requiring minimal prior knowledge, focusing on flexible few-shot object agnostic approaches. For grasping a novel target object, we use as input the color and depth of the scene, an image of an object affordance similar to the target object, and an up to three-word text prompt describing the target object. We demonstrate the system using real-world grasping of objects from the YCB benchmark set, with four distractor objects cluttering the scene. Overall, our pipeline has a success rate of the affordance localization of 96%, object identification of 62.5%, and grasping of 72%. Videos are on the project website: https://sites.google.com/view/ rl-affcorrs-grasp.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item