Tsagkas, N;
Rome, J;
Ramamoorthy, S;
Aodha, OM;
Lu, CX;
(2024)
Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors.
In:
IEEE International Conference on Intelligent Robots and Systems.
(pp. pp. 11610-11617).
IEEE
Preview |
Text
Click to Grasp - Zero-Shot Precise Manipulation via Visual Diffusion Descriptors.pdf - Accepted Version Download (4MB) | Preview |
Abstract
Precise manipulation that is generalizable across scenes and objects remains a persistent challenge in robotics. Current approaches for this task heavily depend on having a significant number of training instances to handle objects with pronounced visual and/or geometric part ambiguities. Our work explores the grounding of fine-grained part descriptors for precise manipulation in a zero-shot setting by utilizing web-trained text-to-image diffusion-based generative models. We tackle the problem by framing it as a dense semantic part correspondence task. Our model returns a gripper pose for manipulating a specific part, using as reference a user-defined click from a source image of a visually different instance of the same object. We require no manual grasping demonstrations as we leverage the intrinsic object geometry and features. Practical experiments in a real-world tabletop scenario validate the efficacy of our approach, demonstrating its potential for advancing semantic-aware robotics manipulation.Web page: https://tsagkas.github.io/click2grasp
| Type: | Proceedings paper |
|---|---|
| Title: | Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors |
| Event: | 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) |
| Dates: | 14 Oct 2024 - 18 Oct 2024 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/IROS58592.2024.10801488 |
| Publisher version: | https://doi.org/10.1109/iros58592.2024.10801488 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10206284 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
