UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Efficient Tactile Sensing-based Learning from Limited Real-world Demonstrations for Dual-arm Fine Pinch-Grasp Skills

Mao, X; Xu, Y; Wen, R; Kasaei, M; Yu, W; Psomopoulou, E; Lepora, NF; (2024) Efficient Tactile Sensing-based Learning from Limited Real-world Demonstrations for Dual-arm Fine Pinch-Grasp Skills. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024. (pp. pp. 5112-5119). IEEE Green open access

[thumbnail of Tactip_LfD_RAL (1).pdf]
Preview
Text
Tactip_LfD_RAL (1).pdf - Accepted Version

Download (4MB) | Preview

Abstract

Imitation learning for robot dexterous manipulation, especially with a real robot setup, typically requires a large number of demonstrations. In this paper, we present a data-efficient learning from demonstration framework which exploits the use of rich tactile sensing data and achieves fine bimanual pinch grasping. Specifically, we employ a convolutional autoencoder network that can effectively extract and encode high-dimensional tactile information. Further, we develop a framework that achieves efficient multi-sensor fusion for imitation learning, allowing the robot to learn contact-aware sensorimotor skills from demonstrations. The ablation studies on encoded tactile features highlighted the effectiveness of incorporating rich contact information, which enabled dexterous bimanual grasping with active contact searching. Extensive experiments demonstrated the robustness of the fine pinch grasp policy directly learned from few-shot demonstration, including grasping of the same object with different initial poses, generalizing to ten unseen new objects, robust and firm grasping against external pushes, as well as contact-aware and reactive re-grasping in case of dropping objects under very large perturbations. Furthermore, the saliency map analysis method is used to describe weight distribution across various modalities during pinch grasping, confirming the effectiveness of our framework at leveraging multimodal information. The video is available online at: https://youtu.be/BlzxGgiKfck.

Type: Proceedings paper
Title: Efficient Tactile Sensing-based Learning from Limited Real-world Demonstrations for Dual-arm Fine Pinch-Grasp Skills
Event: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Location: Abu Dhabi, United Arab Emirates
Dates: 14th-18th October 2024
ISBN-13: 979-8-3503-7771-2
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/IROS58592.2024.10802651
Publisher version: https://doi.org/10.1109/iros58592.2024.10802651
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10210062
Downloads since deposit
28Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item