Donato, E;
Falotico, E;
Thuruthel, TG;
(2024)
Multi-Modal Perception for Soft Robotic Interactions Using Generative Models.
In:
Proceedings of the 7th International Conference on Soft Robotics (RoboSoft) IEEE 2024.
(pp. pp. 311-318).
Institute of Electrical and Electronics Engineers (IEEE)
Preview |
Text
2404.04220v1.pdf - Accepted Version Download (4MB) | Preview |
Abstract
Perception is essential for the active interaction of physical agents with the external environment. The integration of multiple sensory modalities, such as touch and vision, enhances this perceptual process, creating a more comprehensive and robust understanding of the world. Such fusion is particularly useful for highly deformable bodies such as soft robots. Developing a compact, yet comprehensive state representation from multi-sensory inputs can pave the way for the development of complex control strategies. This paper introduces a perception model that harmonizes data from diverse modalities to build a holistic state representation and assimilate essential information. The model relies on the causality between sensory input and robotic actions, employing a generative model to efficiently compress fused information and predict the next observation. We present, for the first time, a study on how touch can be predicted from vision and proprioception on soft robots, the importance of the cross-modal generation and why this is essential for soft robotic interactions in unstructured environments.
Type: | Proceedings paper |
---|---|
Title: | Multi-Modal Perception for Soft Robotic Interactions Using Generative Models |
Event: | 2024 IEEE 7th International Conference on Soft Robotics (RoboSoft) |
Location: | San Diego, CA, USA |
Dates: | 14th-17th April 2024 |
ISBN-13: | 979-8-3503-8181-8 |
Open access status: | An open access version is available from UCL Discovery |
DOI: | 10.1109/RoboSoft60065.2024.10522007 |
Publisher version: | http://dx.doi.org/10.1109/robosoft60065.2024.10522... |
Language: | English |
Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions. |
UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
URI: | https://discovery.ucl.ac.uk/id/eprint/10193301 |
Archive Staff Only
View Item |