eprintid: 10196975 rev_number: 8 eprint_status: archive userid: 699 dir: disk0/10/19/69/75 datestamp: 2024-09-16 10:09:44 lastmod: 2024-09-16 10:10:10 status_changed: 2024-09-16 10:09:44 type: proceedings_section metadata_visibility: show sword_depositor: 699 creators_name: Vecerik, Mel creators_name: Doersch, Carl creators_name: Yang, Yi creators_name: Davchev, Todor creators_name: Aytar, Yusuf creators_name: Zhou, Guangyao creators_name: Hadsell, Raia creators_name: Agapito, Lourdes creators_name: Scholz, Jon title: RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation ispublished: pub divisions: UCL divisions: B04 divisions: F48 keywords: Training, Visualization, Tracking, Stacking, Production facilities, Planning, Task analysis note: This version is the author-accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. abstract: For robots to be useful outside labs and specialized factories we need a way to teach them new useful behaviors quickly. Current approaches lack either the generality to onboard new tasks without task-specific engineering, or else lack the data-efficiency to do so in an amount of time that enables practical use. In this work we explore dense tracking as a representational vehicle to allow faster and more general learning from demonstration. Our approach utilizes Track-Any-Point (TAP) models to isolate the relevant motion in a demonstration, and parameterize a low-level controller to reproduce this motion across changes in the scene configuration. We show this results in robust robot policies that can solve complex object-arrangement tasks such as shape-matching, stacking, and even full path-following tasks such as applying glue and sticking objects together, all from demonstrations that can be collected in minutes. date: 2024-08-08 date_type: published publisher: IEEE official_url: https://doi.org/10.1109/ICRA57147.2024.10611409 oa_status: green full_text_type: other language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 2311663 doi: 10.1109/ICRA57147.2024.10611409 lyricists_name: De Agapito Vicente, Lourdes lyricists_id: LDEAG40 actors_name: De Agapito Vicente, Lourdes actors_id: LDEAG40 actors_role: owner full_text_status: public pres_type: paper series: IEEE International Conference on Robotics and Automation (ICRA) publication: Proceedings - IEEE International Conference on Robotics and Automation volume: 2024 place_of_pub: Yokohama, Japan pagerange: 5397-5403 event_title: 2024 IEEE International Conference on Robotics and Automation (ICRA) event_dates: 13 May 2024 - 17 May 2024 issn: 1050-4729 book_title: Proceedings - IEEE International Conference on Robotics and Automation citation: Vecerik, Mel; Doersch, Carl; Yang, Yi; Davchev, Todor; Aytar, Yusuf; Zhou, Guangyao; Hadsell, Raia; ... Scholz, Jon; + view all <#> Vecerik, Mel; Doersch, Carl; Yang, Yi; Davchev, Todor; Aytar, Yusuf; Zhou, Guangyao; Hadsell, Raia; Agapito, Lourdes; Scholz, Jon; - view fewer <#> (2024) RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation. In: Proceedings - IEEE International Conference on Robotics and Automation. (pp. pp. 5397-5403). IEEE: Yokohama, Japan. Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/10196975/1/robotapicra24.pdf