%0 Generic
%A Vecerik, Mel
%A Doersch, Carl
%A Yang, Yi
%A Davchev, Todor
%A Aytar, Yusuf
%A Zhou, Guangyao
%A Hadsell, Raia
%A Agapito, Lourdes
%A Scholz, Jon
%C Yokohama, Japan
%D 2024
%F discovery:10196975
%I IEEE
%K Training, Visualization, Tracking, Stacking, Production facilities, Planning, Task analysis
%P 5397-5403
%T RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation
%U https://discovery.ucl.ac.uk/id/eprint/10196975/
%V 2024
%X For robots to be useful outside labs and specialized factories we need a way to teach them new useful behaviors quickly. Current approaches lack either the generality to onboard new tasks without task-specific engineering, or else lack the data-efficiency to do so in an amount of time that enables practical use. In this work we explore dense tracking as a representational vehicle to allow faster and more general learning from demonstration. Our approach utilizes Track-Any-Point (TAP) models to isolate the relevant motion in a demonstration, and parameterize a low-level controller to reproduce this motion across changes in the scene configuration. We show this results in robust robot policies that can solve complex object-arrangement tasks such as shape-matching, stacking, and even full path-following tasks such as applying glue and sticking objects together, all from demonstrations that can be collected in minutes.
%Z This version is the author-accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.