Vecerik, Mel; Doersch, Carl; Yang, Yi; Davchev, Todor; Aytar, Yusuf; Zhou, Guangyao; Hadsell, Raia; ... Scholz, Jon; + view all Vecerik, Mel; Doersch, Carl; Yang, Yi; Davchev, Todor; Aytar, Yusuf; Zhou, Guangyao; Hadsell, Raia; Agapito, Lourdes; Scholz, Jon; - view fewer (2024) RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation. In: Proceedings - IEEE International Conference on Robotics and Automation. (pp. pp. 5397-5403). IEEE: Yokohama, Japan.

Preview

PDF robotapicra24.pdf - Published Version Download (19MB) | Preview

Abstract

For robots to be useful outside labs and specialized factories we need a way to teach them new useful behaviors quickly. Current approaches lack either the generality to onboard new tasks without task-specific engineering, or else lack the data-efficiency to do so in an amount of time that enables practical use. In this work we explore dense tracking as a representational vehicle to allow faster and more general learning from demonstration. Our approach utilizes Track-Any-Point (TAP) models to isolate the relevant motion in a demonstration, and parameterize a low-level controller to reproduce this motion across changes in the scene configuration. We show this results in robust robot policies that can solve complex object-arrangement tasks such as shape-matching, stacking, and even full path-following tasks such as applying glue and sticking objects together, all from demonstrations that can be collected in minutes.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item