Koppula, Skanda;
Rocco, Ignacio;
Yang, Yi;
Heyward, Joe;
Carreira, João;
Zisserman, Andrew;
Brostow, Gabriel;
(2024)
TAPVid-3D: A Benchmark for Tracking Any Point in 3D.
In: Globerson, A and Mackey, L and Belgrave, D and Fan, A and Paquet, U and Tomczak, J and Zhang, C, (eds.)
Advances in Neural Information Processing Systems 37 (NeurIPS 2024).
(pp. pp. 1-17).
NeurIPS: San Diego, CA, USA.
Preview |
Text
NeurIPS-2024-tapvid-3d-a-benchmark-for-tracking-any-point-in-3d-Paper-Datasets_and_Benchmarks_Track.pdf - Published Version Download (6MB) | Preview |
Abstract
We introduce a new benchmark, TAPVid-3D, for evaluating the task of long-range Tracking Any Point in 3D (TAP-3D). While point tracking in two dimensions (TAP-2D) has many benchmarks measuring performance on real-world videos, such as TAPVid-DAVIS, three-dimensional point tracking has none. To this end, leveraging existing footage, we build a new benchmark for 3D point tracking featuring 4,000+ real-world videos, composed of three different data sources spanning a variety of object types, motion patterns, and indoor and outdoor environments. To measure performance on the TAP-3D task, we formulate a collection of metrics that extend the Jaccard-based metric used in TAP-2D to handle the complexities of ambiguous depth scales across models, occlusions, and multi-track spatio-temporal smoothness. We manually verify a large sample of trajectories to ensure correct video annotations, and assess the current state of the TAP-3D task by constructing competitive baselines using existing tracking models. We anticipate this benchmark will serve as a guidepost to improve our ability to understand precise 3D motion and surface deformation from monocular video.
| Type: | Proceedings paper |
|---|---|
| Title: | TAPVid-3D: A Benchmark for Tracking Any Point in 3D |
| Event: | 38th Conference on Neural Information Processing Systems (NeurIPS 2024) |
| ISBN-13: | 9798331314385 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://papers.nips.cc/paper_files/paper/2024/hash... |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10207451 |
Archive Staff Only
 |
View Item |
