Watson, J; Vicente, S; Aodha, OM; Godard, C; Brostow, G; Firman, M; (2023) Heightfields for Efficient Scene Reconstruction for AR. In: Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023. (pp. pp. 5839-5849). IEEE: Waikoloa, HI, USA.

Preview

Text Heightfields for Efficient Scene Reconstruction for AR.pdf - Accepted Version Download (6MB) | Preview

Abstract

3D scene reconstruction from a sequence of posed RGB images is a cornerstone task for computer vision and augmented reality (AR). While depth-based fusion is the foundation of most real-time approaches for 3D reconstruction, recent learning based methods that operate directly on RGB images can achieve higher quality reconstructions, but at the cost of increased runtime and memory requirements, making them unsuitable for AR applications. We propose an efficient learning-based method that refines the 3D reconstruction obtained by a traditional fusion approach. By leveraging a top-down heightfield representation, our method remains real-time while approaching the quality of other learning-based methods. Despite being a simplification, our heightfield is perfectly appropriate for robotic path planning or augmented reality character placement. We outline several innovations that push the performance beyond existing top-down prediction baselines, and we present an evaluation framework on the challenging ScanNetV2 dataset, targeting AR tasks.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as CSVJSONXML

Archive Staff Only

View Item