UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Interactive videos: Plausible video editing using sparse structure points

Chang, CS; Chu, HK; Mitra, NJ; (2016) Interactive videos: Plausible video editing using sparse structure points. Computer Graphics Forum , 35 (2) pp. 489-500. 10.1111/cgf.12849. Green open access

[img]
Preview
Text
Chang_Interactive_videos.pdf - ["content_typename_Accepted version" not defined]

Download (39MB) | Preview

Abstract

Video remains the method of choice for capturing temporal events. However, without access to the underlying 3D scene models, it remains difficult to make object level edits in a single video or across multiple videos. While it may be possible to explicitly reconstruct the 3D geometries to facilitate these edits, such a workflow is cumbersome, expensive, and tedious. In this work, we present a much simpler workflow to create plausible editing and mixing of raw video footage using only sparse structure points (SSP) directly recovered from the raw sequences. First, we utilize user-scribbles to structure the point representations obtained using structure-from-motion on the input videos. The resultant structure points, even when noisy and sparse, are then used to enable various video edits in 3D, including view perturbation, keyframe animation, object duplication and transfer across videos, etc. Specifically, we describe how to synthesize object images from new views adopting a novel image-based rendering technique using the SSPs as proxy for the missing 3D scene information. We propose a structure-preserving image warping on multiple input frames adaptively selected from object video, followed by a spatio-temporally coherent image stitching to compose the final object image. Simple planar shadows and depth maps are synthesized for objects to generate plausible video sequence mimicking real-world interactions. We demonstrate our system on a variety of input videos to produce complex edits, which are otherwise difficult to achieve.

Type: Article
Title: Interactive videos: Plausible video editing using sparse structure points
Open access status: An open access version is available from UCL Discovery
DOI: 10.1111/cgf.12849
Publisher version: http://dx.doi.org/10.1111/cgf.12849
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: http://discovery.ucl.ac.uk/id/eprint/1499648
Downloads since deposit
33Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item