Liu, C;
Ritschel, T;
(2024)
Neural Differential Appearance Equations.
ACM Transactions on Graphics
, 43
(6)
, Article 256. 10.1145/3687900.
Preview |
PDF
Neural_Differential_Appearance_Equations_no_acm_traces.pdf - Accepted Version Download (24MB) | Preview |
Abstract
We propose a method to reproduce dynamic appearance textures with space-stationary but time-varying visual statistics. While most previous work decomposes dynamic textures into static appearance and motion, we focus on dynamic appearance that results not from motion but variations of fundamental properties, such as rusting, decaying, melting, and weathering. To this end, we adopt the neural ordinary differential equation (ODE) to learn the underlying dynamics of appearance from a target exemplar. We simulate the ODE in two phases. At the "warm-up"phase, the ODE diffuses a random noise to an initial state. We then constrain the further evolution of this ODE to replicate the evolution of visual feature statistics in the exemplar during the generation phase. The particular innovation of this work is the neural ODE achieving both denoising and evolution for dynamics synthesis, with a proposed temporal training scheme. We study both relightable (BRDF) and non-relightable (RGB) appearance models. For both we introduce new pilot datasets, allowing, for the first time, to study such phenomena: For RGB we provide 22 dynamic textures acquired from free online sources; For BRDFs, we further acquire a dataset of 21 flash-lit videos of time-varying materials, enabled by a simple-to-construct setup. Our experiments show that our method consistently yields realistic and coherent results, whereas prior works falter under pronounced temporal appearance variations. A user study confirms our approach is preferred to previous work for such exemplars.
| Type: | Article |
|---|---|
| Title: | Neural Differential Appearance Equations |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1145/3687900 |
| Publisher version: | https://doi.org/10.1145/3687900 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Science & Technology, Technology, Computer Science, Software Engineering, Computer Science, Material Appearance, Dynamic Texture, Dynamic (sv)BRDF, Neural Differential Equation, Video, ILLUMINATION, REFLECTANCE, SIMULATION |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10203234 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
