Generative Modelling of BRDF Textures from Flash Images

Abstract

We learn a latent space for easy capture, consistent interpolation, and efficient reproduction of visual material appearance. When users provide a photo of a stationary natural material captured under flash light illumination, first it is converted into a latent material code. Then, in the second step, conditioned on the material code, our method produces an infinite and diverse spatial field of BRDF model parameters (diffuse albedo, normals, roughness, specular albedo) that subsequently allows rendering in complex scenes and illuminations, matching the appearance of the input photograph. Technically, we jointly embed all flash images into a latent space using a convolutional encoder, and –conditioned on these latent codes– convert random spatial fields into fields of BRDF parameters using a convolutional neural network (CNN). We condition these BRDF parameters to match the visual characteristics (statistics and spectra of visual features) of the input under matching light. A user study compares our approach favorably to previous work, even those with access to BRDF supervision.

Result overview

Interactive Demo (click here)

The visual quality is best inspected from our interactive WebGL demo. It allows exploring the space by relighting, changing the random seed and visualizing individual BRDF model channels and their combinations. Furthermore, it contains competitor comparisons and lets you pick materials for interpolation as well.

Resynthesis

Input Relit

Diversity

As we can re-seed the input noise our method is capable of producing diverse results.

Input Re-seed

Infinite spatial domain

Our method allows to genreate infinite spatial fields of BRDF parameters without any border artefacts or repetitive patterns.

Interpolation

We can interpolate different exemplars in latent space.

Exemplar 1 Interpolation Exemplar 2

BRDF Decomposition

Resynthesis Diffuse Specular Roughness Normal

Short talk

Related