Starting from an exemplar (top-left) our trained encoder encodes the image to a compact latent space variable z. Additionally, a random infinite field is cropped with the same spatial dimensions as the flash input image. The noise crop is then reshaped based on a convolutional U-Net architecture. Each convolution in the network is followed by an Adaptive Instance Normalization (AdaIN) layer reshaping the statistics (mean and standard deviation) of features. A learned affine transformation T per layer maps z to the desired means and sigmas. The output of the network are the diffuse, specular, roughness, normal parameters of an svBRDF that, when rendered using a camera colocated flash light, look the same as the input. Our unsupervised setting allows us to fine-tune our trained network on materials to acquire.
As we can re-seed the input noise our method is capable of producing diverse results.
Our method allows to genreate infinite spatial fields of BRDF parameters without any border artefacts or repetitive patterns.
Our method allows to genreate infinite spatial fields of BRDF parameters without any border artefacts or repetitive patterns.
Our method allows to genreate infinite spatial fields of BRDF parameters without any border artefacts or repetitive patterns.
@article{henzler2021neuralmaterials,
title = {Generative modelling of BRDF textures from flash images},
author = {Henzler, Philipp and Deschaintre, Valentin and Mitra, Niloy J and Ritschel, Tobias},
journal = {SIGGRAPH Asia},
year = {2021},
}