Kernel-Predicting Convolutional Networks
for Denoising Monte Carlo Renderings

Proceedings of SIGGRAPH 2017

ACM Transactions on Graphics (TOG)
Vol. 36, No. 4, July 2017

     Steve Bako^{* 1}         Thijs Vogels^{* 2,3}         Brian McWilliams ³
Mark Meyer ⁴           Jan Novák ³                   Alex Harvill ⁴
      Pradeep Sen ¹        Tony DeRose ⁴           Fabrice Rousselle ³

¹ University of California, Santa Barbara         ² ETH Zurich         ³ Disney Research         ⁴ Pixar Animation Studios         

Abstract

Regression-based algorithms have shown to be good at denoising Monte Carlo (MC) renderings by leveraging its inexpensive by-products (e.g., feature buffers). However, when using higher-order models to handle complex cases, these techniques often overfit to noise in the input. For this reason, supervised learning methods have been proposed that train on a large collection of reference examples, but they use explicit filters that limit their denoising ability. To address these problems, we propose a novel, supervised learning approach that allows the filtering kernel to be more complex and general by leveraging a deep convolutional neural network (CNN) architecture. In one embodiment of our framework, the CNN directly predicts the final denoised pixel value as a highly non-linear combination of the input features. In a second approach, we introduce a novel, kernel-prediction network which uses the CNN to estimate the local weighting kernels used to compute each denoised pixel from its neighbors. We train and evaluate our networks on production data and observe improvements over state-of-the-art MC denoisers, showing that our methods generalize well to a variety of scenes. We conclude by analyzing various components of our architecture and identify areas of further research in deep learning for MC denoising.

Press

Paper and Additional Materials

Paper 111 MB

Lo-res PDF 13 MB

Renderman supplemental 1 GB

Tungsten supplemental with DPCN [NEW!] 180 MB

Network code 1.2 GB

DPCN Weights [NEW!] 33 MB

Tungsten code 29 MB

External URL

Bibtex

@article{Bako17,
	author 	= {Bako, Steve and Vogels, Thijs and McWilliams, Brian and Meyer, Mark and Nov\'ak, 
		   Jan and Harvill, Alex and Sen, Pradeep and DeRose, Tony and Rousselle, Fabrice},
	title	= {Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings},
	journal	= {ACM Transactions on Graphics (TOG) (Proceedings of SIGGRAPH 2017)},
	volume	= {36},
	number 	= {4},
	article	= {97},
	month	= {July},
	year	= {2017},
}

^*Joint first authors

© 2017 Copyright held by the owner/author(s): Disney / Pixar.

The definitive version of this work was published in ACM TOG and can be accessed through "External URL."

The source code provided on this website is for non-commercial, research purposes only.

This material is partially supported by the National Science Foundation under grants #13-21168 and #16-19376. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.