Transient rendering simulates light in motion. However, the stochastic nature of Monte Carlo is aggravated in transient rendering, since samples are now spread along the temporal domain.

We propose to denoise transient Monte Carlo renders by exploiting the spatio-temporal correlation of transient light transport, extending a recent statistical denoising formulation to achieve a near-optimal trade-off between reduced variance and introduced bias.

Paper Code

|

Statistical Spatio-Temporal Denoising

Our method works by exploiting the spatio-temporal correlation of transient light transport, operating in two stages: First, we collect statistics (as central moments) throughout the transietn render, and the statistics of nearby spatio-temporal information to decide which transient bins are worth combining to reduce noise while minimizing bias.

The "membership function" decides (based on statistics) which bins are worth combining, and the "base filter" is a standard spatio-temporal filter known to reduce variance (i.e. Gaussian filter).

While the raw render is noisy and the base filter adds a significant amount of bias, our weights achieve a near-optimal trade-off between bias and noise.

We compare our statistics-based spatio-temporal transient denoiser against a Joint Bilateral Filter (JBF) (the base filter used in our method), OptiX (NVIDIA OptiX AI-Accelerated Denoiser) and OIDN (Intel's Open Image Denoise). We use the RMSE to compare each method with the reference, and each inset shows a different time bin. JBF blurs out most of the image details, to the point of eliminating many scene features. Learning-based methods aim to obtain more visually pleasant results at the cost of introducing bias (see e.g. the mirror reflection in Bathroom or the shadow in Kitchen), and hallucinating details (such as the caustic discontinuity). The bath inset is rendered with 128 spp.

Related Work

Acknowledgements

This work has received funding from the European Union’s EUROPEAN DEFENSE FUND under grant agreement No 101103242. The views and opinions expressed herein are those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission; neither the European Union nor the granting authority can be held responsible for them. Additionally, Oscar Pueyo-Ciutad was supported by the FPU22/02432 predoctoral grant. Thanks to Guillermo Enguita, Maria Pena, Alfonso Lopez, Diego Royo and Jorge Garcia-Pueyo from the Graphics and Imaging Lab for their help in preparing scenes and figures, and proofreading the manuscript.