Path-Space Differentiable Rendering

Physics-based Differentiable Rendering

Physics-based differentiable rendering, the estimation of derivatives of radiometric measures with respect to arbitrary scene parameters, has a diverse array of applications from solving analysis-by-synthesis problems to training machine learning pipelines incorporating forward rendering processes. Unfortunately, general-purpose differentiable rendering remains challenging due to the lack of efficient estimators as well as the need to identify and handle complex discontinuities such as visibility boundaries.

Project Site

Paper

New Differentiable Rendering

In this project, we show how path integrals can be differentiated with respect to arbitrary differentiable changes of a scene. We provide a detailed theoretical analysis of this process and establish new differentiable rendering formulations based on the resulting differential path integrals. Our path space differentiable rendering formulation allows the design of new Monte Carlo estimators that offer significantly better efficiency than state-of-the-art methods in handling complex geometric discontinuities and light transport phenomena such as caustics.



Differentiable Rendering

More Info

Differentiable rendering computes the derivatives of radiometric measurements with respect to differential changes of such environments. These techniques can enable, for example (i) gradient-based optimization when solving inverse-rendering problems; and (ii) efficient integration of physics-based light transport simulation in machine learning and probabilistic inference pipelines.



Efficient Monte Carlo Estimation

More Info

The differentiation of full path integrals with respect to arbitrary scene parameterizations, resulting in our differential path integrals comprised of completely separated interior and boundary components that can be estimated independently using different Monte Carlo estimators.

