Surface Optimization for NLOS Imaging
Surface vs Volumetric Representation of NLOS Object
Non-line-of-sight (NLOS) imaging is the problem of reconstructing properties of scenes occluded from a sensor, using measurements of light that indirectly travels from the occluded scene to the sensor through intermediate diffuse reflections. We introduce an analysis-by-synthesis framework that can reconstruct complex shape and reflectance of an NLOS object. Our framework deviates from prior work on NLOS reconstruction, by directly optimizing for a surface representation of the NLOS object, in place of commonly employed volumetric representations.
Optimized NLOS Surface Reconstruction
At the core of our framework is a new rendering formulation that efficiently computes derivatives of radiometric measurements with respect to NLOS geometry and reflectance, while accurately modeling the underlying light transport physics. By coupling this with stochastic optimization and geometry processing techniques, we are able to reconstruct NLOS surface at a level of detail significantly exceeding what is possible with previous volumetric reconstruction methods.
Physically Accurate
More Info
We can reconstruct NLOS object shape, in the form of a triangular mesh, and complex reflectance, in the form of a microfacet BRDF, while accurately taking into account the underlying light transport physics.
Computationally Efficient
More Info
This formulation enables the use of Monte Carlo rendering to efficiently estimate derivatives of radiometric measurements with respect to shape and reflectance parameters. We combine an optimized differentiable rendering implementation with stochastic optimization in an inverse rendering framework.
Finer Details
More Info
Through experiments on synthetic and measured data, we show that this pipeline can produce NLOS surface reconstructions at a level of detail comparable to what is achieved by albedo-volume methods using two orders of magnitude more measurements, while additionally recovering non-Lambertian reflectance.
Simulation Result
Surface reconstruction examples: (Top) Ground truth. (Middle) Reconstructions using the light cone transform. (Bottom) Reconstructions from our method.
Experiments with Measured Data
NLOS surface reconstruction using SPAD measurements: (Top) Diffuse object from O’Toole et.al. (Middle) A diffuse horse statue. (Bottom) Digit relief on a planar object. In our experiment, we cover the digits with white paper, to increase SNR.
People
Chia-Yin Tsai
Aswin Sankaranarayanan
Ioannis Gkioulekas