PhaseCam3D

Passive Single-View Depth Sensing

3DImaging is critical for a myriad of applications such as autonomous driving, robotics, virtual reality, and surveillance. The current state of art relies on active illumination based techniques such as LIDAR, radar, structured illumination or continuous-wave time-of-flight. However, many emerging applications, especially on mobile platforms, are severely power and energy constrained. Active approaches are unlikely to scale well for these applications and hence, there is a pressing need for robust passive 3D imaging technologies.

Multi-camera systems provide state of the art performance for passive 3D imaging. In these systems, triangulation between corresponding points on multiple views of the scene allows for 3D estimation. Stereo and multi-view stereo approaches meet some of the needs mentioned above, and an increasing number of mobile platforms have been adopting such technology. Unfortunately, having multiple cameras within a single platform results in increased system cost as well as implementation complexity.

Paper



3D Imaging

More Info

Image sensors capture 2D intensity information. Therefore, estimating the 3D geometry of the actual world from one or multiple 2D images is an essential problem in optics and computer vision. PhaseCam3D uses a novel phase mask
to help with the depth estimation.



PSF Engineering

More Info

Previously, amplitude mask designs have demonstrated applications in depth estimation and light-field imaging. Even though the PSFs of amplitude mask-based system is depth-dependent, the difference in PSFs across depth is only in scale. On the contrary, phase masks produce PSFs with much higher depth dependent variability. As a result, the phase mask should help distinguish the depth better in theory and the feature size can be made smaller.



End-to-end Design

More Info

Deep learning has now been used as a tool for end-to-end optimization of the imaging system. The key idea is to model the optical imaging formation models as parametric neural network layers, connect those layers with the application layers (i.e., image recognition, reconstruction, etc.) and finally use back-propagation to train on a large dataset to update the parameters in optics design.

PhaseCam3D

We propose PhaseCam3D, a passive, single-viewpoint 3D imaging system that jointly optimizes the front-end optics (phase mask) and the back-end reconstruction algorithm. Using end-to-end optimization, we obtain a novel phase mask that provides superior depth estimation performance compared to existing approaches. We fabricated the optimized phase mask and build a coded aperture camera by integrated the phase mask into the aperture plane of the lens. We demonstrate compelling 3D imaging performance using our prototype.