Novel Light Field Sensors
The Need for Better Imaging Sensors
Standard digital image sensors can only tell information about the intensity and wavelength of light, but often useful information is encoded in other dimensions of the plenoptic function, such as angle or phase. Accessing this information is often either impossible or costly – either computationally, or in the physical setup.
Angle-Sensitive Pixels (ASP) and SPADs for Light Field Imaging
We design CMOS sensors that can sense angle, polarization, phase interference, and time of arrival directly – alone or in combination. This allows for cheaper, lower power, more compact imaging setups which could ultimately facilitate a wearable or hand-held depth imaging device. Which aspects of the light field are captured and what data is processed are design choices that are informed by the imaging modality and reconstruction algorithms, and at the same time, our hardware design can inform software design.
Light Field Sensing
More Info
CMOS angle sensitive pixel (ASP) arrays have been reported to enable light field imaging, including computational refocusing and range finding, lens-less far-field imaging, and on-chip 3D localization.
Compact
More Info
we saved area by replacing the quadrature-phase (0°, 180°, 90°, and 270°) sampling with a balanced triphase (180°, 60°, and -60°) sampling. While the reconstruction of the incident angle and intensity is formulated as a two-dimensional complex number and therefore maps naturally to quadrature sampling, balanced tri-phase provides equivalent intensity detection through averaging the triphase sampling outputs. At the same time, three phases are sufficient to resolve angle information.
Low-cost
More Info
Pixel-scale SPADs on CMOS are emerging as favorable low-cost options for time of flight (TOF) or fluorescence lifetime imaging (FLIM) owing to their scalability, integration in conventional monolithic semiconductor technology, and therefore co-integrability with signal processing.
Publications of Our ASP sensors
Huaijin Chen, Suren Jayasuriya, Jiyue Yang, Judy Stephen, Sriram Sivaramakrishnan, Ashok Veeraraghavan, Alyosha Molnar, “ASP Vision: Optically Computing the First Layer of Convolutional Neural Networks using Angle Sensitive Pixels“, CVPR 2016
Changhyuk Lee, Ben Johnson, Alyosha Molnar. “Angle Sensitive Single Photon Avalanche Diode” Applied Physics Letters 2015
M. Hirsch, Sriram Sivaramakrishnan, Suren Jayasuriya, Albert Wang, Alyosha Molnar, R. Raskar, G. Wetzstein. “A Switchable Light Field Camera Architecture using Angle Sensitive Pixels and Dictionary-based Sparse Coding“, IEEE International Conference on Computational Photography 2014
Albert Wang, Sheila Hemami, Alyosha Molnar. “Angle-sensitive pixels: a new paradigm for low-power, low-cost 2D and 3D sensing” Proc. SPIE 8288, Stereoscopic Displays and Applications XXIII, 2012
People
Melissa White
Alyosha Molnar