CAREER: Optimized Sensing and Recovery for Computational Imaging

Cameras have evolved tremendously over the past few decades as they have become compact and widely available in mobile devices. Nevertheless, vast opportunities for improvements still exist, especially as the community moves beyond consumer photography and start building cameras for sensing and understanding new environments under strict constraints.

These constraints can arise because of physical requirements on the size, shape, or weight of the components, time required to capture and process the data, or cost and energy thresholds for the entire system. The focus of this research is to develop novel methods to sense and process visual information while taking into account the constraints and requirements on data, sensors, algorithms, and tasks in the real world.

A successful outcome of the research will benefit diverse applications spanning consumer photography, machine vision and automation, and scientific/medical imaging. Training of a diverse group of undergraduate and graduate students through educational courses and research experience is an integral part of this project. The education and outreach component of this project involves dissemination of computational imaging research to K-12 students and teachers through annual research days on campus, as well as to the general public by collaborating with a museum of photography.

Computational imaging offers a general framework to co-design sensing hardware and computational software to build novel and unconventional cameras. This research will address a number of fundamental theoretical and algorithmic questions related to optimized sensing, representation, and recovery for computational imaging. The research is organized into three inter-related thrusts: (1) Optimize and expand the space of measurements that computational imaging systems can realistically capture using programmable optics. (2) Learn sensing, data representation, and recovery algorithms in an end-to-end manner. (3) Develop efficient algorithms for computational imaging systems beyond linear and shift-invariant models.

The insights gained from this research will help us understand some of the fundamental limits that exist in realistic computational imaging systems and provide tools to optimize them within the given constraints. Models and algorithms developed in this research will be validated with real imaging experiments.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.