Collaborative Research: RI : Medium: Thermal Computational Imaging

Image sensors for thermal wavebands of light are usually noisy and subject to nuisance variations. Yet, it is indisputable that thermal imaging has the potential to spur numerous scientific and engineering applications that can fundamentally transform our society. For example, autonomous vehicles equipped with thermal cameras can navigate in the dark and through fog and rain.

The widespread availability of thermal imaging technology could usher in methods for non-intrusive vital sign monitoring in public spaces, thereby providing a new tool in our public health arsenal, as well as enable environmental and ecosystem monitoring at both local and global scales. This project seeks to enable such applications with inexpensive but noisy thermal sensors.

The research advances made in this progress will be integrated with an educational and outreach program that includes creating new undergraduate and graduate courses, engaging undergraduates in research, and engaging with K-12 communities.

The focus of this research is to advance thermal scene understanding by developing foundational tools for rendering, modeling and imaging at thermal wavebands. The project will develop a physically accurate rendering pipeline for thermal wavebands, that incorporates wavelength-dependent thermal emissivity of complex surfaces, thermal propagation in atmosphere, and accurate sensor modeling and noise characterization.

This modeling will be used to design and develop novel computational imaging systems that allow capture of multi-dimensional thermal signals, such as multi-spectral and polarization measurements. The research will also develop a differentiable renderer and exploit it within an end-to-end learning framework for inverse graphics. This will allow for significant improvements in problems such as denoising, super-resolution, stereo, object detection and others.

The advances in research will be used to explore and advance autonomous navigation, remote health monitoring and remote environmental monitoring.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.