CAREER: Seeing Through Atmospheric Turbulence: Image Restoration and Understanding using Deep Convolutional Neural Networks

Atmospheric turbulence can significantly degrade the quality of images acquired by long-range imaging systems by causing spatially and temporally random fluctuations in the index of refraction of the atmosphere. Variations in the refractive index causes the captured images to be geometrically distorted and blurry. These distortions adversely affect the performance of subsequent computer vision algorithms such as object detection and recognition.

Hence, it is important to compensate for the visual degradation in images caused by atmospheric turbulence. Adaptive optics-based techniques can be used to compensate for turbulence effects in images. However, they require large, complex and expensive hardware.

On the other hand, image processing-based approaches are cheap and effective. The idea of the proposed approach is to pose the turbulence degraded image restoration problem as a nonlinear regression problem, where the optimal parameters are learned from synthetically generated data. As a function approximator, we propose to use deep convolutional neural networks.

The goal of this CAREER project is to develop data-driven learning-based approaches for restoration and understanding of images degraded by atmospheric turbulence. This project will help create new undergraduate/graduate courses on Deep Learning and Image Restoration. This project will also include a broad range of outreach activities.

Our research will provide a comprehensive framework for restoring and understanding images/videos degraded by atmospheric turbulence. We will significantly innovate in the areas of supervised, semi-supervised and unsupervised image restoration techniques by developing novel end-to-end trainable deep convolutional neural networks and corresponding loss functions.

Furthermore, domain transfer learning-based methods for adapting computer vision algorithms such as object detection and segmentation to turbulence-degraded images will be developed. Algorithms that will be developed in this project will significantly enhance the quality of images and videos collected by long-range visible and infrared imagining systems, and will result in improved understanding of images degraded by atmospheric turbulence.

The proposed methods will be useful in many applications of remote sensing, long-range surveillance, optical communications, astronomy, road traffic monitoring, underwater imaging, and autonomous navigation.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.