CAREER: Probabilistic Framework for Self-Supervised, Data-Driven Computational Imaging

Computational imaging systems aim to improve over conventional imaging devices, allowing for advanced capabilities such as super-resolution or three-dimensional imaging. In computational imaging systems, such as computed tomography (CT), magnetic resonance imaging (MRI), and phase microscopy, indirect measurements of an object are taken with specialized hardware, and software is used to compute a final reconstruction.

For example, the measurements in CT are projection images, and software post-processing results in a 3-dimensional reconstruction of the object. The object is the input of the computational imaging system, such as biological cells on a microscope slide or a brain in a CT scan. The drawback of computational imaging systems is that they generally require a large number of measurements per object, which may be costly and slow to collect, making them prohibitive for many applications.

This program develops data-driven, probabilistic methods for computational imaging to reduce imaging time. The developed algorithms will not require a ground truth or reference training dataset, unlike many other machine learning methods. Success of this program will allow scientific advances in biology and medicine by imaging in previously inaccessible spatiotemporal regimes.

The broader impact is wide-ranging, including: lowered dose in CT and electron microscopy, reduced acquisition time in MRI, reduced photobleaching in fluorescence localization microscopy for live cell imaging, and real-time microscopy in surgical procedures. Developed methods can be applied to make medical imaging faster, improving patient comfort, reducing radiation exposure, and lowering costs.

Complementary educational programs will be developed in tandem with the research plan to broaden participation from underrepresented groups in engineering. Initiatives include summer experiences for high school students from nearby underserved communities and early college research involvement for undergraduates.

The proposed work improves the temporal resolution of light-emitting diode (LED) array microscopy, a modality that allows reconstruction of quantitative amplitude and phase (i.e., permittivity) in two and three dimensions with high resolution and high field-of-view. Algorithm development for LED array microscopy will build the foundation for research on other computational imaging methods such as micro-computed tomography and x-ray nano-holographic tomography.

Measurements in computational imaging are taken with varying hardware parameters; in the case of LED array microscopy, the illumination pattern is varied to collect a stack of images. The key innovation in this work is to jointly reconstruct similar objects, each with a low number of measurements taken with hardware parameters that vary from object to object.

By pooling information from measurements across the set of objects and incorporating the forward physics model, prior and posterior distributions can be jointly inferred. To efficiently solve this problem, this program creates a novel technique through a reformulation of variational autoencoders. Crucially, no ground truth dataset is assumed: only noisy, sparse measurements on each object.

The probabilistic formulation considered in this work permits uncertainty quantification, in contrast to reconstruction algorithms that only yield a point estimate. Simulated and experimental data will be used to explore the impact of measurement parameter diversity on accuracy, stability, and robustness. A novel adaptive measurement technique will be evaluated, in which the hardware parameters for object measurement are chosen based on the object data previously collected.

High memory management techniques and considerations for video reconstruction will be created and assessed, with the significance of allowing developed frameworks to be applied to real experimental problems of high scientific interest.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.