Collaborative Research: Bayesian Residual Learning and Random Recursive Partitioning Methods for Gaussian Process Modeling

Rare natural hazards (for example, storm surge and hurricanes) can cause loss of lives and devastating damage to society and the environment. For instance, Hurricane Katrina (2005) caused over 1,500 deaths and total estimated damages of $75 billion in the New Orleans area and along the Mississippi coast as a result of storm surge. Uncertainty quantification (UQ) has been used widely to understand, monitor, and predict these rare natural hazards.

The Gaussian process (GP) modeling framework is one of the most widely used tools to address such UQ applications and has been studied across several areas, including spatial statistics, design and analysis of computer experiments, and machine learning. With the advance of measurement technology and increasing computing power, large numbers of measurements and large-scale numerical simulations at increasing resolutions are routinely collected in modern applications and have given rise to several critical challenges in predicting real-world processes with associated uncertainty.

While GP presents a promising route to carrying out UQ tasks for modern emerging applications such as coastal flood hazard studies, existing GP methods are inadequate in addressing several notable issues such as computational bottleneck due to big datasets and spatial heterogeneity due to complex structures in multi-dimensional domains. This project will develop new Bayesian GP methods to allow scalable computation and to capture spatial heterogeneity.

The new methods, algorithms, theory, and software are expected to improve GP modeling for addressing data analytical issues across a wide range of fields, including physical science, engineering, medical science, public health, and business science. The project will develop and distribute user-friendly open-source software and provide interdisciplinary research training opportunities for undergraduate and graduate students.

This project aims to develop a new Bayesian multi-scale residual learning framework with strong theoretical support that allows scalable computation and spatial nonstationarity for GP modeling. This framework integrates and extends several powerful techniques respectively arising in the literature on GP and that on multi-scale modeling, including predictive process approximation, blockwise shrinkage, and random recursive partitioning on the domain.

This framework decomposes the GP into a cascade of residual processes that characterize the underlying covariance structures at different resolutions and that can be spatially heterogeneous in a variety of ways. The new framework allows for adoption of blockwise shrinkage to infer the covariance of the residual processes and incorporates random partition priors to enable adaptivity to various spatial structures in multi-dimensional domains.

New recursive algorithms inspired by wavelet shrinkage and state-space models will be developed to achieve linear computational complexity and linear storage complexity in terms of the number of observations. The resulting GP method will guarantee linear computational complexity in a serial computing environment and also be easily parallelizable. This Bayesian multi-scale residual learning method provides a new approach to addressing GP modeling issues among spatial statistics, design and analysis of computer experiments, machine learning, and nonparametric regression.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.