HCC: Small: Neural Network-based Solvers for Integral Equations in Light Transport

Neural networks are commonly associated with applications in artificial intelligence such as computer vision, natural language processing, or robotics. They are attractive for these applications because they can model highly complex relationships, such as how colors of millions of individual pixels in an image are related to a high-level description of the image, including the types of objects that are shown etc.

This project, however, will leverage the capacity of neural networks to solve numerical simulation problems, with a focus on light transport as the application domain. Novel techniques will be developed to model the complex relationship between 3D virtual environments and corresponding photo-realistic images. This work will enable numerous innovative computer graphics applications, for example in augmented and virtual reality.

The new techniques will have broad impact, because they can be adapted and applied to a vast range of related scientific simulation problems. The benefits of the project outcomes include improving the achievable accuracy and enabling the solution of problems at a larger scale and with more complex geometries than is currently possible.

This project will develop novel numerical techniques for neural network-based function representations to solve integral equations for light transport problems. The approach will represent continuous solutions using neural networks, and leverage gradient-based numerical solution techniques that minimize appropriate norms of the residuals, after which advanced techniques that improve the performance and accuracy of the baseline approach will be developed and evaluated, including novel neural network architectures that are suitable to effectively represent multi-dimensional functions such as the spatio-angular radiance fields required to solve light and radiative transfer equations, and efficient Monte Carlo sampling and curriculum learning strategies for the residual norms in the numerical problems to allow robust estimation of gradients at lower computational cost. Finally, novel few-shot learning techniques to further accelerate convergence will be investigated.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.