New approaches to training deep probabilistic models

Probabilistic models are a mathematical representation of a given data set in terms of a probability distribution. The probability distribution reveals information about the frequency and correlation of features in the data set. The nuanced probabilistic representation can subsequently be used in numerous downstream tasks such as the

generation of realistic synthetic data points, the classification of data, or the improvement of medical imaging methods. Deep neural networks, on the other hand, have significantly advanced the field of machine learning. They allow computers to read hand-written notes, make predictions in complex scenarios, or translate between languages. However, such networks are typically

not well-suited for learning probabilistic models because the output of neural networks can not be interpreted as a probability. This has led to the development of unnormalised models which allow the modelling via a deep neural network. However, training unnormalised models is difficult, and most methods are not well-grounded in theoretical

works. Theoretical guarantees, however, are critical when applying the obtained probabilistic representation in sensitive areas such as medicine. My research focuses on new methodologies for the training and evaluation of deep unnormalised probabilistic models. The aim of my research is two-fold. Firstly, I work

towards training methodologies that are derived from classical theoretical results in statistics and probability theory. Such methods perform more reliably, and failure cases that appear in conventionally used algorithms can be ruled out. Furthermore, approximation errors remain tractable so that one can prove theoretical guarantees

missing in related works. Secondly, I aim to achieve competitive performance with previous training methods for unnormalised models. In particular, I seek to make our methodology scalable to high-dimensional data without compromising the quality of the learned distribution. In our current numerical experiments, we observe that our method

captures non-linear dependencies in the data and generalises well when used to generate synthetic data samples. Since probabilistic models represent our knowledge about a system obtained from data, my work is universally applicable in big-data settings. They are most known for their capacity to generate realistic artificial images or to fill incomplete images. Probabilistic

models are also impactful in engineering applications where the quantity of interest has to be inferred from values of a known forward process (so-called Bayesian inverse problems). A prominent example of this is the reconstruction of a medical image from measurements of scattered radiation. Additionally, probabilistic models can be used to

conceal sensitive or private information in data sets because they contain relevant statistical information rather than individual data points. Hence, the probabilistic model can be used to optimise processes while not revealing sensitive information such as patient data in hospitals. My current project is joint work with Dr Andrew B. Duncan (Improbable Worlds Limited

and Imperial College, London), Jen Ning Lim (University of Warwick), and Prof. Dr Sebastian J. Vollmer (University of Kaiserslautern, DFKI