Stein’s method and functional inequalities in machine learning

The project aims to develop quality measures for approximations in machine learning and statistics, using tools of probability and functional analysis, such as Stein's method and functional inequalities.

Approximate inference techniques have been used in the recent years as a way to speed up the learning process, which is particularly important in the era of big data. It is, however, necessary for researchers to be able to measure the error of the associated approximations.

Indeed, wrong variance or mean estimates in applications related, for instance, to modelling infectious diseases, may have highly negative outcomes. In this project, I will concentrate on three specific aspects of this problem. I will firstly propose tools for measuring the quality of posterior approximations in Gaussian Process inference.

In order to do this, I will use the theory of Stein discrepancies which has already been successfully applied, in the context of Bayesian inference, to finite-dimensional distributions.

I will combine it with the recent developments in probability theory related to Stein's method for infinite-dimensional measures.

Secondly, I will construct a tool for a simultaneous study of the rate of convergence and the output quality of MCMC schemes based on discretising diffusion processes. Both those objects may be analysed using the infinitesimal generator of the underlying diffusion.

Indeed, for the former we may apply the associated log-Sobolev or Poincare inequalities and, for the latter, utilise the associated Stein operator. The resulting tool will help users choose (or construct) an algorithm which is simultaneously fast and robust.

Thirdly, I will construct a Gaussian-Process goodness-of-fit test, allowing users to test whether the given data come from a marginal of a particular GP.

In order to do this, I will use infinite-dimensional Stein’s method together with techniques used recently to construct kernel goodness-of-fit tests based on Stein discrepancies.