An Investigation into Coupling a Stochastic Approximation with a Pseudo-Marginal Sampler

EPSRC : Max Hird : EP/T517793/1

Algorithms that learn and sample from probability distributions form an important part of machine learning, AI, and the natural sciences. One needn't look far to find such algorithms at the bleeding edge of methodology, and in everyday scientific pursuit.

The Wang-Landau algorithm is an example. It combines a sampling step with a learning step, to learn a probability distribution about which our knowledge is limited. The probability distribution may be over physical states, so an efficiently running algorithm would allow the simulation of the dynamics of protein folding, for instance.

The learning step incorporates information gained from the sampling step, forming a more complete picture of the distribution. The particular form of the learning step is foundational in many neural networks and is called stochastic approximation. Due to our incomplete knowledge of the distribution, we cannot apply standard sampling methods. We therefore need to employ a more exotic sampler.

Coupling exotic samplers alongside stochastic approximation is underexplored, and potentially fruitful. We will try to assess the behaviour of such a coupling, an assessment not yet existing in the literature.