Boltzmann machine, Contrastive Divergence and Information Geometry

The restricted Boltzmann machine (RBM) can be regarded as a generative neural network with extensive successful applications in unsupervised learning.

Al- though the contrastive divergence (CD) method, which is based on Monte-Carlo Markov chain sampling method, is widely used in training an RBM, it has flawed interpretation. This project aims to clarify certain issues concerning CD, including the motivation, training and justification.

Moreover, we provide a deeper insight into the training of RBMs based on maximum-likelihood training and CD training from the perspective of information geometry, which is an interdisciplinary field that applies the techniques of differential geometry to study statistical models.