A Novel Inference of a Restricted Boltzmann Machine

A deep neural network (DNN) pre-trained via stacking restricted Boltzmann machines (RBMs) demonstrates high performance. The binary RBM is usually used to construct the DNN. However, a continuous probability of each node is used as real value state, although the state of the binary RBM's node should be represented by a random binary variable. One of main reasons of this abuse is that it works. One of others is to reduce a computational cost. In this paper, we propose a novel inference of the RBM, considering that the input of the RBM is the random binary variable. Straight forward derivation of the proposed inference is intractable. Then, we also propose the closed-form approximation of it. We convince that the proposed inference is more reasonable than a conventional algorithm of the RBM. Experimental comparisons demonstrate that the proposed inference improves the performance of the DNN.