To be Bernoulli or to be Gaussian, for a Restricted Boltzmann Machine
- Author
- Takayoshi Yamashita, Masayuki Tanaka†, Eiji Yoshida, Yuji Yamauchi, Hironobu Fujiyoshi
- Publication
- International Conference on Pattern Recognition 2014
Download: PDF (English)
Abstract—We introduce a method that automatically selects appropriate RBM types according to the visible unit distribution. The distribution of a visible unit strongly depends on a dataset. For example, binary data can be considered as pseudo binary distribution with high peaks at 0 and 1. For real-value data, the distribution can be modeled by single Gaussian model or Gaussian mixture model. Our proposed method selects appropri- ate RBM according to the distribution of each unit. We employ the Gaussian mixture model to determine whether the visible unit distribution is the pseudo binary or the Gaussian mixture. According to this distribution, we can select a Bernoulli-Bernoulli RBM(BBRBM) or a Gaussian-Bernoulli RBM(GBRBM). Fur- thermore, we employ normalization process to obtain a smoothed Gaussian mixture distribution. This allowed us to reduce vari- ations such as illumination changes in the input data. After experimentation with MNIST, CBCL and our own dataset, our proposed method obtained the best recognition performance and further shortened the convergence time of the learning process.