Properties and Bayesian fitting of restricted Boltzmann machines

Supplemental Files
Date
2018-01-01
Authors
Kaplan, Andee
Nordman, Daniel
Vardeman, Stephen
Vardeman, Stephen
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Altmetrics
Authors
Research Projects
Organizational Units
Statistics
Organizational Unit
Journal Issue
Series
Department
Statistics
Abstract

A restricted Boltzmann machine (RBM) is an undirected graphical model constructed for discrete or continuous random variables, with two layers, one hidden and one visible, and no conditional dependency within a layer. In recent years, RBMs have risen to prominence due to their connection to deep learning. By treating a hidden layer of one RBM as the visible layer in a second RBM, a deep architecture can be created. RBMs are thought to thereby have the ability to encode very complex and rich structures in data, making them attractive for supervised learning. However, the generative behavior of RBMs is largely unexplored and typical fitting methodology does not easily allow for uncertainty quantification in addition to point estimates. In this paper, we discuss the relationship between RBM parameter specification in the binary case and model properties such as degeneracy, instability and uninterpretability. We also describe the associated difficulties that can arise with likelihood-based inference and further discuss the potential Bayes fitting of such (highly flexible) models, especially as Gibbs sampling (quasi-Bayes) methods are often advocated for the RBM model structure.

Comments

This is a preprint of the article Kaplan, Andee, Daniel Nordman, and Stephen Vardeman. "Properties and Bayesian fitting of restricted Boltzmann machines." arXiv preprint arXiv:1612.01158v3 (2018).

Description
Keywords
Citation
DOI
Source
Collections