Properties and Bayesian fitting of restricted Boltzmann machines

Date
2019-02-01
Authors
Kaplan, Andee
Nordman, Daniel
Vardeman, Stephen
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Altmetrics
Authors
Research Projects
Organizational Units
Statistics
Organizational Unit
Journal Issue
Series
Department
StatisticsIndustrial and Manufacturing Systems Engineering
Abstract

A restricted Boltzmann machine (RBM) is an undirected graphical model constructed for discrete or continuous random variables, with two layers, one hidden and one visible, and no conditional dependency within a layer. In recent years, RBMs have risen to prominence due to their connection to deep learning. By treating a hidden layer of one RBM as the visible layer in a second RBM, a deep architecture can be created. RBMs thereby are thought to have the ability to encode very complex and rich structures in data, making them attractive for supervised learning. However, the generative behavior of RBMs largely is unexplored and typical fitting methodology does not easily allow for uncertainty quantification in addition to point estimates. In this paper, we discuss the relationship between RBM parameter specification in the binary case and model properties such as degeneracy, instability and uninterpretability. We also describe the associated difficulties that can arise with likelihood‐based inference and further discuss the potential Bayes fitting of such (highly flexible) models, especially as Gibbs sampling (quasi‐Bayes) methods often are advocated for the RBM model structure.

Comments

This is the peer-reviewed version of the following article: Kaplan, Andee, Daniel Nordman, and Stephen Vardeman. "Properties and Bayesian fitting of restricted Boltzmann machines." Statistical Analysis and Data Mining: The ASA Data Science Journal 12, no. 1 (2019): 23-38, which has been published in final form at DOI: 10.1002/sam.11396. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.

Description
Keywords
Citation
DOI
Collections