On the instability and degeneracy of deep learning models

dc.contributor.author Kaplan, Andee
dc.contributor.author Nordman, Daniel
dc.contributor.author Vardeman, Stephen
dc.contributor.author Vardeman, Stephen
dc.contributor.department Statistics
dc.date 2018-06-08T22:45:35.000
dc.date.accessioned 2020-07-02T06:55:58Z
dc.date.available 2020-07-02T06:55:58Z
dc.date.issued 2017-01-01
dc.description.abstract <p>A probability model exhibits instability if small changes in a data outcome result in large, and often unanticipated, changes in probability. This instability is a property of the probability model, rather than the fitted parameter vector. For correlated data structures found in several application areas, there is increasing interest in predicting/identifying such sensitivity in model probability structure. We consider the problem of quantifying instability for general probability models defined on sequences of observations, where each sequence of length N has a finite number of possible values. A sequence of probability models results, indexed by N, that accommodates data of expanding dimension. Model instability is formally shown to occur when a certain log-probability ratio under such models grows faster than N. In this case, a one component change in the data sequence can shift probability by orders of magnitude. Also, as instability becomes more extreme, the resulting probability models are shown to tend to degeneracy, placing all their probability on potentially small portions of the sample space. These results on instability apply to large classes of models commonly used in random graphs, network analysis, and machine learning contexts.</p>
dc.description.comments <p>This is a preprint of the article Kaplan, Andee, Daniel Nordman, and Stephen Vardeman. "A note on the instability and degeneracy of deep learning models." <em>arXiv preprint arXiv:1612.01159v2</em> (2017).</p>
dc.identifier archive/lib.dr.iastate.edu/stat_las_preprints/133/
dc.identifier.articleid 1133
dc.identifier.contextkey 12281670
dc.identifier.s3bucket isulib-bepress-aws-west
dc.identifier.submissionpath stat_las_preprints/133
dc.identifier.uri https://dr.lib.iastate.edu/handle/20.500.12876/90294
dc.language.iso en
dc.source.bitstream archive/lib.dr.iastate.edu/stat_las_preprints/133/2017_Vardeman_InstabilityDegeneracy.pdf|||Fri Jan 14 19:49:35 UTC 2022
dc.subject.disciplines Industrial Engineering
dc.subject.disciplines Industrial Technology
dc.subject.disciplines Statistics and Probability
dc.subject.keywords Degeneracy
dc.subject.keywords Instability
dc.subject.keywords Classification
dc.subject.keywords Deep Learning
dc.subject.keywords Graphical Models
dc.title On the instability and degeneracy of deep learning models
dc.type article
dc.type.genre article
dspace.entity.type Publication
relation.isAuthorOfPublication d398ec8c-7612-4286-b7b8-88f844f10410
relation.isOrgUnitOfPublication 264904d9-9e66-4169-8e11-034e537ddbca
File
Original bundle
Now showing 1 - 1 of 1
Name:
2017_Vardeman_InstabilityDegeneracy.pdf
Size:
566.08 KB
Format:
Adobe Portable Document Format
Description:
Collections