A Saddle-Point Dynamical System Approach for Robust Deep Learning

dc.contributor.author Esfandiari, Yasaman
dc.contributor.author Ebrahimi, Keivan
dc.contributor.author Balu, Aditya
dc.contributor.author Vaidya, Umesh
dc.contributor.author Elia, Nicola
dc.contributor.author Vaidya, Umesh
dc.contributor.author Sarkar, Soumik
dc.contributor.department Mechanical Engineering
dc.contributor.department Electrical and Computer Engineering
dc.date 2019-10-28T17:34:21.000
dc.date.accessioned 2020-06-30T02:03:01Z
dc.date.available 2020-06-30T02:03:01Z
dc.date.copyright Tue Jan 01 00:00:00 UTC 2019
dc.date.issued 2019-01-01
dc.description.abstract <p>We propose a novel discrete-time dynamical system-based framework for achieving adversarial robustness in machine learning models. Our algorithm is originated from robust optimization, which aims to find the saddle point of a min-max optimization problem in the presence of uncertainties. The robust learning problem is formulated as a robust optimization problem, and we introduce a discrete-time algorithm based on a saddle-point dynamical system (SDS) to solve this problem. Under the assumptions that the cost function is convex and uncertainties enter concavely in the robust learning problem, we analytically show that using a diminishing step-size, the stochastic version of our algorithm, SSDS converges asymptotically to the robust optimal solution. The algorithm is deployed for the training of adversarially robust deep neural networks. Although such training involves highly non-convex non-concave robust optimization problems, empirical results show that the algorithm can achieve significant robustness for deep learning. We compare the performance of our SSDS model to other state-of-the-art robust models, e.g., trained using the projected gradient descent (PGD)-training approach. From the empirical results, we find that SSDS training is computationally inexpensive (compared to PGD-training) while achieving comparable performances. SSDS training also helps robust models to maintain a relatively high level of performance for clean data as well as under black-box attacks.</p>
dc.description.comments <p>This is a pre-print of the article Esfandiari, Yasaman, Keivan Ebrahimi, Aditya Balu, Nicola Elia, Umesh Vaidya, and Soumik Sarkar. "A Saddle-Point Dynamical System Approach for Robust Deep Learning." arXiv preprint <em>arXiv</em>:1910.08623 (2019). Posted with permission.</p>
dc.format.mimetype application/pdf
dc.identifier archive/lib.dr.iastate.edu/ece_pubs/233/
dc.identifier.articleid 1234
dc.identifier.contextkey 15636134
dc.identifier.s3bucket isulib-bepress-aws-west
dc.identifier.submissionpath ece_pubs/233
dc.identifier.uri https://dr.lib.iastate.edu/handle/20.500.12876/21062
dc.language.iso en
dc.source.bitstream archive/lib.dr.iastate.edu/ece_pubs/233/2019_VaidyaUmesh_SaddlePoint.pdf|||Fri Jan 14 22:48:12 UTC 2022
dc.subject.disciplines Electrical and Computer Engineering
dc.subject.disciplines Mechanical Engineering
dc.title A Saddle-Point Dynamical System Approach for Robust Deep Learning
dc.type article
dc.type.genre article
dspace.entity.type Publication
relation.isAuthorOfPublication 8c6830ca-b80e-4e7f-b142-36531c311f1b
relation.isOrgUnitOfPublication 6d38ab0f-8cc2-4ad3-90b1-67a60c5a6f59
relation.isOrgUnitOfPublication a75a044c-d11e-44cd-af4f-dab1d83339ff
File
Original bundle
Now showing 1 - 1 of 1
Name:
2019_VaidyaUmesh_SaddlePoint.pdf
Size:
1.04 MB
Format:
Adobe Portable Document Format
Description:
Collections