Communication-Efficient Network-Distributed Optimization with Differential-Coded Compressors

dc.contributor.author Zhang, Xin
dc.contributor.author Zhu, Zhengyuan
dc.contributor.author Liu, Jia
dc.contributor.author Zhu, Zhengyuan
dc.contributor.author Bentley, Elizabeth
dc.contributor.department Computer Science
dc.contributor.department Statistics
dc.date 2020-06-18T03:31:29.000
dc.date.accessioned 2020-07-02T06:55:44Z
dc.date.available 2020-07-02T06:55:44Z
dc.date.embargo 2020-06-17
dc.date.issued 2019-01-01
dc.description.abstract <p>Network-distributed optimization has attracted significant attention in recent years due to its ever-increasing applications. However, the classic decentralized gradient descent (DGD) algorithm is communication-inefficient for large-scale and high-dimensional network-distributed optimization problems. To address this challenge, many compressed DGD-based algorithms have been proposed. However, most of the existing works have high complexity and assume compressors with bounded noise power. To overcome these limitations, in this paper, we propose a new differential-coded compressed DGD (DC-DGD) algorithm. The key features of DC-DGD include: i) DC-DGD works with general SNR-constrained compressors, relaxing the bounded noise power assumption; ii) The differential-coded design entails the same convergence rate as the original DGD algorithm; and iii) DC-DGD has the same low-complexity structure as the original DGD due to a {\em self-noise-reduction effect}. Moreover, the above features inspire us to develop a hybrid compression scheme that offers a systematic mechanism to minimize the communication cost. Finally, we conduct extensive experiments to verify the efficacy of the proposed DC-DGD and hybrid compressor.</p>
dc.description.comments <p>This is a pre-print of the proceeding Zhang, Xin, Jia Liu, Zhengyuan Zhu, and Elizabeth S. Bentley. "Communication-Efficient Network-Distributed Optimization with Differential-Coded Compressors." <em>arXiv preprint arXiv:1912.03208</em> (2019). </p>
dc.format.mimetype application/pdf
dc.identifier archive/lib.dr.iastate.edu/stat_las_conf/14/
dc.identifier.articleid 1013
dc.identifier.contextkey 18148074
dc.identifier.s3bucket isulib-bepress-aws-west
dc.identifier.submissionpath stat_las_conf/14
dc.identifier.uri https://dr.lib.iastate.edu/handle/20.500.12876/90247
dc.language.iso en
dc.source.bitstream archive/lib.dr.iastate.edu/stat_las_conf/14/2020_ZhuZhengyuan_CommunicationEfficient.pdf|||Fri Jan 14 20:11:21 UTC 2022
dc.subject.disciplines Applied Statistics
dc.subject.disciplines Theory and Algorithms
dc.title Communication-Efficient Network-Distributed Optimization with Differential-Coded Compressors
dc.type article
dc.type.genre conference
dspace.entity.type Publication
relation.isAuthorOfPublication 51db2a08-8f9d-4f97-bdbc-6790b3d5a608
relation.isOrgUnitOfPublication f7be4eb9-d1d0-4081-859b-b15cee251456
relation.isOrgUnitOfPublication 264904d9-9e66-4169-8e11-034e537ddbca
File
Original bundle
Now showing 1 - 1 of 1
Name:
2020_ZhuZhengyuan_CommunicationEfficient.pdf
Size:
463.91 KB
Format:
Adobe Portable Document Format
Description: