DID: Distributed Incremental Block Coordinate Descent for Nonnegative Matrix Factorization
Association for the Advancement of Artificial Intelligence
Is Version Of
Electrical and Computer Engineering
Nonnegative matrix factorization (NMF) has attracted much attention in the last decade as a dimension reduction method in many applications. Due to the explosion in the size of data, naturally the samples are collected and stored distributively in local computational nodes. Thus, there is a growing need to develop algorithms in a distributed memory architecture. We propose a novel distributed algorithm, called distributed incremental block coordinate descent (DID), to solve the problem. By adapting the block coordinate descent framework, closed-form update rules are obtained in DID. Moreover, DID performs updates incrementally based on the most recently updated residual matrix. As a result, only one communication step per iteration is required. The correctness, efficiency, and scalability of the proposed algorithm are verified in a series of numerical experiments.
This is a manuscript of a proceeding published as Gao, Tianxiang, and Chris Chu. "DID: distributed incremental block coordinate descent for nonnegative matrix factorization." In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, pp. 2991-2998. 2018. DOI: 10.1609/aaai.v32i1.11736. Copyright 2018 Association for the Advancement of Artificial Intelligence. Posted with permission.