Parallel Framework for Dimensionality Reduction of Large-Scale Datasets

Thumbnail Image
Date
2015-01-01
Authors
Samudrala, Sai
Zola, Jaroslaw
Aluru, Srinivas
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract

Dimensionality reduction refers to a set of mathematical techniques used to reduce complexity of the original high-dimensional data, while preserving its selected properties. Improvements in simulation strategies and experimental data collection methods are resulting in a deluge of heterogeneous and high-dimensional data, which often makes dimensionality reduction the only viable way to gain qualitative and quantitative understanding of the data. However, existing dimensionality reduction software often does not scale to datasets arising in real-life applications, which may consist of thousands of points with millions of dimensions. In this paper, we propose a parallel framework for dimensionality reduction of large-scale data. We identify key components underlying the spectral dimensionality reduction techniques, and propose their efficient parallel implementation. We show that the resulting framework can be used to process datasets consisting of millions of points when executed on a 16,000-core cluster, which is beyond the reach of currently available methods. To further demonstrate applicability of our framework we perform dimensionality reduction of 75,000 images representing morphology evolution during manufacturing of organic solar cells in order to identify how processing parameters affect morphology evolution.

Series Number
Journal Issue
Is Version Of
Versions
Series
Type
article
Comments

This is an article published as Samudrala, Sai Kiranmayee, Jaroslaw Zola, Srinivas Aluru, and Baskar Ganapathysubramanian. "Parallel framework for dimensionality reduction of large-scale datasets." Scientific Programming 2015 (2015). Posted with permission.

Rights Statement
Copyright
Thu Jan 01 00:00:00 UTC 2015
Funding
DOI
Supplemental Resources
Source
Collections