Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation
Date
2020-10-26
Authors
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Multivariate time series modeling and prediction problems are abundant in many machine learning application domains. Accurate interpretation of such prediction outcomes from a machine learning model that explicitly captures temporal correlations can significantly benefit the domain experts. In this context, temporal attention has been successfully applied to isolate the important time steps for the input time series. However, in multivariate time series problems, spatial interpretation is also critical to understand the contributions of different variables on the model outputs. We propose a novel deep learning architecture, called spatiotemporal attention mechanism (STAM) for simultaneous learning of the most important time steps and variables. STAM is a causal (i.e., only depends on past inputs and does not use future inputs) and scalable (i.e., scales well with an increase in the number of variables) approach that is comparable to the state-of-the-art models in terms of computational tractability. We demonstrate our models' performance on two popular public datasets and a domain-specific dataset. When compared with the baseline models, the results show that STAM maintains state-of-the-art prediction accuracy while offering the benefit of accurate spatiotemporal interpretability. The learned attention weights are validated from a domain knowledge perspective for these real-world datasets.
Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
Preprint
Comments
This is a preprint from Gangopadhyay, Tryambak, Sin Yong Tan, Zhanhong Jiang, Rui Meng, and Soumik Sarkar. "Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation." arXiv preprint arXiv:2008.04882 (2020). doi: https://doi.org/10.48550/arXiv.2008.04882. Published as Gangopadhyay, Tryambak, Sin Yong Tan, Zhanhong Jiang, Rui Meng, and Soumik Sarkar. "Spatiotemporal attention for multivariate time series prediction and interpretation." In ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 3560-3564. IEEE, 2021. doi: https://doi.org/10.1109/ICASSP39728.2021.9413914.