Estimating quantiles from the union of historical and streaming data

Thumbnail Image
Date
2016-11-01
Authors
Singh, Sneha
Major Professor
Advisor
Committee Member
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract

Modern enterprises generate huge amounts of streaming data, for example, micro-blog feeds, financial data, network monitoring and industrial application monitoring. While Data Stream Management Systems have proven successful in providing support for real-time alerting, many applications, such as network monitoring for intrusion detection and real-time bidding, require complex analytics over historical and real-time data over the data streams. We present a new method to process one of the most fundamental analytical primitives, quantile queries, on the union of historical and streaming data. Our method combines an index on historical data with a memory-efficient sketch on streaming data to answer quantile queries with accuracy-resource tradeoffs that are significantly better than current solutions that are based solely on disk-resident indexes or solely on streaming algorithms.

Series Number
Journal Issue
Is Version Of
Versions
Series
Academic or Administrative Unit
Type
article
Comments

This is a manuscript of a proceeding published as Singh, Sneha Aman, Divesh Srivastava, and Srikanta Tirthapura. "Estimating quantiles from the union of historical and streaming data." Proceedings of the VLDB Endowment 10, no. 4 (2016): 433-444. 10.14778/3025111.3025124. Posted with permission.

Rights Statement
Copyright
Fri Jan 01 00:00:00 UTC 2016
Funding
DOI
Supplemental Resources