Optimizing Druid with Roaring bitmaps [r-libre/950]

Chambi, Samy; Lemire, Daniel; Godin, Robert; Boukhalfa, Kamel; Allen, Charles, & Yang, Fangjin (2016). Optimizing Druid with Roaring bitmaps. In Proceedings of the 20th International Database Engineering & Applications Symposium. ACM. ISBN 978-1-4503-4118-9 https://doi.org/10.1145/2938503.2938515

File(s) available for this item:

PDF - Schambi_ideas16.pdf
Content : Submitted Version
License : Creative Commons Attribution.

Download

Item Type:	Papers in Conference Proceedings
Refereed:	Yes
Status:	Published
Abstract:	In the current Big Data era, systems for collecting, storing and efficiently exploiting huge amounts of data are continually introduced, such as Hadoop, Apache Spark, Dremel, etc. Druid is one of theses systems especially designed to manage such data quantities, and allows to perform detailed real-time analysis on terabytes of data within sub-second latencies. One of the important Druid's requirements is fast data filtering. To insure that, Druid makes an extensive use of bitmap indexes. Previously, we introduced a new compressed bitmap index scheme called Roaring bitmap that has shown interesting results when compared to the bitmap compression scheme adopted by Druid: Concise. Since, Roaring bitmap has been integrated to Druid as an indexing solution. In this work, we produce an extensive series of experiments in order to compare Roaring bitmap and Concise time-space performances when used to accelerate Druid's OLAP queries and other kinds of operations Druid realizes on bitmaps, like: retrieving set bits from bitmaps, computing bitmap complements, aggregating several bitmaps with logical ORs and ANDs operations. Roaring bitmap has shown to improve up to 5 times analytical queries response times under Druid compared to Concise.
Official URL:	http://dl.acm.org/citation.cfm?id=2938515&CFID=637...
Depositor:	Lemire, Daniel
Owner / Manager:	Daniel Lemire
Deposited:	27 May 2016 13:15
Last Modified:	12 Sep 2016 18:28

Actions (login required)

RÉVISER