Better bitmap performance with Roaring bitmaps
Summary Bitmap indexes are commonly used in databases and search engines. By exploiting bit‐level parallelism, they can significantly accelerate queries. However, they can use much memory, and thus, we might prefer compressed bitmap indexes. Following Oracle's lead, bitmaps are often compressed...
Gespeichert in:
| Veröffentlicht in: | Software, practice & experience practice & experience, 2016-05, Vol.46 (5), p.709-719 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 719 |
|---|---|
| container_issue | 5 |
| container_start_page | 709 |
| container_title | Software, practice & experience |
| container_volume | 46 |
| creator | Chambi, Samy Lemire, Daniel Kaser, Owen Godin, Robert |
| description | Summary
Bitmap indexes are commonly used in databases and search engines. By exploiting bit‐level parallelism, they can significantly accelerate queries. However, they can use much memory, and thus, we might prefer compressed bitmap indexes. Following Oracle's lead, bitmaps are often compressed using run‐length encoding (RLE). Building on prior work, we introduce the Roaring compressed bitmap format: it uses packed arrays for compression instead of RLE. We compare it to two high‐performance RLE‐based bitmap encoding techniques: Word Aligned Hybrid compression scheme and Compressed ‘n’ Composable Integer Set. On synthetic and real data, we find that Roaring bitmaps (1) often compress significantly better (e.g., 2×) and (2) are faster than the compressed alternatives (up to 900× faster for intersections). Our results challenge the view that RLE‐based bitmap compression is best. Copyright © 2015 John Wiley & Sons, Ltd. |
| doi_str_mv | 10.1002/spe.2325 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_1808120168</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1808120168</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3645-5fe2680dd30d27d8910f787028bd2e4bf682a3f800b7edb06b02c985fd595fec3</originalsourceid><addsrcrecordid>eNp10FtLwzAUB_AgCs4p-BEKvvjSeZI0lz7qmJsyLzhF30LaJtrZm0nH3Le3Y0NR8Ok8nN-58EfoGMMAA5Az35gBoYTtoB6GWIRAopdd1AOgMgQeRfvowPs5AMaM8B4aXJi2NS5I8rbUTdAYZ2tX6io1wTJv34KHWru8et32_SHas7rw5mhb--jpcvQ4nITTu_HV8HwappRHLGTWEC4hyyhkRGQyxmCFFEBkkhETJZZLoqmVAIkwWQI8AZLGktmMxd1sSvvodLO3cfXHwvhWlblPTVHoytQLr7AEiQlgLjt68ofO64Wruu8UFt0FTCPKfhamrvbeGasal5farRQGtQ5OdcGpdXAdDTd0mRdm9a9Ts_vRb5_71nx-e-3eFRdUMPV8O1a3E3ED_HqmYvoFT7N7_Q</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1780013435</pqid></control><display><type>article</type><title>Better bitmap performance with Roaring bitmaps</title><source>Wiley Online Library - AutoHoldings Journals</source><creator>Chambi, Samy ; Lemire, Daniel ; Kaser, Owen ; Godin, Robert</creator><creatorcontrib>Chambi, Samy ; Lemire, Daniel ; Kaser, Owen ; Godin, Robert</creatorcontrib><description>Summary
Bitmap indexes are commonly used in databases and search engines. By exploiting bit‐level parallelism, they can significantly accelerate queries. However, they can use much memory, and thus, we might prefer compressed bitmap indexes. Following Oracle's lead, bitmaps are often compressed using run‐length encoding (RLE). Building on prior work, we introduce the Roaring compressed bitmap format: it uses packed arrays for compression instead of RLE. We compare it to two high‐performance RLE‐based bitmap encoding techniques: Word Aligned Hybrid compression scheme and Compressed ‘n’ Composable Integer Set. On synthetic and real data, we find that Roaring bitmaps (1) often compress significantly better (e.g., 2×) and (2) are faster than the compressed alternatives (up to 900× faster for intersections). Our results challenge the view that RLE‐based bitmap compression is best. Copyright © 2015 John Wiley & Sons, Ltd.</description><identifier>ISSN: 0038-0644</identifier><identifier>EISSN: 1097-024X</identifier><identifier>DOI: 10.1002/spe.2325</identifier><language>eng</language><publisher>Bognor Regis: Blackwell Publishing Ltd</publisher><subject>Arrays ; bitmap index ; Compressed ; Compressing ; Computer programs ; Encoding ; index compression ; measurement ; Performance indices ; Queries ; Search engines</subject><ispartof>Software, practice & experience, 2016-05, Vol.46 (5), p.709-719</ispartof><rights>Copyright © 2015 John Wiley & Sons, Ltd.</rights><rights>Copyright © 2016 John Wiley & Sons, Ltd.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3645-5fe2680dd30d27d8910f787028bd2e4bf682a3f800b7edb06b02c985fd595fec3</citedby><cites>FETCH-LOGICAL-c3645-5fe2680dd30d27d8910f787028bd2e4bf682a3f800b7edb06b02c985fd595fec3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://onlinelibrary.wiley.com/doi/pdf/10.1002%2Fspe.2325$$EPDF$$P50$$Gwiley$$H</linktopdf><linktohtml>$$Uhttps://onlinelibrary.wiley.com/doi/full/10.1002%2Fspe.2325$$EHTML$$P50$$Gwiley$$H</linktohtml><link.rule.ids>314,776,780,1411,27901,27902,45550,45551</link.rule.ids></links><search><creatorcontrib>Chambi, Samy</creatorcontrib><creatorcontrib>Lemire, Daniel</creatorcontrib><creatorcontrib>Kaser, Owen</creatorcontrib><creatorcontrib>Godin, Robert</creatorcontrib><title>Better bitmap performance with Roaring bitmaps</title><title>Software, practice & experience</title><addtitle>Softw. Pract. Exper</addtitle><description>Summary
Bitmap indexes are commonly used in databases and search engines. By exploiting bit‐level parallelism, they can significantly accelerate queries. However, they can use much memory, and thus, we might prefer compressed bitmap indexes. Following Oracle's lead, bitmaps are often compressed using run‐length encoding (RLE). Building on prior work, we introduce the Roaring compressed bitmap format: it uses packed arrays for compression instead of RLE. We compare it to two high‐performance RLE‐based bitmap encoding techniques: Word Aligned Hybrid compression scheme and Compressed ‘n’ Composable Integer Set. On synthetic and real data, we find that Roaring bitmaps (1) often compress significantly better (e.g., 2×) and (2) are faster than the compressed alternatives (up to 900× faster for intersections). Our results challenge the view that RLE‐based bitmap compression is best. Copyright © 2015 John Wiley & Sons, Ltd.</description><subject>Arrays</subject><subject>bitmap index</subject><subject>Compressed</subject><subject>Compressing</subject><subject>Computer programs</subject><subject>Encoding</subject><subject>index compression</subject><subject>measurement</subject><subject>Performance indices</subject><subject>Queries</subject><subject>Search engines</subject><issn>0038-0644</issn><issn>1097-024X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2016</creationdate><recordtype>article</recordtype><recordid>eNp10FtLwzAUB_AgCs4p-BEKvvjSeZI0lz7qmJsyLzhF30LaJtrZm0nH3Le3Y0NR8Ok8nN-58EfoGMMAA5Az35gBoYTtoB6GWIRAopdd1AOgMgQeRfvowPs5AMaM8B4aXJi2NS5I8rbUTdAYZ2tX6io1wTJv34KHWru8et32_SHas7rw5mhb--jpcvQ4nITTu_HV8HwappRHLGTWEC4hyyhkRGQyxmCFFEBkkhETJZZLoqmVAIkwWQI8AZLGktmMxd1sSvvodLO3cfXHwvhWlblPTVHoytQLr7AEiQlgLjt68ofO64Wruu8UFt0FTCPKfhamrvbeGasal5farRQGtQ5OdcGpdXAdDTd0mRdm9a9Ts_vRb5_71nx-e-3eFRdUMPV8O1a3E3ED_HqmYvoFT7N7_Q</recordid><startdate>201605</startdate><enddate>201605</enddate><creator>Chambi, Samy</creator><creator>Lemire, Daniel</creator><creator>Kaser, Owen</creator><creator>Godin, Robert</creator><general>Blackwell Publishing Ltd</general><general>Wiley Subscription Services, Inc</general><scope>BSCLL</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>8FD</scope><scope>F28</scope><scope>FR3</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope></search><sort><creationdate>201605</creationdate><title>Better bitmap performance with Roaring bitmaps</title><author>Chambi, Samy ; Lemire, Daniel ; Kaser, Owen ; Godin, Robert</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3645-5fe2680dd30d27d8910f787028bd2e4bf682a3f800b7edb06b02c985fd595fec3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2016</creationdate><topic>Arrays</topic><topic>bitmap index</topic><topic>Compressed</topic><topic>Compressing</topic><topic>Computer programs</topic><topic>Encoding</topic><topic>index compression</topic><topic>measurement</topic><topic>Performance indices</topic><topic>Queries</topic><topic>Search engines</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Chambi, Samy</creatorcontrib><creatorcontrib>Lemire, Daniel</creatorcontrib><creatorcontrib>Kaser, Owen</creatorcontrib><creatorcontrib>Godin, Robert</creatorcontrib><collection>Istex</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Technology Research Database</collection><collection>ANTE: Abstracts in New Technology & Engineering</collection><collection>Engineering Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>Software, practice & experience</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Chambi, Samy</au><au>Lemire, Daniel</au><au>Kaser, Owen</au><au>Godin, Robert</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Better bitmap performance with Roaring bitmaps</atitle><jtitle>Software, practice & experience</jtitle><addtitle>Softw. Pract. Exper</addtitle><date>2016-05</date><risdate>2016</risdate><volume>46</volume><issue>5</issue><spage>709</spage><epage>719</epage><pages>709-719</pages><issn>0038-0644</issn><eissn>1097-024X</eissn><abstract>Summary
Bitmap indexes are commonly used in databases and search engines. By exploiting bit‐level parallelism, they can significantly accelerate queries. However, they can use much memory, and thus, we might prefer compressed bitmap indexes. Following Oracle's lead, bitmaps are often compressed using run‐length encoding (RLE). Building on prior work, we introduce the Roaring compressed bitmap format: it uses packed arrays for compression instead of RLE. We compare it to two high‐performance RLE‐based bitmap encoding techniques: Word Aligned Hybrid compression scheme and Compressed ‘n’ Composable Integer Set. On synthetic and real data, we find that Roaring bitmaps (1) often compress significantly better (e.g., 2×) and (2) are faster than the compressed alternatives (up to 900× faster for intersections). Our results challenge the view that RLE‐based bitmap compression is best. Copyright © 2015 John Wiley & Sons, Ltd.</abstract><cop>Bognor Regis</cop><pub>Blackwell Publishing Ltd</pub><doi>10.1002/spe.2325</doi><tpages>11</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0038-0644 |
| ispartof | Software, practice & experience, 2016-05, Vol.46 (5), p.709-719 |
| issn | 0038-0644 1097-024X |
| language | eng |
| recordid | cdi_proquest_miscellaneous_1808120168 |
| source | Wiley Online Library - AutoHoldings Journals |
| subjects | Arrays bitmap index Compressed Compressing Computer programs Encoding index compression measurement Performance indices Queries Search engines |
| title | Better bitmap performance with Roaring bitmaps |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-04T04%3A26%3A22IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Better%20bitmap%20performance%20with%20Roaring%20bitmaps&rft.jtitle=Software,%20practice%20&%20experience&rft.au=Chambi,%20Samy&rft.date=2016-05&rft.volume=46&rft.issue=5&rft.spage=709&rft.epage=719&rft.pages=709-719&rft.issn=0038-0644&rft.eissn=1097-024X&rft_id=info:doi/10.1002/spe.2325&rft_dat=%3Cproquest_cross%3E1808120168%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1780013435&rft_id=info:pmid/&rfr_iscdi=true |