Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile
Jet is an open-source, high-performance, distributed stream processor built at Hazelcast during the last five years. Jet was engineered with millisecond latency on the 99.99th percentile as its primary design goal. Originally Jet's purpose was to be an execution engine that performs complex bus...
Gespeichert in:
| Veröffentlicht in: | arXiv.org 2021-03 |
|---|---|
| Hauptverfasser: | , , , , , , , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | arXiv.org |
| container_volume | |
| creator | Gencer, Can Topolnik, Marko Ďurina, Viliam Demirci, Emin Kahveci, Ensar B Ali Gürbüz Ondřej Lukáš Bartók, József Gierlach, Grzegorz Hartman, František Yılmaz, Ufuk Doğan, Mehmet Mandouh, Mohamed Fragkoulis, Marios Katsifodimos, Asterios |
| description | Jet is an open-source, high-performance, distributed stream processor built at Hazelcast during the last five years. Jet was engineered with millisecond latency on the 99.99th percentile as its primary design goal. Originally Jet's purpose was to be an execution engine that performs complex business logic on top of streams generated by Hazelcast's In-memory Data Grid (IMDG): a set of high-performance, in-memory, partitioned and replicated data structures. With time, Jet evolved into a full-fledged, scale-out stream processor that can handle out-of-order streams and exactly-once processing guarantees. Jet's end-to-end latency lies in the order of milliseconds, and its throughput in the order of millions of events per CPU-core. This paper presents main design decisions we made in order to maximize the performance per CPU-core, alongside lessons learned, and an empirical performance evaluation. |
| format | Article |
| fullrecord | <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2503045280</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2503045280</sourcerecordid><originalsourceid>FETCH-proquest_journals_25030452803</originalsourceid><addsrcrecordid>eNqNyrEKwjAUQNEgCBbtPzxwrsSk1cZVLCIOBd1LCE_bEhNNXhH9eh38AKc7nDtiiZBymZW5EBOWxthzzsVqLYpCJqza6zdaoyPBAWkDR__MrCZ05gUnCqhvUAdvMMbOXUETUIug1EIpaqHGYNBRZ3HGxhdtI6a_Ttm82p23--we_GPASE3vh-C-1IiCS54XouTyv-sDImk56A</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2503045280</pqid></control><display><type>article</type><title>Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile</title><source>Free E- Journals</source><creator>Gencer, Can ; Topolnik, Marko ; Ďurina, Viliam ; Demirci, Emin ; Kahveci, Ensar B ; Ali Gürbüz Ondřej Lukáš ; Bartók, József ; Gierlach, Grzegorz ; Hartman, František ; Yılmaz, Ufuk ; Doğan, Mehmet ; Mandouh, Mohamed ; Fragkoulis, Marios ; Katsifodimos, Asterios</creator><creatorcontrib>Gencer, Can ; Topolnik, Marko ; Ďurina, Viliam ; Demirci, Emin ; Kahveci, Ensar B ; Ali Gürbüz Ondřej Lukáš ; Bartók, József ; Gierlach, Grzegorz ; Hartman, František ; Yılmaz, Ufuk ; Doğan, Mehmet ; Mandouh, Mohamed ; Fragkoulis, Marios ; Katsifodimos, Asterios</creatorcontrib><description>Jet is an open-source, high-performance, distributed stream processor built at Hazelcast during the last five years. Jet was engineered with millisecond latency on the 99.99th percentile as its primary design goal. Originally Jet's purpose was to be an execution engine that performs complex business logic on top of streams generated by Hazelcast's In-memory Data Grid (IMDG): a set of high-performance, in-memory, partitioned and replicated data structures. With time, Jet evolved into a full-fledged, scale-out stream processor that can handle out-of-order streams and exactly-once processing guarantees. Jet's end-to-end latency lies in the order of milliseconds, and its throughput in the order of millions of events per CPU-core. This paper presents main design decisions we made in order to maximize the performance per CPU-core, alongside lessons learned, and an empirical performance evaluation.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Data structures ; Microprocessors ; Performance evaluation ; Streams</subject><ispartof>arXiv.org, 2021-03</ispartof><rights>2021. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>780,784</link.rule.ids></links><search><creatorcontrib>Gencer, Can</creatorcontrib><creatorcontrib>Topolnik, Marko</creatorcontrib><creatorcontrib>Ďurina, Viliam</creatorcontrib><creatorcontrib>Demirci, Emin</creatorcontrib><creatorcontrib>Kahveci, Ensar B</creatorcontrib><creatorcontrib>Ali Gürbüz Ondřej Lukáš</creatorcontrib><creatorcontrib>Bartók, József</creatorcontrib><creatorcontrib>Gierlach, Grzegorz</creatorcontrib><creatorcontrib>Hartman, František</creatorcontrib><creatorcontrib>Yılmaz, Ufuk</creatorcontrib><creatorcontrib>Doğan, Mehmet</creatorcontrib><creatorcontrib>Mandouh, Mohamed</creatorcontrib><creatorcontrib>Fragkoulis, Marios</creatorcontrib><creatorcontrib>Katsifodimos, Asterios</creatorcontrib><title>Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile</title><title>arXiv.org</title><description>Jet is an open-source, high-performance, distributed stream processor built at Hazelcast during the last five years. Jet was engineered with millisecond latency on the 99.99th percentile as its primary design goal. Originally Jet's purpose was to be an execution engine that performs complex business logic on top of streams generated by Hazelcast's In-memory Data Grid (IMDG): a set of high-performance, in-memory, partitioned and replicated data structures. With time, Jet evolved into a full-fledged, scale-out stream processor that can handle out-of-order streams and exactly-once processing guarantees. Jet's end-to-end latency lies in the order of milliseconds, and its throughput in the order of millions of events per CPU-core. This paper presents main design decisions we made in order to maximize the performance per CPU-core, alongside lessons learned, and an empirical performance evaluation.</description><subject>Data structures</subject><subject>Microprocessors</subject><subject>Performance evaluation</subject><subject>Streams</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><recordid>eNqNyrEKwjAUQNEgCBbtPzxwrsSk1cZVLCIOBd1LCE_bEhNNXhH9eh38AKc7nDtiiZBymZW5EBOWxthzzsVqLYpCJqza6zdaoyPBAWkDR__MrCZ05gUnCqhvUAdvMMbOXUETUIug1EIpaqHGYNBRZ3HGxhdtI6a_Ttm82p23--we_GPASE3vh-C-1IiCS54XouTyv-sDImk56A</recordid><startdate>20210318</startdate><enddate>20210318</enddate><creator>Gencer, Can</creator><creator>Topolnik, Marko</creator><creator>Ďurina, Viliam</creator><creator>Demirci, Emin</creator><creator>Kahveci, Ensar B</creator><creator>Ali Gürbüz Ondřej Lukáš</creator><creator>Bartók, József</creator><creator>Gierlach, Grzegorz</creator><creator>Hartman, František</creator><creator>Yılmaz, Ufuk</creator><creator>Doğan, Mehmet</creator><creator>Mandouh, Mohamed</creator><creator>Fragkoulis, Marios</creator><creator>Katsifodimos, Asterios</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20210318</creationdate><title>Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile</title><author>Gencer, Can ; Topolnik, Marko ; Ďurina, Viliam ; Demirci, Emin ; Kahveci, Ensar B ; Ali Gürbüz Ondřej Lukáš ; Bartók, József ; Gierlach, Grzegorz ; Hartman, František ; Yılmaz, Ufuk ; Doğan, Mehmet ; Mandouh, Mohamed ; Fragkoulis, Marios ; Katsifodimos, Asterios</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_25030452803</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Data structures</topic><topic>Microprocessors</topic><topic>Performance evaluation</topic><topic>Streams</topic><toplevel>online_resources</toplevel><creatorcontrib>Gencer, Can</creatorcontrib><creatorcontrib>Topolnik, Marko</creatorcontrib><creatorcontrib>Ďurina, Viliam</creatorcontrib><creatorcontrib>Demirci, Emin</creatorcontrib><creatorcontrib>Kahveci, Ensar B</creatorcontrib><creatorcontrib>Ali Gürbüz Ondřej Lukáš</creatorcontrib><creatorcontrib>Bartók, József</creatorcontrib><creatorcontrib>Gierlach, Grzegorz</creatorcontrib><creatorcontrib>Hartman, František</creatorcontrib><creatorcontrib>Yılmaz, Ufuk</creatorcontrib><creatorcontrib>Doğan, Mehmet</creatorcontrib><creatorcontrib>Mandouh, Mohamed</creatorcontrib><creatorcontrib>Fragkoulis, Marios</creatorcontrib><creatorcontrib>Katsifodimos, Asterios</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Access via ProQuest (Open Access)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Gencer, Can</au><au>Topolnik, Marko</au><au>Ďurina, Viliam</au><au>Demirci, Emin</au><au>Kahveci, Ensar B</au><au>Ali Gürbüz Ondřej Lukáš</au><au>Bartók, József</au><au>Gierlach, Grzegorz</au><au>Hartman, František</au><au>Yılmaz, Ufuk</au><au>Doğan, Mehmet</au><au>Mandouh, Mohamed</au><au>Fragkoulis, Marios</au><au>Katsifodimos, Asterios</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile</atitle><jtitle>arXiv.org</jtitle><date>2021-03-18</date><risdate>2021</risdate><eissn>2331-8422</eissn><abstract>Jet is an open-source, high-performance, distributed stream processor built at Hazelcast during the last five years. Jet was engineered with millisecond latency on the 99.99th percentile as its primary design goal. Originally Jet's purpose was to be an execution engine that performs complex business logic on top of streams generated by Hazelcast's In-memory Data Grid (IMDG): a set of high-performance, in-memory, partitioned and replicated data structures. With time, Jet evolved into a full-fledged, scale-out stream processor that can handle out-of-order streams and exactly-once processing guarantees. Jet's end-to-end latency lies in the order of milliseconds, and its throughput in the order of millions of events per CPU-core. This paper presents main design decisions we made in order to maximize the performance per CPU-core, alongside lessons learned, and an empirical performance evaluation.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | EISSN: 2331-8422 |
| ispartof | arXiv.org, 2021-03 |
| issn | 2331-8422 |
| language | eng |
| recordid | cdi_proquest_journals_2503045280 |
| source | Free E- Journals |
| subjects | Data structures Microprocessors Performance evaluation Streams |
| title | Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-20T17%3A37%3A31IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Hazelcast%20Jet:%20Low-latency%20Stream%20Processing%20at%20the%2099.99th%20Percentile&rft.jtitle=arXiv.org&rft.au=Gencer,%20Can&rft.date=2021-03-18&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2503045280%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2503045280&rft_id=info:pmid/&rfr_iscdi=true |