LogPlayer: Fault-tolerant Exactly-once Delivery using gRPC Asynchronous Streaming
In this paper, we present the design of our LogPlayer that is a component responsible for fault-tolerant delivery of transactional mutations recorded on a WAL to the backend storage shards. LogPlayer relies on gRPC for asynchronous streaming. However, the design provided in this paper can be used wi...
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | Roohitavaf, Mohammad Ren, Kun Zhang, Gene Ben-romdhane, Sami |
| description | In this paper, we present the design of our LogPlayer that is a component
responsible for fault-tolerant delivery of transactional mutations recorded on
a WAL to the backend storage shards. LogPlayer relies on gRPC for asynchronous
streaming. However, the design provided in this paper can be used with other
asynchronous streaming platforms. We model check the correctness of LogPlayer
by TLA+. In particular, our TLA+ specification shows that LogPlayer guarantees
in-order exactly-once delivery of WAL entries to the storage shards, even in
the presence of shards or LogPlayer failures. Our experiments show LogPlayer is
capable of efficient delivery with sub-millisecond latency, and it is
significantly more efficient than Apache Kafka for designing a WAL system with
exactly-once guarantee. |
| doi_str_mv | 10.48550/arxiv.1911.11286 |
| format | Article |
| fullrecord | <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_1911_11286</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1911_11286</sourcerecordid><originalsourceid>FETCH-LOGICAL-a676-6bc1d060d1c1d4a7c20696a4a07fec448bb97dd2dcbd086dbaaeb2f699623a9a3</originalsourceid><addsrcrecordid>eNotz71OwzAYhWEvDKhwAUz4BhxsJ3Vitiq0UCkSBbpHn38SIrk2cpKqvnvawvQORzrSg9ADo1lRLZf0CeJpOGZMMpYxxitxiz6a0O8cJBuf8QZmN5EpOBvBT3h9Aj25RILXFr9YNxxtTHgeB9_j_nNX49WYvP6OwYd5xF9TtHA4b3fopgM32vv_LtB-s97Xb6R5f93Wq4aAKAURSjNDBTXs3AJKzamQAgqgZWd1UVRKydIYbrQytBJGAVjFOyGl4DlIyBfo8e_2amp_4nCAmNqLrb3a8l_89Us5</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>LogPlayer: Fault-tolerant Exactly-once Delivery using gRPC Asynchronous Streaming</title><source>arXiv.org</source><creator>Roohitavaf, Mohammad ; Ren, Kun ; Zhang, Gene ; Ben-romdhane, Sami</creator><creatorcontrib>Roohitavaf, Mohammad ; Ren, Kun ; Zhang, Gene ; Ben-romdhane, Sami</creatorcontrib><description>In this paper, we present the design of our LogPlayer that is a component
responsible for fault-tolerant delivery of transactional mutations recorded on
a WAL to the backend storage shards. LogPlayer relies on gRPC for asynchronous
streaming. However, the design provided in this paper can be used with other
asynchronous streaming platforms. We model check the correctness of LogPlayer
by TLA+. In particular, our TLA+ specification shows that LogPlayer guarantees
in-order exactly-once delivery of WAL entries to the storage shards, even in
the presence of shards or LogPlayer failures. Our experiments show LogPlayer is
capable of efficient delivery with sub-millisecond latency, and it is
significantly more efficient than Apache Kafka for designing a WAL system with
exactly-once guarantee.</description><identifier>DOI: 10.48550/arxiv.1911.11286</identifier><language>eng</language><subject>Computer Science - Databases ; Computer Science - Distributed, Parallel, and Cluster Computing</subject><creationdate>2019-11</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/1911.11286$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.1911.11286$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Roohitavaf, Mohammad</creatorcontrib><creatorcontrib>Ren, Kun</creatorcontrib><creatorcontrib>Zhang, Gene</creatorcontrib><creatorcontrib>Ben-romdhane, Sami</creatorcontrib><title>LogPlayer: Fault-tolerant Exactly-once Delivery using gRPC Asynchronous Streaming</title><description>In this paper, we present the design of our LogPlayer that is a component
responsible for fault-tolerant delivery of transactional mutations recorded on
a WAL to the backend storage shards. LogPlayer relies on gRPC for asynchronous
streaming. However, the design provided in this paper can be used with other
asynchronous streaming platforms. We model check the correctness of LogPlayer
by TLA+. In particular, our TLA+ specification shows that LogPlayer guarantees
in-order exactly-once delivery of WAL entries to the storage shards, even in
the presence of shards or LogPlayer failures. Our experiments show LogPlayer is
capable of efficient delivery with sub-millisecond latency, and it is
significantly more efficient than Apache Kafka for designing a WAL system with
exactly-once guarantee.</description><subject>Computer Science - Databases</subject><subject>Computer Science - Distributed, Parallel, and Cluster Computing</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotz71OwzAYhWEvDKhwAUz4BhxsJ3Vitiq0UCkSBbpHn38SIrk2cpKqvnvawvQORzrSg9ADo1lRLZf0CeJpOGZMMpYxxitxiz6a0O8cJBuf8QZmN5EpOBvBT3h9Aj25RILXFr9YNxxtTHgeB9_j_nNX49WYvP6OwYd5xF9TtHA4b3fopgM32vv_LtB-s97Xb6R5f93Wq4aAKAURSjNDBTXs3AJKzamQAgqgZWd1UVRKydIYbrQytBJGAVjFOyGl4DlIyBfo8e_2amp_4nCAmNqLrb3a8l_89Us5</recordid><startdate>20191125</startdate><enddate>20191125</enddate><creator>Roohitavaf, Mohammad</creator><creator>Ren, Kun</creator><creator>Zhang, Gene</creator><creator>Ben-romdhane, Sami</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20191125</creationdate><title>LogPlayer: Fault-tolerant Exactly-once Delivery using gRPC Asynchronous Streaming</title><author>Roohitavaf, Mohammad ; Ren, Kun ; Zhang, Gene ; Ben-romdhane, Sami</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a676-6bc1d060d1c1d4a7c20696a4a07fec448bb97dd2dcbd086dbaaeb2f699623a9a3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Computer Science - Databases</topic><topic>Computer Science - Distributed, Parallel, and Cluster Computing</topic><toplevel>online_resources</toplevel><creatorcontrib>Roohitavaf, Mohammad</creatorcontrib><creatorcontrib>Ren, Kun</creatorcontrib><creatorcontrib>Zhang, Gene</creatorcontrib><creatorcontrib>Ben-romdhane, Sami</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Roohitavaf, Mohammad</au><au>Ren, Kun</au><au>Zhang, Gene</au><au>Ben-romdhane, Sami</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>LogPlayer: Fault-tolerant Exactly-once Delivery using gRPC Asynchronous Streaming</atitle><date>2019-11-25</date><risdate>2019</risdate><abstract>In this paper, we present the design of our LogPlayer that is a component
responsible for fault-tolerant delivery of transactional mutations recorded on
a WAL to the backend storage shards. LogPlayer relies on gRPC for asynchronous
streaming. However, the design provided in this paper can be used with other
asynchronous streaming platforms. We model check the correctness of LogPlayer
by TLA+. In particular, our TLA+ specification shows that LogPlayer guarantees
in-order exactly-once delivery of WAL entries to the storage shards, even in
the presence of shards or LogPlayer failures. Our experiments show LogPlayer is
capable of efficient delivery with sub-millisecond latency, and it is
significantly more efficient than Apache Kafka for designing a WAL system with
exactly-once guarantee.</abstract><doi>10.48550/arxiv.1911.11286</doi><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | DOI: 10.48550/arxiv.1911.11286 |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_arxiv_primary_1911_11286 |
| source | arXiv.org |
| subjects | Computer Science - Databases Computer Science - Distributed, Parallel, and Cluster Computing |
| title | LogPlayer: Fault-tolerant Exactly-once Delivery using gRPC Asynchronous Streaming |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-14T13%3A39%3A24IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=LogPlayer:%20Fault-tolerant%20Exactly-once%20Delivery%20using%20gRPC%20Asynchronous%20Streaming&rft.au=Roohitavaf,%20Mohammad&rft.date=2019-11-25&rft_id=info:doi/10.48550/arxiv.1911.11286&rft_dat=%3Carxiv_GOX%3E1911_11286%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |