PALF: Replicated Write-Ahead Logging for Distributed Databases
Distributed databases have been widely researched and developed in recent years due to their scalability, availability, and consistency guarantees. The write-ahead logging (WAL) system is one of the most vital components in a database. It is still a non-trivial problem to design a replicated logging...
Gespeichert in:
| Veröffentlicht in: | Proceedings of the VLDB Endowment 2024-08, Vol.17 (12), p.3745-3758 |
|---|---|
| Hauptverfasser: | , , , , , , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 3758 |
|---|---|
| container_issue | 12 |
| container_start_page | 3745 |
| container_title | Proceedings of the VLDB Endowment |
| container_volume | 17 |
| creator | Han, Fusheng Liu, Hao Chen, Bin Jia, Debin Zhou, Jianfeng Teng, Xuwang Yang, Chuanhui Xi, Huafeng Tian, Wei Tao, Shuning Wang, Sen Xu, Quanqing Yang, Zhenkun |
| description | Distributed databases have been widely researched and developed in recent years due to their scalability, availability, and consistency guarantees. The write-ahead logging (WAL) system is one of the most vital components in a database. It is still a non-trivial problem to design a replicated logging system as the foundation of a distributed database with the power of ACID transactions. This paper proposes PALF, a Paxos-backed Append-only Log File System, to address these challenges. The basic idea behind PALF is to co-design the logging system with the entire database for supporting database-specific functions and to abstract the functions as PALF primitives to power other distributed systems. Many database functions, including transaction processing, database restore, and physical standby databases, have been built based on PALF primitives. Evaluation shows that PALF greatly outperforms well-known implementations of consensus protocols and is fully competent for distributed database workloads. PALF has been deployed as a component of the OceanBase 4.0 database and has been made open-source along with it. |
| doi_str_mv | 10.14778/3685800.3685803 |
| format | Article |
| fullrecord | <record><control><sourceid>crossref</sourceid><recordid>TN_cdi_crossref_primary_10_14778_3685800_3685803</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>10_14778_3685800_3685803</sourcerecordid><originalsourceid>FETCH-crossref_primary_10_14778_3685800_36858033</originalsourceid><addsrcrecordid>eNpjYJAwNNAzNDE3t9A3NrMwtTAw0IPQxkwMnEaGpga6FgaW5ixIbA4GruLiLAMDMwszQwtOBrsARx83K4Wg1IKczOTEktQUhfCizJJUXceM1MQUBZ_89PTMvHSFtPwiBZfM4pKizKRSkBqXxJLEpMTi1GIeBta0xJziVF4ozc1g4OYa4uyhm1yUX1xclJoWX1CUmZtYVBlvaBAPdmc81J1Q2tiYDC0AfjNDWg</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>PALF: Replicated Write-Ahead Logging for Distributed Databases</title><source>ACM Digital Library Complete</source><creator>Han, Fusheng ; Liu, Hao ; Chen, Bin ; Jia, Debin ; Zhou, Jianfeng ; Teng, Xuwang ; Yang, Chuanhui ; Xi, Huafeng ; Tian, Wei ; Tao, Shuning ; Wang, Sen ; Xu, Quanqing ; Yang, Zhenkun</creator><creatorcontrib>Han, Fusheng ; Liu, Hao ; Chen, Bin ; Jia, Debin ; Zhou, Jianfeng ; Teng, Xuwang ; Yang, Chuanhui ; Xi, Huafeng ; Tian, Wei ; Tao, Shuning ; Wang, Sen ; Xu, Quanqing ; Yang, Zhenkun</creatorcontrib><description>Distributed databases have been widely researched and developed in recent years due to their scalability, availability, and consistency guarantees. The write-ahead logging (WAL) system is one of the most vital components in a database. It is still a non-trivial problem to design a replicated logging system as the foundation of a distributed database with the power of ACID transactions. This paper proposes PALF, a Paxos-backed Append-only Log File System, to address these challenges. The basic idea behind PALF is to co-design the logging system with the entire database for supporting database-specific functions and to abstract the functions as PALF primitives to power other distributed systems. Many database functions, including transaction processing, database restore, and physical standby databases, have been built based on PALF primitives. Evaluation shows that PALF greatly outperforms well-known implementations of consensus protocols and is fully competent for distributed database workloads. PALF has been deployed as a component of the OceanBase 4.0 database and has been made open-source along with it.</description><identifier>ISSN: 2150-8097</identifier><identifier>EISSN: 2150-8097</identifier><identifier>DOI: 10.14778/3685800.3685803</identifier><language>eng</language><ispartof>Proceedings of the VLDB Endowment, 2024-08, Vol.17 (12), p.3745-3758</ispartof><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-crossref_primary_10_14778_3685800_36858033</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids></links><search><creatorcontrib>Han, Fusheng</creatorcontrib><creatorcontrib>Liu, Hao</creatorcontrib><creatorcontrib>Chen, Bin</creatorcontrib><creatorcontrib>Jia, Debin</creatorcontrib><creatorcontrib>Zhou, Jianfeng</creatorcontrib><creatorcontrib>Teng, Xuwang</creatorcontrib><creatorcontrib>Yang, Chuanhui</creatorcontrib><creatorcontrib>Xi, Huafeng</creatorcontrib><creatorcontrib>Tian, Wei</creatorcontrib><creatorcontrib>Tao, Shuning</creatorcontrib><creatorcontrib>Wang, Sen</creatorcontrib><creatorcontrib>Xu, Quanqing</creatorcontrib><creatorcontrib>Yang, Zhenkun</creatorcontrib><title>PALF: Replicated Write-Ahead Logging for Distributed Databases</title><title>Proceedings of the VLDB Endowment</title><description>Distributed databases have been widely researched and developed in recent years due to their scalability, availability, and consistency guarantees. The write-ahead logging (WAL) system is one of the most vital components in a database. It is still a non-trivial problem to design a replicated logging system as the foundation of a distributed database with the power of ACID transactions. This paper proposes PALF, a Paxos-backed Append-only Log File System, to address these challenges. The basic idea behind PALF is to co-design the logging system with the entire database for supporting database-specific functions and to abstract the functions as PALF primitives to power other distributed systems. Many database functions, including transaction processing, database restore, and physical standby databases, have been built based on PALF primitives. Evaluation shows that PALF greatly outperforms well-known implementations of consensus protocols and is fully competent for distributed database workloads. PALF has been deployed as a component of the OceanBase 4.0 database and has been made open-source along with it.</description><issn>2150-8097</issn><issn>2150-8097</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNpjYJAwNNAzNDE3t9A3NrMwtTAw0IPQxkwMnEaGpga6FgaW5ixIbA4GruLiLAMDMwszQwtOBrsARx83K4Wg1IKczOTEktQUhfCizJJUXceM1MQUBZ_89PTMvHSFtPwiBZfM4pKizKRSkBqXxJLEpMTi1GIeBta0xJziVF4ozc1g4OYa4uyhm1yUX1xclJoWX1CUmZtYVBlvaBAPdmc81J1Q2tiYDC0AfjNDWg</recordid><startdate>202408</startdate><enddate>202408</enddate><creator>Han, Fusheng</creator><creator>Liu, Hao</creator><creator>Chen, Bin</creator><creator>Jia, Debin</creator><creator>Zhou, Jianfeng</creator><creator>Teng, Xuwang</creator><creator>Yang, Chuanhui</creator><creator>Xi, Huafeng</creator><creator>Tian, Wei</creator><creator>Tao, Shuning</creator><creator>Wang, Sen</creator><creator>Xu, Quanqing</creator><creator>Yang, Zhenkun</creator><scope>AAYXX</scope><scope>CITATION</scope></search><sort><creationdate>202408</creationdate><title>PALF: Replicated Write-Ahead Logging for Distributed Databases</title><author>Han, Fusheng ; Liu, Hao ; Chen, Bin ; Jia, Debin ; Zhou, Jianfeng ; Teng, Xuwang ; Yang, Chuanhui ; Xi, Huafeng ; Tian, Wei ; Tao, Shuning ; Wang, Sen ; Xu, Quanqing ; Yang, Zhenkun</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-crossref_primary_10_14778_3685800_36858033</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Han, Fusheng</creatorcontrib><creatorcontrib>Liu, Hao</creatorcontrib><creatorcontrib>Chen, Bin</creatorcontrib><creatorcontrib>Jia, Debin</creatorcontrib><creatorcontrib>Zhou, Jianfeng</creatorcontrib><creatorcontrib>Teng, Xuwang</creatorcontrib><creatorcontrib>Yang, Chuanhui</creatorcontrib><creatorcontrib>Xi, Huafeng</creatorcontrib><creatorcontrib>Tian, Wei</creatorcontrib><creatorcontrib>Tao, Shuning</creatorcontrib><creatorcontrib>Wang, Sen</creatorcontrib><creatorcontrib>Xu, Quanqing</creatorcontrib><creatorcontrib>Yang, Zhenkun</creatorcontrib><collection>CrossRef</collection><jtitle>Proceedings of the VLDB Endowment</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Han, Fusheng</au><au>Liu, Hao</au><au>Chen, Bin</au><au>Jia, Debin</au><au>Zhou, Jianfeng</au><au>Teng, Xuwang</au><au>Yang, Chuanhui</au><au>Xi, Huafeng</au><au>Tian, Wei</au><au>Tao, Shuning</au><au>Wang, Sen</au><au>Xu, Quanqing</au><au>Yang, Zhenkun</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>PALF: Replicated Write-Ahead Logging for Distributed Databases</atitle><jtitle>Proceedings of the VLDB Endowment</jtitle><date>2024-08</date><risdate>2024</risdate><volume>17</volume><issue>12</issue><spage>3745</spage><epage>3758</epage><pages>3745-3758</pages><issn>2150-8097</issn><eissn>2150-8097</eissn><abstract>Distributed databases have been widely researched and developed in recent years due to their scalability, availability, and consistency guarantees. The write-ahead logging (WAL) system is one of the most vital components in a database. It is still a non-trivial problem to design a replicated logging system as the foundation of a distributed database with the power of ACID transactions. This paper proposes PALF, a Paxos-backed Append-only Log File System, to address these challenges. The basic idea behind PALF is to co-design the logging system with the entire database for supporting database-specific functions and to abstract the functions as PALF primitives to power other distributed systems. Many database functions, including transaction processing, database restore, and physical standby databases, have been built based on PALF primitives. Evaluation shows that PALF greatly outperforms well-known implementations of consensus protocols and is fully competent for distributed database workloads. PALF has been deployed as a component of the OceanBase 4.0 database and has been made open-source along with it.</abstract><doi>10.14778/3685800.3685803</doi></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2150-8097 |
| ispartof | Proceedings of the VLDB Endowment, 2024-08, Vol.17 (12), p.3745-3758 |
| issn | 2150-8097 2150-8097 |
| language | eng |
| recordid | cdi_crossref_primary_10_14778_3685800_3685803 |
| source | ACM Digital Library Complete |
| title | PALF: Replicated Write-Ahead Logging for Distributed Databases |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-07T15%3A44%3A07IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-crossref&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=PALF:%20Replicated%20Write-Ahead%20Logging%20for%20Distributed%20Databases&rft.jtitle=Proceedings%20of%20the%20VLDB%20Endowment&rft.au=Han,%20Fusheng&rft.date=2024-08&rft.volume=17&rft.issue=12&rft.spage=3745&rft.epage=3758&rft.pages=3745-3758&rft.issn=2150-8097&rft.eissn=2150-8097&rft_id=info:doi/10.14778/3685800.3685803&rft_dat=%3Ccrossref%3E10_14778_3685800_3685803%3C/crossref%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |