PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory

Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tai...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on parallel and distributed systems 2023-03, Vol.34 (3), p.843-855
Hauptverfasser: Zhang, Yiwen, Zhou, Jian, Min, Xinhao, Ge, Song, Wan, Jiguang, Yao, Ting, Wang, Daohui
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 855
container_issue 3
container_start_page 843
container_title IEEE transactions on parallel and distributed systems
container_volume 34
creator Zhang, Yiwen
Zhou, Jian
Min, Xinhao
Ge, Song
Wan, Jiguang
Yao, Ting
Wang, Daohui
description Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan 2.5\times 2.5× , 3.2\times 3.2× , and 2.8\times 2.8× on average, respectively. Moreover, the file system built with PetaKV achieves 1.2\times 1.2× to 6.4\times 6.4× speedup compared to those built with tree-based KV stores on the metadata operations.
doi_str_mv 10.1109/TPDS.2022.3232382
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_ieee_primary_9999527</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9999527</ieee_id><sourcerecordid>2767316561</sourcerecordid><originalsourceid>FETCH-LOGICAL-c245t-73a795c6e2a1b8541e65ea0dbd5da749aa924ccfec9734d7f35d2ab51a4f94d53</originalsourceid><addsrcrecordid>eNo9kE1LAzEQhhdRsFZ_gHgJeN6aZJPNxpsfrUpbLLQWbyHdTCRlu1uT3cP-e1NanDnMB-87A0-S3BI8IgTLh9XidTmimNJRRmMW9CwZEM6LlJIiO489ZjyVlMjL5CqELcaEccwGyfcCWj1dP6LnzlXG1T9obK0rHdQtmkKfrnXVAVq2jQdkG48mropjH1rYoXm0Gt1q1NRoAT64uI22Oewa318nF1ZXAW5OdZh8Tcarl_d09vn28fI0S0vKeJuKTAvJyxyoJpuCMwI5B43NxnCjBZNaS8rK0kIpRcaMsBk3VG840cxKZng2TO6Pd_e--e0gtGrbdL6OLxUVuchIznMSVeSoKn0Tgger9t7ttO8VweoAUB0AqgNAdQIYPXdHjwOAf72MwanI_gBmj2w_</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2767316561</pqid></control><display><type>article</type><title>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</title><source>IEEE Electronic Library (IEL)</source><creator>Zhang, Yiwen ; Zhou, Jian ; Min, Xinhao ; Ge, Song ; Wan, Jiguang ; Yao, Ting ; Wang, Daohui</creator><creatorcontrib>Zhang, Yiwen ; Zhou, Jian ; Min, Xinhao ; Ge, Song ; Wan, Jiguang ; Yao, Ting ; Wang, Daohui</creatorcontrib><description><![CDATA[Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan <inline-formula><tex-math notation="LaTeX">2.5\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq1-3232382.gif"/> </inline-formula>, <inline-formula><tex-math notation="LaTeX">3.2\times</tex-math> <mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq2-3232382.gif"/> </inline-formula>, and <inline-formula><tex-math notation="LaTeX">2.8\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq3-3232382.gif"/> </inline-formula> on average, respectively. Moreover, the file system built with PetaKV achieves <inline-formula><tex-math notation="LaTeX">1.2\times</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq4-3232382.gif"/> </inline-formula> to <inline-formula><tex-math notation="LaTeX">6.4\times</tex-math> <mml:math><mml:mrow><mml:mn>6</mml:mn><mml:mo>.</mml:mo><mml:mn>4</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq5-3232382.gif"/> </inline-formula> speedup compared to those built with tree-based KV stores on the metadata operations.]]></description><identifier>ISSN: 1045-9219</identifier><identifier>EISSN: 1558-2183</identifier><identifier>DOI: 10.1109/TPDS.2022.3232382</identifier><identifier>CODEN: ITDSEO</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Buildings ; Complexity theory ; Data management ; file system metadata ; File systems ; hash index ; Indexing ; Key-Value Store ; log-structure ; Metadata ; persistent memory ; Stores ; Three-dimensional displays ; Throughput</subject><ispartof>IEEE transactions on parallel and distributed systems, 2023-03, Vol.34 (3), p.843-855</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c245t-73a795c6e2a1b8541e65ea0dbd5da749aa924ccfec9734d7f35d2ab51a4f94d53</cites><orcidid>0000-0002-9358-9373 ; 0000-0001-5279-4816 ; 0000-0001-5295-4680 ; 0000-0002-4160-9475 ; 0000-0001-6216-1537</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9999527$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,777,781,793,27905,27906,54739</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/9999527$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Zhang, Yiwen</creatorcontrib><creatorcontrib>Zhou, Jian</creatorcontrib><creatorcontrib>Min, Xinhao</creatorcontrib><creatorcontrib>Ge, Song</creatorcontrib><creatorcontrib>Wan, Jiguang</creatorcontrib><creatorcontrib>Yao, Ting</creatorcontrib><creatorcontrib>Wang, Daohui</creatorcontrib><title>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</title><title>IEEE transactions on parallel and distributed systems</title><addtitle>TPDS</addtitle><description><![CDATA[Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan <inline-formula><tex-math notation="LaTeX">2.5\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq1-3232382.gif"/> </inline-formula>, <inline-formula><tex-math notation="LaTeX">3.2\times</tex-math> <mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq2-3232382.gif"/> </inline-formula>, and <inline-formula><tex-math notation="LaTeX">2.8\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq3-3232382.gif"/> </inline-formula> on average, respectively. Moreover, the file system built with PetaKV achieves <inline-formula><tex-math notation="LaTeX">1.2\times</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq4-3232382.gif"/> </inline-formula> to <inline-formula><tex-math notation="LaTeX">6.4\times</tex-math> <mml:math><mml:mrow><mml:mn>6</mml:mn><mml:mo>.</mml:mo><mml:mn>4</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq5-3232382.gif"/> </inline-formula> speedup compared to those built with tree-based KV stores on the metadata operations.]]></description><subject>Buildings</subject><subject>Complexity theory</subject><subject>Data management</subject><subject>file system metadata</subject><subject>File systems</subject><subject>hash index</subject><subject>Indexing</subject><subject>Key-Value Store</subject><subject>log-structure</subject><subject>Metadata</subject><subject>persistent memory</subject><subject>Stores</subject><subject>Three-dimensional displays</subject><subject>Throughput</subject><issn>1045-9219</issn><issn>1558-2183</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNo9kE1LAzEQhhdRsFZ_gHgJeN6aZJPNxpsfrUpbLLQWbyHdTCRlu1uT3cP-e1NanDnMB-87A0-S3BI8IgTLh9XidTmimNJRRmMW9CwZEM6LlJIiO489ZjyVlMjL5CqELcaEccwGyfcCWj1dP6LnzlXG1T9obK0rHdQtmkKfrnXVAVq2jQdkG48mropjH1rYoXm0Gt1q1NRoAT64uI22Oewa318nF1ZXAW5OdZh8Tcarl_d09vn28fI0S0vKeJuKTAvJyxyoJpuCMwI5B43NxnCjBZNaS8rK0kIpRcaMsBk3VG840cxKZng2TO6Pd_e--e0gtGrbdL6OLxUVuchIznMSVeSoKn0Tgger9t7ttO8VweoAUB0AqgNAdQIYPXdHjwOAf72MwanI_gBmj2w_</recordid><startdate>20230301</startdate><enddate>20230301</enddate><creator>Zhang, Yiwen</creator><creator>Zhou, Jian</creator><creator>Min, Xinhao</creator><creator>Ge, Song</creator><creator>Wan, Jiguang</creator><creator>Yao, Ting</creator><creator>Wang, Daohui</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0002-9358-9373</orcidid><orcidid>https://orcid.org/0000-0001-5279-4816</orcidid><orcidid>https://orcid.org/0000-0001-5295-4680</orcidid><orcidid>https://orcid.org/0000-0002-4160-9475</orcidid><orcidid>https://orcid.org/0000-0001-6216-1537</orcidid></search><sort><creationdate>20230301</creationdate><title>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</title><author>Zhang, Yiwen ; Zhou, Jian ; Min, Xinhao ; Ge, Song ; Wan, Jiguang ; Yao, Ting ; Wang, Daohui</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c245t-73a795c6e2a1b8541e65ea0dbd5da749aa924ccfec9734d7f35d2ab51a4f94d53</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Buildings</topic><topic>Complexity theory</topic><topic>Data management</topic><topic>file system metadata</topic><topic>File systems</topic><topic>hash index</topic><topic>Indexing</topic><topic>Key-Value Store</topic><topic>log-structure</topic><topic>Metadata</topic><topic>persistent memory</topic><topic>Stores</topic><topic>Three-dimensional displays</topic><topic>Throughput</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhang, Yiwen</creatorcontrib><creatorcontrib>Zhou, Jian</creatorcontrib><creatorcontrib>Min, Xinhao</creatorcontrib><creatorcontrib>Ge, Song</creatorcontrib><creatorcontrib>Wan, Jiguang</creatorcontrib><creatorcontrib>Yao, Ting</creatorcontrib><creatorcontrib>Wang, Daohui</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on parallel and distributed systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Zhang, Yiwen</au><au>Zhou, Jian</au><au>Min, Xinhao</au><au>Ge, Song</au><au>Wan, Jiguang</au><au>Yao, Ting</au><au>Wang, Daohui</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</atitle><jtitle>IEEE transactions on parallel and distributed systems</jtitle><stitle>TPDS</stitle><date>2023-03-01</date><risdate>2023</risdate><volume>34</volume><issue>3</issue><spage>843</spage><epage>855</epage><pages>843-855</pages><issn>1045-9219</issn><eissn>1558-2183</eissn><coden>ITDSEO</coden><abstract><![CDATA[Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan <inline-formula><tex-math notation="LaTeX">2.5\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq1-3232382.gif"/> </inline-formula>, <inline-formula><tex-math notation="LaTeX">3.2\times</tex-math> <mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq2-3232382.gif"/> </inline-formula>, and <inline-formula><tex-math notation="LaTeX">2.8\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq3-3232382.gif"/> </inline-formula> on average, respectively. Moreover, the file system built with PetaKV achieves <inline-formula><tex-math notation="LaTeX">1.2\times</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq4-3232382.gif"/> </inline-formula> to <inline-formula><tex-math notation="LaTeX">6.4\times</tex-math> <mml:math><mml:mrow><mml:mn>6</mml:mn><mml:mo>.</mml:mo><mml:mn>4</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq5-3232382.gif"/> </inline-formula> speedup compared to those built with tree-based KV stores on the metadata operations.]]></abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/TPDS.2022.3232382</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0002-9358-9373</orcidid><orcidid>https://orcid.org/0000-0001-5279-4816</orcidid><orcidid>https://orcid.org/0000-0001-5295-4680</orcidid><orcidid>https://orcid.org/0000-0002-4160-9475</orcidid><orcidid>https://orcid.org/0000-0001-6216-1537</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 1045-9219
ispartof IEEE transactions on parallel and distributed systems, 2023-03, Vol.34 (3), p.843-855
issn 1045-9219
1558-2183
language eng
recordid cdi_ieee_primary_9999527
source IEEE Electronic Library (IEL)
subjects Buildings
Complexity theory
Data management
file system metadata
File systems
hash index
Indexing
Key-Value Store
log-structure
Metadata
persistent memory
Stores
Three-dimensional displays
Throughput
title PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-19T02%3A07%3A48IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=PetaKV:%20Building%20Efficient%20Key-Value%20Store%20for%20File%20System%20Metadata%20on%20Persistent%20Memory&rft.jtitle=IEEE%20transactions%20on%20parallel%20and%20distributed%20systems&rft.au=Zhang,%20Yiwen&rft.date=2023-03-01&rft.volume=34&rft.issue=3&rft.spage=843&rft.epage=855&rft.pages=843-855&rft.issn=1045-9219&rft.eissn=1558-2183&rft.coden=ITDSEO&rft_id=info:doi/10.1109/TPDS.2022.3232382&rft_dat=%3Cproquest_RIE%3E2767316561%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2767316561&rft_id=info:pmid/&rft_ieee_id=9999527&rfr_iscdi=true