PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory
Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tai...
Gespeichert in:
Veröffentlicht in: | IEEE transactions on parallel and distributed systems 2023-03, Vol.34 (3), p.843-855 |
---|---|
Hauptverfasser: | , , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 855 |
---|---|
container_issue | 3 |
container_start_page | 843 |
container_title | IEEE transactions on parallel and distributed systems |
container_volume | 34 |
creator | Zhang, Yiwen Zhou, Jian Min, Xinhao Ge, Song Wan, Jiguang Yao, Ting Wang, Daohui |
description | Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan 2.5\times 2.5× , 3.2\times 3.2× , and 2.8\times 2.8× on average, respectively. Moreover, the file system built with PetaKV achieves 1.2\times 1.2× to 6.4\times 6.4× speedup compared to those built with tree-based KV stores on the metadata operations. |
doi_str_mv | 10.1109/TPDS.2022.3232382 |
format | Article |
fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_ieee_primary_9999527</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9999527</ieee_id><sourcerecordid>2767316561</sourcerecordid><originalsourceid>FETCH-LOGICAL-c245t-73a795c6e2a1b8541e65ea0dbd5da749aa924ccfec9734d7f35d2ab51a4f94d53</originalsourceid><addsrcrecordid>eNo9kE1LAzEQhhdRsFZ_gHgJeN6aZJPNxpsfrUpbLLQWbyHdTCRlu1uT3cP-e1NanDnMB-87A0-S3BI8IgTLh9XidTmimNJRRmMW9CwZEM6LlJIiO489ZjyVlMjL5CqELcaEccwGyfcCWj1dP6LnzlXG1T9obK0rHdQtmkKfrnXVAVq2jQdkG48mropjH1rYoXm0Gt1q1NRoAT64uI22Oewa318nF1ZXAW5OdZh8Tcarl_d09vn28fI0S0vKeJuKTAvJyxyoJpuCMwI5B43NxnCjBZNaS8rK0kIpRcaMsBk3VG840cxKZng2TO6Pd_e--e0gtGrbdL6OLxUVuchIznMSVeSoKn0Tgger9t7ttO8VweoAUB0AqgNAdQIYPXdHjwOAf72MwanI_gBmj2w_</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2767316561</pqid></control><display><type>article</type><title>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</title><source>IEEE Electronic Library (IEL)</source><creator>Zhang, Yiwen ; Zhou, Jian ; Min, Xinhao ; Ge, Song ; Wan, Jiguang ; Yao, Ting ; Wang, Daohui</creator><creatorcontrib>Zhang, Yiwen ; Zhou, Jian ; Min, Xinhao ; Ge, Song ; Wan, Jiguang ; Yao, Ting ; Wang, Daohui</creatorcontrib><description><![CDATA[Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan <inline-formula><tex-math notation="LaTeX">2.5\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq1-3232382.gif"/> </inline-formula>, <inline-formula><tex-math notation="LaTeX">3.2\times</tex-math> <mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq2-3232382.gif"/> </inline-formula>, and <inline-formula><tex-math notation="LaTeX">2.8\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq3-3232382.gif"/> </inline-formula> on average, respectively. Moreover, the file system built with PetaKV achieves <inline-formula><tex-math notation="LaTeX">1.2\times</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq4-3232382.gif"/> </inline-formula> to <inline-formula><tex-math notation="LaTeX">6.4\times</tex-math> <mml:math><mml:mrow><mml:mn>6</mml:mn><mml:mo>.</mml:mo><mml:mn>4</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq5-3232382.gif"/> </inline-formula> speedup compared to those built with tree-based KV stores on the metadata operations.]]></description><identifier>ISSN: 1045-9219</identifier><identifier>EISSN: 1558-2183</identifier><identifier>DOI: 10.1109/TPDS.2022.3232382</identifier><identifier>CODEN: ITDSEO</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Buildings ; Complexity theory ; Data management ; file system metadata ; File systems ; hash index ; Indexing ; Key-Value Store ; log-structure ; Metadata ; persistent memory ; Stores ; Three-dimensional displays ; Throughput</subject><ispartof>IEEE transactions on parallel and distributed systems, 2023-03, Vol.34 (3), p.843-855</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c245t-73a795c6e2a1b8541e65ea0dbd5da749aa924ccfec9734d7f35d2ab51a4f94d53</cites><orcidid>0000-0002-9358-9373 ; 0000-0001-5279-4816 ; 0000-0001-5295-4680 ; 0000-0002-4160-9475 ; 0000-0001-6216-1537</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9999527$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,777,781,793,27905,27906,54739</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/9999527$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Zhang, Yiwen</creatorcontrib><creatorcontrib>Zhou, Jian</creatorcontrib><creatorcontrib>Min, Xinhao</creatorcontrib><creatorcontrib>Ge, Song</creatorcontrib><creatorcontrib>Wan, Jiguang</creatorcontrib><creatorcontrib>Yao, Ting</creatorcontrib><creatorcontrib>Wang, Daohui</creatorcontrib><title>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</title><title>IEEE transactions on parallel and distributed systems</title><addtitle>TPDS</addtitle><description><![CDATA[Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan <inline-formula><tex-math notation="LaTeX">2.5\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq1-3232382.gif"/> </inline-formula>, <inline-formula><tex-math notation="LaTeX">3.2\times</tex-math> <mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq2-3232382.gif"/> </inline-formula>, and <inline-formula><tex-math notation="LaTeX">2.8\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq3-3232382.gif"/> </inline-formula> on average, respectively. Moreover, the file system built with PetaKV achieves <inline-formula><tex-math notation="LaTeX">1.2\times</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq4-3232382.gif"/> </inline-formula> to <inline-formula><tex-math notation="LaTeX">6.4\times</tex-math> <mml:math><mml:mrow><mml:mn>6</mml:mn><mml:mo>.</mml:mo><mml:mn>4</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq5-3232382.gif"/> </inline-formula> speedup compared to those built with tree-based KV stores on the metadata operations.]]></description><subject>Buildings</subject><subject>Complexity theory</subject><subject>Data management</subject><subject>file system metadata</subject><subject>File systems</subject><subject>hash index</subject><subject>Indexing</subject><subject>Key-Value Store</subject><subject>log-structure</subject><subject>Metadata</subject><subject>persistent memory</subject><subject>Stores</subject><subject>Three-dimensional displays</subject><subject>Throughput</subject><issn>1045-9219</issn><issn>1558-2183</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNo9kE1LAzEQhhdRsFZ_gHgJeN6aZJPNxpsfrUpbLLQWbyHdTCRlu1uT3cP-e1NanDnMB-87A0-S3BI8IgTLh9XidTmimNJRRmMW9CwZEM6LlJIiO489ZjyVlMjL5CqELcaEccwGyfcCWj1dP6LnzlXG1T9obK0rHdQtmkKfrnXVAVq2jQdkG48mropjH1rYoXm0Gt1q1NRoAT64uI22Oewa318nF1ZXAW5OdZh8Tcarl_d09vn28fI0S0vKeJuKTAvJyxyoJpuCMwI5B43NxnCjBZNaS8rK0kIpRcaMsBk3VG840cxKZng2TO6Pd_e--e0gtGrbdL6OLxUVuchIznMSVeSoKn0Tgger9t7ttO8VweoAUB0AqgNAdQIYPXdHjwOAf72MwanI_gBmj2w_</recordid><startdate>20230301</startdate><enddate>20230301</enddate><creator>Zhang, Yiwen</creator><creator>Zhou, Jian</creator><creator>Min, Xinhao</creator><creator>Ge, Song</creator><creator>Wan, Jiguang</creator><creator>Yao, Ting</creator><creator>Wang, Daohui</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0002-9358-9373</orcidid><orcidid>https://orcid.org/0000-0001-5279-4816</orcidid><orcidid>https://orcid.org/0000-0001-5295-4680</orcidid><orcidid>https://orcid.org/0000-0002-4160-9475</orcidid><orcidid>https://orcid.org/0000-0001-6216-1537</orcidid></search><sort><creationdate>20230301</creationdate><title>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</title><author>Zhang, Yiwen ; Zhou, Jian ; Min, Xinhao ; Ge, Song ; Wan, Jiguang ; Yao, Ting ; Wang, Daohui</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c245t-73a795c6e2a1b8541e65ea0dbd5da749aa924ccfec9734d7f35d2ab51a4f94d53</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Buildings</topic><topic>Complexity theory</topic><topic>Data management</topic><topic>file system metadata</topic><topic>File systems</topic><topic>hash index</topic><topic>Indexing</topic><topic>Key-Value Store</topic><topic>log-structure</topic><topic>Metadata</topic><topic>persistent memory</topic><topic>Stores</topic><topic>Three-dimensional displays</topic><topic>Throughput</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhang, Yiwen</creatorcontrib><creatorcontrib>Zhou, Jian</creatorcontrib><creatorcontrib>Min, Xinhao</creatorcontrib><creatorcontrib>Ge, Song</creatorcontrib><creatorcontrib>Wan, Jiguang</creatorcontrib><creatorcontrib>Yao, Ting</creatorcontrib><creatorcontrib>Wang, Daohui</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on parallel and distributed systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Zhang, Yiwen</au><au>Zhou, Jian</au><au>Min, Xinhao</au><au>Ge, Song</au><au>Wan, Jiguang</au><au>Yao, Ting</au><au>Wang, Daohui</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory</atitle><jtitle>IEEE transactions on parallel and distributed systems</jtitle><stitle>TPDS</stitle><date>2023-03-01</date><risdate>2023</risdate><volume>34</volume><issue>3</issue><spage>843</spage><epage>855</epage><pages>843-855</pages><issn>1045-9219</issn><eissn>1558-2183</eissn><coden>ITDSEO</coden><abstract><![CDATA[Previous works proposed building file systems and organizing the metadata with KV stores because KV stores handle entries of various sizes efficiently and have excellent scalability. The emergence of the byte-addressable persistent memory (PM) enables metadata service to be faster than before by tailoring the KV store for the PM. However, existing PM-based KV stores cannot handle the workloads of file systems' metadata well because simply depending on hash tables or trees cannot simultaneously provide fast file accessing and efficient directory traversing. In this paper, we exploit the insight of the metadata operations and propose the PetaKV, a KV store tailored for the metadata management of file systems on PM. PetaKV leverages dual hash indexing to achieve fast file put and get operations. Moreover, it cooperates with PM-tailored peta logs to collocate KV entries for each directory, thus supporting efficient directory scans. Our evaluation indicates PetaKV outperforms state-of-art tree-based KV stores on put, get and scan <inline-formula><tex-math notation="LaTeX">2.5\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>5</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq1-3232382.gif"/> </inline-formula>, <inline-formula><tex-math notation="LaTeX">3.2\times</tex-math> <mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq2-3232382.gif"/> </inline-formula>, and <inline-formula><tex-math notation="LaTeX">2.8\times</tex-math> <mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mo>.</mml:mo><mml:mn>8</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq3-3232382.gif"/> </inline-formula> on average, respectively. Moreover, the file system built with PetaKV achieves <inline-formula><tex-math notation="LaTeX">1.2\times</tex-math> <mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq4-3232382.gif"/> </inline-formula> to <inline-formula><tex-math notation="LaTeX">6.4\times</tex-math> <mml:math><mml:mrow><mml:mn>6</mml:mn><mml:mo>.</mml:mo><mml:mn>4</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="zhou-ieq5-3232382.gif"/> </inline-formula> speedup compared to those built with tree-based KV stores on the metadata operations.]]></abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/TPDS.2022.3232382</doi><tpages>13</tpages><orcidid>https://orcid.org/0000-0002-9358-9373</orcidid><orcidid>https://orcid.org/0000-0001-5279-4816</orcidid><orcidid>https://orcid.org/0000-0001-5295-4680</orcidid><orcidid>https://orcid.org/0000-0002-4160-9475</orcidid><orcidid>https://orcid.org/0000-0001-6216-1537</orcidid></addata></record> |
fulltext | fulltext_linktorsrc |
identifier | ISSN: 1045-9219 |
ispartof | IEEE transactions on parallel and distributed systems, 2023-03, Vol.34 (3), p.843-855 |
issn | 1045-9219 1558-2183 |
language | eng |
recordid | cdi_ieee_primary_9999527 |
source | IEEE Electronic Library (IEL) |
subjects | Buildings Complexity theory Data management file system metadata File systems hash index Indexing Key-Value Store log-structure Metadata persistent memory Stores Three-dimensional displays Throughput |
title | PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-19T02%3A07%3A48IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=PetaKV:%20Building%20Efficient%20Key-Value%20Store%20for%20File%20System%20Metadata%20on%20Persistent%20Memory&rft.jtitle=IEEE%20transactions%20on%20parallel%20and%20distributed%20systems&rft.au=Zhang,%20Yiwen&rft.date=2023-03-01&rft.volume=34&rft.issue=3&rft.spage=843&rft.epage=855&rft.pages=843-855&rft.issn=1045-9219&rft.eissn=1558-2183&rft.coden=ITDSEO&rft_id=info:doi/10.1109/TPDS.2022.3232382&rft_dat=%3Cproquest_RIE%3E2767316561%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2767316561&rft_id=info:pmid/&rft_ieee_id=9999527&rfr_iscdi=true |