Hybrid hierarchy storage system in MilkyWay-2 supercomputer
With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabyt...
Gespeichert in:
| Veröffentlicht in: | Frontiers of Computer Science 2014-06, Vol.8 (3), p.367-377 |
|---|---|
| Hauptverfasser: | , , , , , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 377 |
|---|---|
| container_issue | 3 |
| container_start_page | 367 |
| container_title | Frontiers of Computer Science |
| container_volume | 8 |
| creator | XU, Weixia LU, Yutong LI, Qiong ZHOU, Enqiang SONG, Zhenlong DONG, Yong ZHANG, Wei WEI, Dengping ZHANG, Xiaoming CHEN, Haitao XING, Jianying YUAN, Yuan |
| description | With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability. MilkyWay-2 （TH-2） supercomputer system with peak performance 54.9 Props, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H^2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability. |
| doi_str_mv | 10.1007/s11704-014-3499-6 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2918718149</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><cqvip_id>50099665</cqvip_id><sourcerecordid>2918718149</sourcerecordid><originalsourceid>FETCH-LOGICAL-c391t-449843eafaf075fe1e51990550a749a4f0302dbb54f59c748b9dd5e8f7fb3fec3</originalsourceid><addsrcrecordid>eNp9kE1Pg0AQhjdGE5vaH-AN4xmd_QI2nkyj1qTGi8bjZoFZoLZAd-HAv5eGpt56mjm8zzuTh5BbCg8UIH70lMYgQqAi5EKpMLogMwZKhozx6PK0s-SaLLzfAAADJiVjM_K0GlJX5UFZoTMuK4fAd40zBQZ-8B3ugqoOPqrt7_BjhpAFvm_RZc2u7Tt0N-TKmq3HxXHOyffry9dyFa4_396Xz-sw44p2oRAqERyNNRZiaZGipEqBlGBioYywwIHlaSqFlSqLRZKqPJeY2Nim3GLG5-R-6m1ds-_Rd3rT9K4eT2qmaBLThAo1puiUylzjvUOrW1ftjBs0BX3QpCdNetSkD5p0NDJsYvyYrQt0_83noGSCyqoo0WHeOvReW9fU3WjxPHp3_LFs6mI_njw9KQGUiiLJ_wB2d4cx</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2918718149</pqid></control><display><type>article</type><title>Hybrid hierarchy storage system in MilkyWay-2 supercomputer</title><source>ProQuest Central UK/Ireland</source><source>SpringerLink Journals - AutoHoldings</source><source>ProQuest Central</source><creator>XU, Weixia ; LU, Yutong ; LI, Qiong ; ZHOU, Enqiang ; SONG, Zhenlong ; DONG, Yong ; ZHANG, Wei ; WEI, Dengping ; ZHANG, Xiaoming ; CHEN, Haitao ; XING, Jianying ; YUAN, Yuan</creator><creatorcontrib>XU, Weixia ; LU, Yutong ; LI, Qiong ; ZHOU, Enqiang ; SONG, Zhenlong ; DONG, Yong ; ZHANG, Wei ; WEI, Dengping ; ZHANG, Xiaoming ; CHEN, Haitao ; XING, Jianying ; YUAN, Yuan</creatorcontrib><description>With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability. MilkyWay-2 （TH-2） supercomputer system with peak performance 54.9 Props, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H^2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability.</description><identifier>ISSN: 2095-2228</identifier><identifier>EISSN: 2095-2236</identifier><identifier>DOI: 10.1007/s11704-014-3499-6</identifier><language>eng</language><publisher>Heidelberg: Higher Education Press</publisher><subject>Computation ; Computer architecture ; Computer Science ; file system ; hierarchy ; hybrid ; Hybrid systems ; MilkyWay-2 ; Research Article ; Storage capacity ; storage system ; Storage systems ; Subsystems ; supercomputer ; Supercomputers ; 可扩展性 ; 存储子系统 ; 存储架构 ; 存储系统 ; 并行文件系统 ; 混合型 ; 硬件体系结构 ; 超级计算机系统</subject><ispartof>Frontiers of Computer Science, 2014-06, Vol.8 (3), p.367-377</ispartof><rights>Copyright reserved, 2014, Higher Education Press and Springer-Verlag Berlin Heidelberg</rights><rights>Higher Education Press and Springer-Verlag Berlin Heidelberg 2014</rights><rights>Higher Education Press and Springer-Verlag Berlin Heidelberg 2014.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c391t-449843eafaf075fe1e51990550a749a4f0302dbb54f59c748b9dd5e8f7fb3fec3</citedby><cites>FETCH-LOGICAL-c391t-449843eafaf075fe1e51990550a749a4f0302dbb54f59c748b9dd5e8f7fb3fec3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Uhttp://image.cqvip.com/vip1000/qk/71018X/71018X.jpg</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s11704-014-3499-6$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://www.proquest.com/docview/2918718149?pq-origsite=primo$$EHTML$$P50$$Gproquest$$H</linktohtml><link.rule.ids>315,781,785,21390,27926,27927,33746,41490,42559,43807,51321,64387,64391,72471</link.rule.ids></links><search><creatorcontrib>XU, Weixia</creatorcontrib><creatorcontrib>LU, Yutong</creatorcontrib><creatorcontrib>LI, Qiong</creatorcontrib><creatorcontrib>ZHOU, Enqiang</creatorcontrib><creatorcontrib>SONG, Zhenlong</creatorcontrib><creatorcontrib>DONG, Yong</creatorcontrib><creatorcontrib>ZHANG, Wei</creatorcontrib><creatorcontrib>WEI, Dengping</creatorcontrib><creatorcontrib>ZHANG, Xiaoming</creatorcontrib><creatorcontrib>CHEN, Haitao</creatorcontrib><creatorcontrib>XING, Jianying</creatorcontrib><creatorcontrib>YUAN, Yuan</creatorcontrib><title>Hybrid hierarchy storage system in MilkyWay-2 supercomputer</title><title>Frontiers of Computer Science</title><addtitle>Front. Comput. Sci</addtitle><addtitle>Frontiers of Computer Science in China</addtitle><description>With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability. MilkyWay-2 （TH-2） supercomputer system with peak performance 54.9 Props, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H^2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability.</description><subject>Computation</subject><subject>Computer architecture</subject><subject>Computer Science</subject><subject>file system</subject><subject>hierarchy</subject><subject>hybrid</subject><subject>Hybrid systems</subject><subject>MilkyWay-2</subject><subject>Research Article</subject><subject>Storage capacity</subject><subject>storage system</subject><subject>Storage systems</subject><subject>Subsystems</subject><subject>supercomputer</subject><subject>Supercomputers</subject><subject>可扩展性</subject><subject>存储子系统</subject><subject>存储架构</subject><subject>存储系统</subject><subject>并行文件系统</subject><subject>混合型</subject><subject>硬件体系结构</subject><subject>超级计算机系统</subject><issn>2095-2228</issn><issn>2095-2236</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2014</creationdate><recordtype>article</recordtype><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><recordid>eNp9kE1Pg0AQhjdGE5vaH-AN4xmd_QI2nkyj1qTGi8bjZoFZoLZAd-HAv5eGpt56mjm8zzuTh5BbCg8UIH70lMYgQqAi5EKpMLogMwZKhozx6PK0s-SaLLzfAAADJiVjM_K0GlJX5UFZoTMuK4fAd40zBQZ-8B3ugqoOPqrt7_BjhpAFvm_RZc2u7Tt0N-TKmq3HxXHOyffry9dyFa4_396Xz-sw44p2oRAqERyNNRZiaZGipEqBlGBioYywwIHlaSqFlSqLRZKqPJeY2Nim3GLG5-R-6m1ds-_Rd3rT9K4eT2qmaBLThAo1puiUylzjvUOrW1ftjBs0BX3QpCdNetSkD5p0NDJsYvyYrQt0_83noGSCyqoo0WHeOvReW9fU3WjxPHp3_LFs6mI_njw9KQGUiiLJ_wB2d4cx</recordid><startdate>20140601</startdate><enddate>20140601</enddate><creator>XU, Weixia</creator><creator>LU, Yutong</creator><creator>LI, Qiong</creator><creator>ZHOU, Enqiang</creator><creator>SONG, Zhenlong</creator><creator>DONG, Yong</creator><creator>ZHANG, Wei</creator><creator>WEI, Dengping</creator><creator>ZHANG, Xiaoming</creator><creator>CHEN, Haitao</creator><creator>XING, Jianying</creator><creator>YUAN, Yuan</creator><general>Higher Education Press</general><general>Springer Nature B.V</general><scope>2RA</scope><scope>92L</scope><scope>CQIGP</scope><scope>W92</scope><scope>~WA</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>8FE</scope><scope>8FG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K7-</scope><scope>P5Z</scope><scope>P62</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope></search><sort><creationdate>20140601</creationdate><title>Hybrid hierarchy storage system in MilkyWay-2 supercomputer</title><author>XU, Weixia ; LU, Yutong ; LI, Qiong ; ZHOU, Enqiang ; SONG, Zhenlong ; DONG, Yong ; ZHANG, Wei ; WEI, Dengping ; ZHANG, Xiaoming ; CHEN, Haitao ; XING, Jianying ; YUAN, Yuan</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c391t-449843eafaf075fe1e51990550a749a4f0302dbb54f59c748b9dd5e8f7fb3fec3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2014</creationdate><topic>Computation</topic><topic>Computer architecture</topic><topic>Computer Science</topic><topic>file system</topic><topic>hierarchy</topic><topic>hybrid</topic><topic>Hybrid systems</topic><topic>MilkyWay-2</topic><topic>Research Article</topic><topic>Storage capacity</topic><topic>storage system</topic><topic>Storage systems</topic><topic>Subsystems</topic><topic>supercomputer</topic><topic>Supercomputers</topic><topic>可扩展性</topic><topic>存储子系统</topic><topic>存储架构</topic><topic>存储系统</topic><topic>并行文件系统</topic><topic>混合型</topic><topic>硬件体系结构</topic><topic>超级计算机系统</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>XU, Weixia</creatorcontrib><creatorcontrib>LU, Yutong</creatorcontrib><creatorcontrib>LI, Qiong</creatorcontrib><creatorcontrib>ZHOU, Enqiang</creatorcontrib><creatorcontrib>SONG, Zhenlong</creatorcontrib><creatorcontrib>DONG, Yong</creatorcontrib><creatorcontrib>ZHANG, Wei</creatorcontrib><creatorcontrib>WEI, Dengping</creatorcontrib><creatorcontrib>ZHANG, Xiaoming</creatorcontrib><creatorcontrib>CHEN, Haitao</creatorcontrib><creatorcontrib>XING, Jianying</creatorcontrib><creatorcontrib>YUAN, Yuan</creatorcontrib><collection>中文科技期刊数据库</collection><collection>中文科技期刊数据库-CALIS站点</collection><collection>中文科技期刊数据库-7.0平台</collection><collection>中文科技期刊数据库-工程技术</collection><collection>中文科技期刊数据库- 镜像站点</collection><collection>CrossRef</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>Computer Science Database</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><jtitle>Frontiers of Computer Science</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>XU, Weixia</au><au>LU, Yutong</au><au>LI, Qiong</au><au>ZHOU, Enqiang</au><au>SONG, Zhenlong</au><au>DONG, Yong</au><au>ZHANG, Wei</au><au>WEI, Dengping</au><au>ZHANG, Xiaoming</au><au>CHEN, Haitao</au><au>XING, Jianying</au><au>YUAN, Yuan</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Hybrid hierarchy storage system in MilkyWay-2 supercomputer</atitle><jtitle>Frontiers of Computer Science</jtitle><stitle>Front. Comput. Sci</stitle><addtitle>Frontiers of Computer Science in China</addtitle><date>2014-06-01</date><risdate>2014</risdate><volume>8</volume><issue>3</issue><spage>367</spage><epage>377</epage><pages>367-377</pages><issn>2095-2228</issn><eissn>2095-2236</eissn><abstract>With the rapid improvement of computation capability in high performance supercomputer system, the imbalance of performance between computation subsystem and storage subsystem has become more and more serious, especially when various big data are produced ranging from tens of gigabytes up to terabytes. To reduce this gap, large-scale storage systems need to be designed and implemented with high performance and scalability. MilkyWay-2 （TH-2） supercomputer system with peak performance 54.9 Props, definitely has this kind of requirement for storage system. This paper mainly introduces the storage system in MilkyWay-2 supercomputer, including the hardware architecture and the parallel file system. The storage system in MilkyWay-2 supercomputer exploits a novel hybrid hierarchy storage architecture to enable high scalability of I/O clients, I/O bandwidth and storage capacity. To fit this architecture, a user level virtualized file system, named H^2FS, is designed and implemented which can cooperate local storage and shared storage together into a dynamic single namespace to optimize I/O performance in IO-intensive applications. The evaluation results show that the storage system in MilkyWay-2 supercomputer can satisfy the critical requirements in large scale supercomputer, such as performance and scalability.</abstract><cop>Heidelberg</cop><pub>Higher Education Press</pub><doi>10.1007/s11704-014-3499-6</doi><tpages>11</tpages><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2095-2228 |
| ispartof | Frontiers of Computer Science, 2014-06, Vol.8 (3), p.367-377 |
| issn | 2095-2228 2095-2236 |
| language | eng |
| recordid | cdi_proquest_journals_2918718149 |
| source | ProQuest Central UK/Ireland; SpringerLink Journals - AutoHoldings; ProQuest Central |
| subjects | Computation Computer architecture Computer Science file system hierarchy hybrid Hybrid systems MilkyWay-2 Research Article Storage capacity storage system Storage systems Subsystems supercomputer Supercomputers 可扩展性 存储子系统 存储架构 存储系统 并行文件系统 混合型 硬件体系结构 超级计算机系统 |
| title | Hybrid hierarchy storage system in MilkyWay-2 supercomputer |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-18T09%3A40%3A42IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Hybrid%20hierarchy%20storage%20system%20in%20MilkyWay-2%20supercomputer&rft.jtitle=Frontiers%20of%20Computer%20Science&rft.au=XU,%20Weixia&rft.date=2014-06-01&rft.volume=8&rft.issue=3&rft.spage=367&rft.epage=377&rft.pages=367-377&rft.issn=2095-2228&rft.eissn=2095-2236&rft_id=info:doi/10.1007/s11704-014-3499-6&rft_dat=%3Cproquest_cross%3E2918718149%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2918718149&rft_id=info:pmid/&rft_cqvip_id=50099665&rfr_iscdi=true |