Data-locality-aware mapreduce real-time scheduling framework
•A framework to manage interactive MapReduce applications with deadline constraint.•A dispatcher to assign jobs to resources considering blocking and data-locality.•A dynamic power management for MapReduce tasks to improve run-time energy efficiency.•A schedulability test to ensure that all MapReduc...
Gespeichert in:
Veröffentlicht in: | The Journal of systems and software 2016-02, Vol.112, p.65-77 |
---|---|
Hauptverfasser: | , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 77 |
---|---|
container_issue | |
container_start_page | 65 |
container_title | The Journal of systems and software |
container_volume | 112 |
creator | Kao, Yu-Chon Chen, Ya-Shu |
description | •A framework to manage interactive MapReduce applications with deadline constraint.•A dispatcher to assign jobs to resources considering blocking and data-locality.•A dynamic power management for MapReduce tasks to improve run-time energy efficiency.•A schedulability test to ensure that all MapReduce tasks meet the timing constraints.
MapReduce is widely used in cloud applications for large-scale data processing. The increasing number of interactive cloud applications has led to an increasing need for MapReduce real-time scheduling. Most MapReduce applications are data-oriented and nonpreemptively executed. Therefore, the problem of MapReduce real-time scheduling is complicated because of the trade-off between run-time blocking for nonpreemptive execution and data-locality. This paper proposes a data-locality-aware MapReduce real-time scheduling framework for guaranteeing quality of service for interactive MapReduce applications. A scheduler and dispatcher that can be used for scheduling two-phase MapReduce jobs and for assigning jobs to computing resources are presented, and the dispatcher enable the consideration of blocking and data-locality. Furthermore, dynamic power management for run-time energy saving is discussed. Finally, the proposed methodology is evaluated by considering synthetic workloads, and a comparative study of different scheduling algorithms is conducted. |
doi_str_mv | 10.1016/j.jss.2015.11.001 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_1747608251</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0164121215002344</els_id><sourcerecordid>3891616671</sourcerecordid><originalsourceid>FETCH-LOGICAL-c325t-201644b1804c88434bc13adc3dad913a86d926c2053750a2f4319bb67f820ea43</originalsourceid><addsrcrecordid>eNp9kEtPwzAQhC0EEqXwA7hF4pywazuJK7ig8pQqcYGz5TgbcMij2ClV_z2uypnTjlYzO6uPsUuEDAGL6zZrQ8g4YJ4hZgB4xGaoSpEi5-qYzaJHRo38lJ2F0AJAyYHP2O29mUzajdZ0btqlZms8Jb1Ze6o3lhJPpksn11MS7GdcdW74SBpvetqO_uucnTSmC3TxN-fs_fHhbfmcrl6fXpZ3q9QKnk8p35fLChVIq5QUsrIoTG1FbepFVKqoF7ywHHJR5mB4IwUuqqooG8WBjBRzdnW4u_bj94bCpNtx44dYqbGUZQGK5xhdeHBZP4bgqdFr73rjdxpB7yHpVkdIeg9JI-oIKWZuDhmK7_848jpYR4Ol2nmyk65H90_6F0XlbeA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1747608251</pqid></control><display><type>article</type><title>Data-locality-aware mapreduce real-time scheduling framework</title><source>Elsevier ScienceDirect Journals Complete</source><creator>Kao, Yu-Chon ; Chen, Ya-Shu</creator><creatorcontrib>Kao, Yu-Chon ; Chen, Ya-Shu</creatorcontrib><description>•A framework to manage interactive MapReduce applications with deadline constraint.•A dispatcher to assign jobs to resources considering blocking and data-locality.•A dynamic power management for MapReduce tasks to improve run-time energy efficiency.•A schedulability test to ensure that all MapReduce tasks meet the timing constraints.
MapReduce is widely used in cloud applications for large-scale data processing. The increasing number of interactive cloud applications has led to an increasing need for MapReduce real-time scheduling. Most MapReduce applications are data-oriented and nonpreemptively executed. Therefore, the problem of MapReduce real-time scheduling is complicated because of the trade-off between run-time blocking for nonpreemptive execution and data-locality. This paper proposes a data-locality-aware MapReduce real-time scheduling framework for guaranteeing quality of service for interactive MapReduce applications. A scheduler and dispatcher that can be used for scheduling two-phase MapReduce jobs and for assigning jobs to computing resources are presented, and the dispatcher enable the consideration of blocking and data-locality. Furthermore, dynamic power management for run-time energy saving is discussed. Finally, the proposed methodology is evaluated by considering synthetic workloads, and a comparative study of different scheduling algorithms is conducted.</description><identifier>ISSN: 0164-1212</identifier><identifier>EISSN: 1873-1228</identifier><identifier>DOI: 10.1016/j.jss.2015.11.001</identifier><identifier>CODEN: JSSODM</identifier><language>eng</language><publisher>New York: Elsevier Inc</publisher><subject>Cloud computing ; Cloud computing systems ; Comparative studies ; Data processing ; Data-locality ; Energy efficiency ; Quality of service ; Real time ; Real-time scheduling ; Scheduling ; Scheduling algorithms</subject><ispartof>The Journal of systems and software, 2016-02, Vol.112, p.65-77</ispartof><rights>2015 Elsevier Inc.</rights><rights>Copyright Elsevier Sequoia S.A. Feb 2016</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c325t-201644b1804c88434bc13adc3dad913a86d926c2053750a2f4319bb67f820ea43</citedby><cites>FETCH-LOGICAL-c325t-201644b1804c88434bc13adc3dad913a86d926c2053750a2f4319bb67f820ea43</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/j.jss.2015.11.001$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>314,780,784,3550,27924,27925,45995</link.rule.ids></links><search><creatorcontrib>Kao, Yu-Chon</creatorcontrib><creatorcontrib>Chen, Ya-Shu</creatorcontrib><title>Data-locality-aware mapreduce real-time scheduling framework</title><title>The Journal of systems and software</title><description>•A framework to manage interactive MapReduce applications with deadline constraint.•A dispatcher to assign jobs to resources considering blocking and data-locality.•A dynamic power management for MapReduce tasks to improve run-time energy efficiency.•A schedulability test to ensure that all MapReduce tasks meet the timing constraints.
MapReduce is widely used in cloud applications for large-scale data processing. The increasing number of interactive cloud applications has led to an increasing need for MapReduce real-time scheduling. Most MapReduce applications are data-oriented and nonpreemptively executed. Therefore, the problem of MapReduce real-time scheduling is complicated because of the trade-off between run-time blocking for nonpreemptive execution and data-locality. This paper proposes a data-locality-aware MapReduce real-time scheduling framework for guaranteeing quality of service for interactive MapReduce applications. A scheduler and dispatcher that can be used for scheduling two-phase MapReduce jobs and for assigning jobs to computing resources are presented, and the dispatcher enable the consideration of blocking and data-locality. Furthermore, dynamic power management for run-time energy saving is discussed. Finally, the proposed methodology is evaluated by considering synthetic workloads, and a comparative study of different scheduling algorithms is conducted.</description><subject>Cloud computing</subject><subject>Cloud computing systems</subject><subject>Comparative studies</subject><subject>Data processing</subject><subject>Data-locality</subject><subject>Energy efficiency</subject><subject>Quality of service</subject><subject>Real time</subject><subject>Real-time scheduling</subject><subject>Scheduling</subject><subject>Scheduling algorithms</subject><issn>0164-1212</issn><issn>1873-1228</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2016</creationdate><recordtype>article</recordtype><recordid>eNp9kEtPwzAQhC0EEqXwA7hF4pywazuJK7ig8pQqcYGz5TgbcMij2ClV_z2uypnTjlYzO6uPsUuEDAGL6zZrQ8g4YJ4hZgB4xGaoSpEi5-qYzaJHRo38lJ2F0AJAyYHP2O29mUzajdZ0btqlZms8Jb1Ze6o3lhJPpksn11MS7GdcdW74SBpvetqO_uucnTSmC3TxN-fs_fHhbfmcrl6fXpZ3q9QKnk8p35fLChVIq5QUsrIoTG1FbepFVKqoF7ywHHJR5mB4IwUuqqooG8WBjBRzdnW4u_bj94bCpNtx44dYqbGUZQGK5xhdeHBZP4bgqdFr73rjdxpB7yHpVkdIeg9JI-oIKWZuDhmK7_848jpYR4Ol2nmyk65H90_6F0XlbeA</recordid><startdate>201602</startdate><enddate>201602</enddate><creator>Kao, Yu-Chon</creator><creator>Chen, Ya-Shu</creator><general>Elsevier Inc</general><general>Elsevier Sequoia S.A</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope></search><sort><creationdate>201602</creationdate><title>Data-locality-aware mapreduce real-time scheduling framework</title><author>Kao, Yu-Chon ; Chen, Ya-Shu</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c325t-201644b1804c88434bc13adc3dad913a86d926c2053750a2f4319bb67f820ea43</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2016</creationdate><topic>Cloud computing</topic><topic>Cloud computing systems</topic><topic>Comparative studies</topic><topic>Data processing</topic><topic>Data-locality</topic><topic>Energy efficiency</topic><topic>Quality of service</topic><topic>Real time</topic><topic>Real-time scheduling</topic><topic>Scheduling</topic><topic>Scheduling algorithms</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Kao, Yu-Chon</creatorcontrib><creatorcontrib>Chen, Ya-Shu</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>The Journal of systems and software</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Kao, Yu-Chon</au><au>Chen, Ya-Shu</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Data-locality-aware mapreduce real-time scheduling framework</atitle><jtitle>The Journal of systems and software</jtitle><date>2016-02</date><risdate>2016</risdate><volume>112</volume><spage>65</spage><epage>77</epage><pages>65-77</pages><issn>0164-1212</issn><eissn>1873-1228</eissn><coden>JSSODM</coden><abstract>•A framework to manage interactive MapReduce applications with deadline constraint.•A dispatcher to assign jobs to resources considering blocking and data-locality.•A dynamic power management for MapReduce tasks to improve run-time energy efficiency.•A schedulability test to ensure that all MapReduce tasks meet the timing constraints.
MapReduce is widely used in cloud applications for large-scale data processing. The increasing number of interactive cloud applications has led to an increasing need for MapReduce real-time scheduling. Most MapReduce applications are data-oriented and nonpreemptively executed. Therefore, the problem of MapReduce real-time scheduling is complicated because of the trade-off between run-time blocking for nonpreemptive execution and data-locality. This paper proposes a data-locality-aware MapReduce real-time scheduling framework for guaranteeing quality of service for interactive MapReduce applications. A scheduler and dispatcher that can be used for scheduling two-phase MapReduce jobs and for assigning jobs to computing resources are presented, and the dispatcher enable the consideration of blocking and data-locality. Furthermore, dynamic power management for run-time energy saving is discussed. Finally, the proposed methodology is evaluated by considering synthetic workloads, and a comparative study of different scheduling algorithms is conducted.</abstract><cop>New York</cop><pub>Elsevier Inc</pub><doi>10.1016/j.jss.2015.11.001</doi><tpages>13</tpages></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0164-1212 |
ispartof | The Journal of systems and software, 2016-02, Vol.112, p.65-77 |
issn | 0164-1212 1873-1228 |
language | eng |
recordid | cdi_proquest_journals_1747608251 |
source | Elsevier ScienceDirect Journals Complete |
subjects | Cloud computing Cloud computing systems Comparative studies Data processing Data-locality Energy efficiency Quality of service Real time Real-time scheduling Scheduling Scheduling algorithms |
title | Data-locality-aware mapreduce real-time scheduling framework |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-01T07%3A40%3A33IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Data-locality-aware%20mapreduce%20real-time%20scheduling%20framework&rft.jtitle=The%20Journal%20of%20systems%20and%20software&rft.au=Kao,%20Yu-Chon&rft.date=2016-02&rft.volume=112&rft.spage=65&rft.epage=77&rft.pages=65-77&rft.issn=0164-1212&rft.eissn=1873-1228&rft.coden=JSSODM&rft_id=info:doi/10.1016/j.jss.2015.11.001&rft_dat=%3Cproquest_cross%3E3891616671%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1747608251&rft_id=info:pmid/&rft_els_id=S0164121215002344&rfr_iscdi=true |