Use of Run Time Predictions for Automatic Co-Allocation of Multi-Cluster Resources for Iterative Parallel Applications
Metaschedulers co-allocate resources by requesting a fixed number of processors and usage time for each cluster. These static requests, defined by users, limit the initial scheduling and prevent rescheduling of applications to other resource sets. It is also difficult for users to estimate applicati...
Gespeichert in:
Veröffentlicht in: | Journal of parallel and distributed computing 2011-10, Vol.71 (10), p.1388-1399 |
---|---|
Hauptverfasser: | , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 1399 |
---|---|
container_issue | 10 |
container_start_page | 1388 |
container_title | Journal of parallel and distributed computing |
container_volume | 71 |
creator | Netto, Marco A.S. Vecchiola, Christian Kirley, Michael Varela, Carlos A. Buyya, Rajkumar |
description | Metaschedulers co-allocate resources by requesting a fixed number of processors and usage time for each cluster. These static requests, defined by users, limit the initial scheduling and prevent rescheduling of applications to other resource sets. It is also difficult for users to estimate application execution times, especially on heterogeneous environments. To overcome these problems, metaschedulers can use performance predictions for automaticresource selection. This paper proposes a resourceco-allocation technique with rescheduling support based on performance predictions for multi-clusteriterativeparallelapplications. Iterativeapplications have been used to solve a variety of problems in science and engineering, including large-scale computations based on the asynchronous model more recently. We performed experiments using an iterativeparallelapplication, which consists of benchmark multiobjective problems, with both synchronous and asynchronous communication models on Grid'5000. The results show runtimepredictions with an average error of 7% and prevention of up to 35% and 57% of runtime overestimations to support rescheduling for synchronous and asynchronous models, respectively. The performance predictions require no application source code access. One of the main findings is that as the asynchronous model masks communication and computation, it requires no network information to predict execution times. By using our co-allocation technique, metaschedulers become responsible for runtimepredictions, process mapping, and application rescheduling; releasing the user from these burden tasks. |
doi_str_mv | 10.1016/j.jpdc.2011.05.007 |
format | Article |
fullrecord | <record><control><sourceid>hal</sourceid><recordid>TN_cdi_hal_primary_oai_HAL_hal_00688431v1</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>oai_HAL_hal_00688431v1</sourcerecordid><originalsourceid>FETCH-hal_primary_oai_HAL_hal_00688431v13</originalsourceid><addsrcrecordid>eNqVjMtKxDAYhYMoWEdfwFW2LhL_2FtclqLMgIIMM-sSMimm_J2UXAq-vS3jC7g6cM73HUIeBXABonoe-DCdNH8BITiUHKC-IpmA14qBLOQ1yaAuclbnorwldyEMsIBlLTMyH4Ohrqf7dKYHOxr65c3J6mjdOdDeedqk6EYVraatYw2i02odV-czYbSsxRSi8XRvgktem4u2W6oFnJdD5RWiQdpME9qLHe7JTa8wmIe_3JCn97dDu2XfCrvJ21H5n84p222bj27tACopi1zMIv8P-wsbrVgb</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Use of Run Time Predictions for Automatic Co-Allocation of Multi-Cluster Resources for Iterative Parallel Applications</title><source>Access via ScienceDirect (Elsevier)</source><creator>Netto, Marco A.S. ; Vecchiola, Christian ; Kirley, Michael ; Varela, Carlos A. ; Buyya, Rajkumar</creator><creatorcontrib>Netto, Marco A.S. ; Vecchiola, Christian ; Kirley, Michael ; Varela, Carlos A. ; Buyya, Rajkumar</creatorcontrib><description>Metaschedulers co-allocate resources by requesting a fixed number of processors and usage time for each cluster. These static requests, defined by users, limit the initial scheduling and prevent rescheduling of applications to other resource sets. It is also difficult for users to estimate application execution times, especially on heterogeneous environments. To overcome these problems, metaschedulers can use performance predictions for automaticresource selection. This paper proposes a resourceco-allocation technique with rescheduling support based on performance predictions for multi-clusteriterativeparallelapplications. Iterativeapplications have been used to solve a variety of problems in science and engineering, including large-scale computations based on the asynchronous model more recently. We performed experiments using an iterativeparallelapplication, which consists of benchmark multiobjective problems, with both synchronous and asynchronous communication models on Grid'5000. The results show runtimepredictions with an average error of 7% and prevention of up to 35% and 57% of runtime overestimations to support rescheduling for synchronous and asynchronous models, respectively. The performance predictions require no application source code access. One of the main findings is that as the asynchronous model masks communication and computation, it requires no network information to predict execution times. By using our co-allocation technique, metaschedulers become responsible for runtimepredictions, process mapping, and application rescheduling; releasing the user from these burden tasks.</description><identifier>ISSN: 0743-7315</identifier><identifier>EISSN: 1096-0848</identifier><identifier>DOI: 10.1016/j.jpdc.2011.05.007</identifier><language>eng</language><publisher>Elsevier</publisher><subject>Computer Science ; Other</subject><ispartof>Journal of parallel and distributed computing, 2011-10, Vol.71 (10), p.1388-1399</ispartof><rights>Distributed under a Creative Commons Attribution 4.0 International License</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>230,314,780,784,885,27924,27925</link.rule.ids><backlink>$$Uhttps://inria.hal.science/hal-00688431$$DView record in HAL$$Hfree_for_read</backlink></links><search><creatorcontrib>Netto, Marco A.S.</creatorcontrib><creatorcontrib>Vecchiola, Christian</creatorcontrib><creatorcontrib>Kirley, Michael</creatorcontrib><creatorcontrib>Varela, Carlos A.</creatorcontrib><creatorcontrib>Buyya, Rajkumar</creatorcontrib><title>Use of Run Time Predictions for Automatic Co-Allocation of Multi-Cluster Resources for Iterative Parallel Applications</title><title>Journal of parallel and distributed computing</title><description>Metaschedulers co-allocate resources by requesting a fixed number of processors and usage time for each cluster. These static requests, defined by users, limit the initial scheduling and prevent rescheduling of applications to other resource sets. It is also difficult for users to estimate application execution times, especially on heterogeneous environments. To overcome these problems, metaschedulers can use performance predictions for automaticresource selection. This paper proposes a resourceco-allocation technique with rescheduling support based on performance predictions for multi-clusteriterativeparallelapplications. Iterativeapplications have been used to solve a variety of problems in science and engineering, including large-scale computations based on the asynchronous model more recently. We performed experiments using an iterativeparallelapplication, which consists of benchmark multiobjective problems, with both synchronous and asynchronous communication models on Grid'5000. The results show runtimepredictions with an average error of 7% and prevention of up to 35% and 57% of runtime overestimations to support rescheduling for synchronous and asynchronous models, respectively. The performance predictions require no application source code access. One of the main findings is that as the asynchronous model masks communication and computation, it requires no network information to predict execution times. By using our co-allocation technique, metaschedulers become responsible for runtimepredictions, process mapping, and application rescheduling; releasing the user from these burden tasks.</description><subject>Computer Science</subject><subject>Other</subject><issn>0743-7315</issn><issn>1096-0848</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2011</creationdate><recordtype>article</recordtype><recordid>eNqVjMtKxDAYhYMoWEdfwFW2LhL_2FtclqLMgIIMM-sSMimm_J2UXAq-vS3jC7g6cM73HUIeBXABonoe-DCdNH8BITiUHKC-IpmA14qBLOQ1yaAuclbnorwldyEMsIBlLTMyH4Ohrqf7dKYHOxr65c3J6mjdOdDeedqk6EYVraatYw2i02odV-czYbSsxRSi8XRvgktem4u2W6oFnJdD5RWiQdpME9qLHe7JTa8wmIe_3JCn97dDu2XfCrvJ21H5n84p222bj27tACopi1zMIv8P-wsbrVgb</recordid><startdate>201110</startdate><enddate>201110</enddate><creator>Netto, Marco A.S.</creator><creator>Vecchiola, Christian</creator><creator>Kirley, Michael</creator><creator>Varela, Carlos A.</creator><creator>Buyya, Rajkumar</creator><general>Elsevier</general><scope>1XC</scope></search><sort><creationdate>201110</creationdate><title>Use of Run Time Predictions for Automatic Co-Allocation of Multi-Cluster Resources for Iterative Parallel Applications</title><author>Netto, Marco A.S. ; Vecchiola, Christian ; Kirley, Michael ; Varela, Carlos A. ; Buyya, Rajkumar</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-hal_primary_oai_HAL_hal_00688431v13</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2011</creationdate><topic>Computer Science</topic><topic>Other</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Netto, Marco A.S.</creatorcontrib><creatorcontrib>Vecchiola, Christian</creatorcontrib><creatorcontrib>Kirley, Michael</creatorcontrib><creatorcontrib>Varela, Carlos A.</creatorcontrib><creatorcontrib>Buyya, Rajkumar</creatorcontrib><collection>Hyper Article en Ligne (HAL)</collection><jtitle>Journal of parallel and distributed computing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Netto, Marco A.S.</au><au>Vecchiola, Christian</au><au>Kirley, Michael</au><au>Varela, Carlos A.</au><au>Buyya, Rajkumar</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Use of Run Time Predictions for Automatic Co-Allocation of Multi-Cluster Resources for Iterative Parallel Applications</atitle><jtitle>Journal of parallel and distributed computing</jtitle><date>2011-10</date><risdate>2011</risdate><volume>71</volume><issue>10</issue><spage>1388</spage><epage>1399</epage><pages>1388-1399</pages><issn>0743-7315</issn><eissn>1096-0848</eissn><abstract>Metaschedulers co-allocate resources by requesting a fixed number of processors and usage time for each cluster. These static requests, defined by users, limit the initial scheduling and prevent rescheduling of applications to other resource sets. It is also difficult for users to estimate application execution times, especially on heterogeneous environments. To overcome these problems, metaschedulers can use performance predictions for automaticresource selection. This paper proposes a resourceco-allocation technique with rescheduling support based on performance predictions for multi-clusteriterativeparallelapplications. Iterativeapplications have been used to solve a variety of problems in science and engineering, including large-scale computations based on the asynchronous model more recently. We performed experiments using an iterativeparallelapplication, which consists of benchmark multiobjective problems, with both synchronous and asynchronous communication models on Grid'5000. The results show runtimepredictions with an average error of 7% and prevention of up to 35% and 57% of runtime overestimations to support rescheduling for synchronous and asynchronous models, respectively. The performance predictions require no application source code access. One of the main findings is that as the asynchronous model masks communication and computation, it requires no network information to predict execution times. By using our co-allocation technique, metaschedulers become responsible for runtimepredictions, process mapping, and application rescheduling; releasing the user from these burden tasks.</abstract><pub>Elsevier</pub><doi>10.1016/j.jpdc.2011.05.007</doi></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0743-7315 |
ispartof | Journal of parallel and distributed computing, 2011-10, Vol.71 (10), p.1388-1399 |
issn | 0743-7315 1096-0848 |
language | eng |
recordid | cdi_hal_primary_oai_HAL_hal_00688431v1 |
source | Access via ScienceDirect (Elsevier) |
subjects | Computer Science Other |
title | Use of Run Time Predictions for Automatic Co-Allocation of Multi-Cluster Resources for Iterative Parallel Applications |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T10%3A38%3A05IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-hal&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Use%20of%20Run%20Time%20Predictions%20for%20Automatic%20Co-Allocation%20of%20Multi-Cluster%20Resources%20for%20Iterative%20Parallel%20Applications&rft.jtitle=Journal%20of%20parallel%20and%20distributed%20computing&rft.au=Netto,%20Marco%20A.S.&rft.date=2011-10&rft.volume=71&rft.issue=10&rft.spage=1388&rft.epage=1399&rft.pages=1388-1399&rft.issn=0743-7315&rft.eissn=1096-0848&rft_id=info:doi/10.1016/j.jpdc.2011.05.007&rft_dat=%3Chal%3Eoai_HAL_hal_00688431v1%3C/hal%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |