Optimizing Hadoop Performance for Big Data Analytics in Smart Grid
The rapid deployment of Phasor Measurement Units (PMUs) in power systems globally is leading to Big Data challenges. New high performance computing techniques are now required to process an ever increasing volume of data from PMUs. To that extent the Hadoop framework, an open source implementation o...
Gespeichert in:
| Veröffentlicht in: | Mathematical problems in engineering 2017-01, Vol.2017 (2017), p.1-11 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 11 |
|---|---|
| container_issue | 2017 |
| container_start_page | 1 |
| container_title | Mathematical problems in engineering |
| container_volume | 2017 |
| creator | Ashton, Phillip M. Taylor, Gareth A. Li, Maozhen Huang, Zhengwen Khan, Mukhtaj Khan, Mushtaq |
| description | The rapid deployment of Phasor Measurement Units (PMUs) in power systems globally is leading to Big Data challenges. New high performance computing techniques are now required to process an ever increasing volume of data from PMUs. To that extent the Hadoop framework, an open source implementation of the MapReduce computing model, is gaining momentum for Big Data analytics in smart grid applications. However, Hadoop has over 190 configuration parameters, which can have a significant impact on the performance of the Hadoop framework. This paper presents an Enhanced Parallel Detrended Fluctuation Analysis (EPDFA) algorithm for scalable analytics on massive volumes of PMU data. The novel EPDFA algorithm builds on an enhanced Hadoop platform whose configuration parameters are optimized by Gene Expression Programming. Experimental results show that the EPDFA is 29 times faster than the sequential DFA in processing PMU data and 1.87 times faster than a parallel DFA, which utilizes the default Hadoop configuration settings. |
| doi_str_mv | 10.1155/2017/2198262 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_1973512795</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1973512795</sourcerecordid><originalsourceid>FETCH-LOGICAL-c360t-841b7efb52add73e03bc30056638e2d85e500e085b2491544d781ae09fcebcc23</originalsourceid><addsrcrecordid>eNqF0MFLwzAUBvAgCs7pzbMEPGpdXtI06XGbOoXBBBW8hTRNZ8bW1qRD5l9vRgcePb3v8OPx8SF0CeQOgPMRJSBGFHJJM3qEBsAzlnBIxXHMhKYJUPZxis5CWBFCgYMcoMmi7dzG_bh6iZ902TQtfrG-avxG18biGPDELfG97jQe13q965wJ2NX4daN9h2felefopNLrYC8Od4jeHx_epk_JfDF7no7niWEZ6RKZQiFsVXCqy1IwS1hhGCE8y5i0tJTcckIskbygaQ48TUshQVuSV8YWxlA2RNf939Y3X1sbOrVqtj52CgpywThQkfOobntlfBOCt5VqvYtddwqI2q-k9iupw0qR3_T809Wl_nb_6ate22hspf80yFzklP0C0a1u9w</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1973512795</pqid></control><display><type>article</type><title>Optimizing Hadoop Performance for Big Data Analytics in Smart Grid</title><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><source>Wiley-Blackwell Open Access Titles</source><source>Alma/SFX Local Collection</source><creator>Ashton, Phillip M. ; Taylor, Gareth A. ; Li, Maozhen ; Huang, Zhengwen ; Khan, Mukhtaj ; Khan, Mushtaq</creator><contributor>Liatsis, Panos</contributor><creatorcontrib>Ashton, Phillip M. ; Taylor, Gareth A. ; Li, Maozhen ; Huang, Zhengwen ; Khan, Mukhtaj ; Khan, Mushtaq ; Liatsis, Panos</creatorcontrib><description>The rapid deployment of Phasor Measurement Units (PMUs) in power systems globally is leading to Big Data challenges. New high performance computing techniques are now required to process an ever increasing volume of data from PMUs. To that extent the Hadoop framework, an open source implementation of the MapReduce computing model, is gaining momentum for Big Data analytics in smart grid applications. However, Hadoop has over 190 configuration parameters, which can have a significant impact on the performance of the Hadoop framework. This paper presents an Enhanced Parallel Detrended Fluctuation Analysis (EPDFA) algorithm for scalable analytics on massive volumes of PMU data. The novel EPDFA algorithm builds on an enhanced Hadoop platform whose configuration parameters are optimized by Gene Expression Programming. Experimental results show that the EPDFA is 29 times faster than the sequential DFA in processing PMU data and 1.87 times faster than a parallel DFA, which utilizes the default Hadoop configuration settings.</description><identifier>ISSN: 1024-123X</identifier><identifier>EISSN: 1563-5147</identifier><identifier>DOI: 10.1155/2017/2198262</identifier><language>eng</language><publisher>Cairo, Egypt: Hindawi Publishing Corporation</publisher><subject>Algorithms ; Analytics ; Big Data ; Computation ; Computer science ; Configurations ; Data analysis ; Data management ; Datasets ; Electricity distribution ; Employment ; Fault tolerance ; Gene expression ; Smart grid ; Variations</subject><ispartof>Mathematical problems in engineering, 2017-01, Vol.2017 (2017), p.1-11</ispartof><rights>Copyright © 2017 Mukhtaj Khan et al.</rights><rights>Copyright © 2017 Mukhtaj Khan et al.; This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c360t-841b7efb52add73e03bc30056638e2d85e500e085b2491544d781ae09fcebcc23</citedby><cites>FETCH-LOGICAL-c360t-841b7efb52add73e03bc30056638e2d85e500e085b2491544d781ae09fcebcc23</cites><orcidid>0000-0002-4933-6192</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids></links><search><contributor>Liatsis, Panos</contributor><creatorcontrib>Ashton, Phillip M.</creatorcontrib><creatorcontrib>Taylor, Gareth A.</creatorcontrib><creatorcontrib>Li, Maozhen</creatorcontrib><creatorcontrib>Huang, Zhengwen</creatorcontrib><creatorcontrib>Khan, Mukhtaj</creatorcontrib><creatorcontrib>Khan, Mushtaq</creatorcontrib><title>Optimizing Hadoop Performance for Big Data Analytics in Smart Grid</title><title>Mathematical problems in engineering</title><description>The rapid deployment of Phasor Measurement Units (PMUs) in power systems globally is leading to Big Data challenges. New high performance computing techniques are now required to process an ever increasing volume of data from PMUs. To that extent the Hadoop framework, an open source implementation of the MapReduce computing model, is gaining momentum for Big Data analytics in smart grid applications. However, Hadoop has over 190 configuration parameters, which can have a significant impact on the performance of the Hadoop framework. This paper presents an Enhanced Parallel Detrended Fluctuation Analysis (EPDFA) algorithm for scalable analytics on massive volumes of PMU data. The novel EPDFA algorithm builds on an enhanced Hadoop platform whose configuration parameters are optimized by Gene Expression Programming. Experimental results show that the EPDFA is 29 times faster than the sequential DFA in processing PMU data and 1.87 times faster than a parallel DFA, which utilizes the default Hadoop configuration settings.</description><subject>Algorithms</subject><subject>Analytics</subject><subject>Big Data</subject><subject>Computation</subject><subject>Computer science</subject><subject>Configurations</subject><subject>Data analysis</subject><subject>Data management</subject><subject>Datasets</subject><subject>Electricity distribution</subject><subject>Employment</subject><subject>Fault tolerance</subject><subject>Gene expression</subject><subject>Smart grid</subject><subject>Variations</subject><issn>1024-123X</issn><issn>1563-5147</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>RHX</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><recordid>eNqF0MFLwzAUBvAgCs7pzbMEPGpdXtI06XGbOoXBBBW8hTRNZ8bW1qRD5l9vRgcePb3v8OPx8SF0CeQOgPMRJSBGFHJJM3qEBsAzlnBIxXHMhKYJUPZxis5CWBFCgYMcoMmi7dzG_bh6iZ902TQtfrG-avxG18biGPDELfG97jQe13q965wJ2NX4daN9h2felefopNLrYC8Od4jeHx_epk_JfDF7no7niWEZ6RKZQiFsVXCqy1IwS1hhGCE8y5i0tJTcckIskbygaQ48TUshQVuSV8YWxlA2RNf939Y3X1sbOrVqtj52CgpywThQkfOobntlfBOCt5VqvYtddwqI2q-k9iupw0qR3_T809Wl_nb_6ate22hspf80yFzklP0C0a1u9w</recordid><startdate>20170101</startdate><enddate>20170101</enddate><creator>Ashton, Phillip M.</creator><creator>Taylor, Gareth A.</creator><creator>Li, Maozhen</creator><creator>Huang, Zhengwen</creator><creator>Khan, Mukhtaj</creator><creator>Khan, Mushtaq</creator><general>Hindawi Publishing Corporation</general><general>Hindawi</general><general>Hindawi Limited</general><scope>ADJCN</scope><scope>AHFXO</scope><scope>RHU</scope><scope>RHW</scope><scope>RHX</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7TB</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>CWDGH</scope><scope>DWQXO</scope><scope>FR3</scope><scope>GNUQQ</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K7-</scope><scope>KR7</scope><scope>L6V</scope><scope>M7S</scope><scope>P5Z</scope><scope>P62</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope><orcidid>https://orcid.org/0000-0002-4933-6192</orcidid></search><sort><creationdate>20170101</creationdate><title>Optimizing Hadoop Performance for Big Data Analytics in Smart Grid</title><author>Ashton, Phillip M. ; Taylor, Gareth A. ; Li, Maozhen ; Huang, Zhengwen ; Khan, Mukhtaj ; Khan, Mushtaq</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c360t-841b7efb52add73e03bc30056638e2d85e500e085b2491544d781ae09fcebcc23</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Algorithms</topic><topic>Analytics</topic><topic>Big Data</topic><topic>Computation</topic><topic>Computer science</topic><topic>Configurations</topic><topic>Data analysis</topic><topic>Data management</topic><topic>Datasets</topic><topic>Electricity distribution</topic><topic>Employment</topic><topic>Fault tolerance</topic><topic>Gene expression</topic><topic>Smart grid</topic><topic>Variations</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Ashton, Phillip M.</creatorcontrib><creatorcontrib>Taylor, Gareth A.</creatorcontrib><creatorcontrib>Li, Maozhen</creatorcontrib><creatorcontrib>Huang, Zhengwen</creatorcontrib><creatorcontrib>Khan, Mukhtaj</creatorcontrib><creatorcontrib>Khan, Mushtaq</creatorcontrib><collection>الدوريات العلمية والإحصائية - e-Marefa Academic and Statistical Periodicals</collection><collection>معرفة - المحتوى العربي الأكاديمي المتكامل - e-Marefa Academic Complete</collection><collection>Hindawi Publishing Complete</collection><collection>Hindawi Publishing Subscription Journals</collection><collection>Hindawi Publishing Open Access Journals</collection><collection>CrossRef</collection><collection>Mechanical & Transportation Engineering Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>Middle East & Africa Database</collection><collection>ProQuest Central Korea</collection><collection>Engineering Research Database</collection><collection>ProQuest Central Student</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>Computer Science Database</collection><collection>Civil Engineering Abstracts</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection><jtitle>Mathematical problems in engineering</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Ashton, Phillip M.</au><au>Taylor, Gareth A.</au><au>Li, Maozhen</au><au>Huang, Zhengwen</au><au>Khan, Mukhtaj</au><au>Khan, Mushtaq</au><au>Liatsis, Panos</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Optimizing Hadoop Performance for Big Data Analytics in Smart Grid</atitle><jtitle>Mathematical problems in engineering</jtitle><date>2017-01-01</date><risdate>2017</risdate><volume>2017</volume><issue>2017</issue><spage>1</spage><epage>11</epage><pages>1-11</pages><issn>1024-123X</issn><eissn>1563-5147</eissn><abstract>The rapid deployment of Phasor Measurement Units (PMUs) in power systems globally is leading to Big Data challenges. New high performance computing techniques are now required to process an ever increasing volume of data from PMUs. To that extent the Hadoop framework, an open source implementation of the MapReduce computing model, is gaining momentum for Big Data analytics in smart grid applications. However, Hadoop has over 190 configuration parameters, which can have a significant impact on the performance of the Hadoop framework. This paper presents an Enhanced Parallel Detrended Fluctuation Analysis (EPDFA) algorithm for scalable analytics on massive volumes of PMU data. The novel EPDFA algorithm builds on an enhanced Hadoop platform whose configuration parameters are optimized by Gene Expression Programming. Experimental results show that the EPDFA is 29 times faster than the sequential DFA in processing PMU data and 1.87 times faster than a parallel DFA, which utilizes the default Hadoop configuration settings.</abstract><cop>Cairo, Egypt</cop><pub>Hindawi Publishing Corporation</pub><doi>10.1155/2017/2198262</doi><tpages>11</tpages><orcidid>https://orcid.org/0000-0002-4933-6192</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1024-123X |
| ispartof | Mathematical problems in engineering, 2017-01, Vol.2017 (2017), p.1-11 |
| issn | 1024-123X 1563-5147 |
| language | eng |
| recordid | cdi_proquest_journals_1973512795 |
| source | Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals; Wiley-Blackwell Open Access Titles; Alma/SFX Local Collection |
| subjects | Algorithms Analytics Big Data Computation Computer science Configurations Data analysis Data management Datasets Electricity distribution Employment Fault tolerance Gene expression Smart grid Variations |
| title | Optimizing Hadoop Performance for Big Data Analytics in Smart Grid |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-05T13%3A17%3A18IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Optimizing%20Hadoop%20Performance%20for%20Big%20Data%20Analytics%20in%20Smart%20Grid&rft.jtitle=Mathematical%20problems%20in%20engineering&rft.au=Ashton,%20Phillip%20M.&rft.date=2017-01-01&rft.volume=2017&rft.issue=2017&rft.spage=1&rft.epage=11&rft.pages=1-11&rft.issn=1024-123X&rft.eissn=1563-5147&rft_id=info:doi/10.1155/2017/2198262&rft_dat=%3Cproquest_cross%3E1973512795%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1973512795&rft_id=info:pmid/&rfr_iscdi=true |