Research on Attribute Dimension Partition Based on SVM Classifying and MapReduce
The data analysis is closely related to data attribute dimension. The traditional extraction and partition of data attribute dimension is so manual and inefficiency as to not meet the needs of analysing big data. This paper proposed an attribute dimension partition scheme based on SVM classifying an...
Gespeichert in:
| Veröffentlicht in: | Wireless personal communications 2018-10, Vol.102 (4), p.2759-2774 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 2774 |
|---|---|
| container_issue | 4 |
| container_start_page | 2759 |
| container_title | Wireless personal communications |
| container_volume | 102 |
| creator | Zhao, Wenbin Fan, Tongrang Nie, Yongchuan Wu, Feng Wen, Hou |
| description | The data analysis is closely related to data attribute dimension. The traditional extraction and partition of data attribute dimension is so manual and inefficiency as to not meet the needs of analysing big data. This paper proposed an attribute dimension partition scheme based on SVM classifying and MapReduce for analysing big data. This scheme improve traditional SVM classifying method by combining Euclidean distance theory for overcoming its disadvantages, and adopts punish coefficient to reduce the unbalance of data distribution. With the improved SVM classifying method, the implementation of attribute dimension partition take MapReduce model of Hadoop as process engine, use TF–IDF vector to save the extracted attribute dimension, and use
k
-means clustering algorithm to clustering partition. The experiment result shows that the execution efficiency of the proposed method is enhanced, and while the rationality of partition is guaranteed, the increasing of data attributes does not significantly increase the execution time. |
| doi_str_mv | 10.1007/s11277-018-5301-9 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2103763747</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2103763747</sourcerecordid><originalsourceid>FETCH-LOGICAL-c316t-b68d1387a976daada0af26339abe72daf96df891e372261f5e458841273d87943</originalsourceid><addsrcrecordid>eNp1kMtOwzAQRS0EEqXwAewisTZ47MSPZSlPqRVVeYid5cZOSdUmxXYW_XscBYkVqxmNzp3RHIQugVwDIeImAFAhMAGJC0YAqyM0gkJQLFn-eYxGRFGFOQV6is5C2BCSUoqO0GLpgjO-_MraJpvE6OtVF112V-9cE-o0Wxgf69h3tyY422OvH_NsujUh1NWhbtaZaWw2N_uls13pztFJZbbBXfzWMXp_uH-bPuHZy-PzdDLDJQMe8YpLC0wKowS3xlhDTEU5Y8qsnKDWVIrbSipwTFDKoSpcXkiZpyeZlULlbIyuhr173353LkS9aTvfpJOaAmGCM5GLRMFAlb4NwbtK7329M_6ggehenB7E6SRO9-K0Shk6ZEJim7Xzf5v_D_0AJBZvaA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2103763747</pqid></control><display><type>article</type><title>Research on Attribute Dimension Partition Based on SVM Classifying and MapReduce</title><source>Springer Online Journals Complete</source><creator>Zhao, Wenbin ; Fan, Tongrang ; Nie, Yongchuan ; Wu, Feng ; Wen, Hou</creator><creatorcontrib>Zhao, Wenbin ; Fan, Tongrang ; Nie, Yongchuan ; Wu, Feng ; Wen, Hou</creatorcontrib><description>The data analysis is closely related to data attribute dimension. The traditional extraction and partition of data attribute dimension is so manual and inefficiency as to not meet the needs of analysing big data. This paper proposed an attribute dimension partition scheme based on SVM classifying and MapReduce for analysing big data. This scheme improve traditional SVM classifying method by combining Euclidean distance theory for overcoming its disadvantages, and adopts punish coefficient to reduce the unbalance of data distribution. With the improved SVM classifying method, the implementation of attribute dimension partition take MapReduce model of Hadoop as process engine, use TF–IDF vector to save the extracted attribute dimension, and use
k
-means clustering algorithm to clustering partition. The experiment result shows that the execution efficiency of the proposed method is enhanced, and while the rationality of partition is guaranteed, the increasing of data attributes does not significantly increase the execution time.</description><identifier>ISSN: 0929-6212</identifier><identifier>EISSN: 1572-834X</identifier><identifier>DOI: 10.1007/s11277-018-5301-9</identifier><language>eng</language><publisher>New York: Springer US</publisher><subject>Algorithms ; Classification ; Cluster analysis ; Clustering ; Communications Engineering ; Computer Communication Networks ; Data analysis ; Engineering ; Euclidean geometry ; Networks ; Partitions ; Signal,Image and Speech Processing ; Vector quantization</subject><ispartof>Wireless personal communications, 2018-10, Vol.102 (4), p.2759-2774</ispartof><rights>Springer Science+Business Media, LLC, part of Springer Nature 2018</rights><rights>Copyright Springer Science & Business Media 2018</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c316t-b68d1387a976daada0af26339abe72daf96df891e372261f5e458841273d87943</citedby><cites>FETCH-LOGICAL-c316t-b68d1387a976daada0af26339abe72daf96df891e372261f5e458841273d87943</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s11277-018-5301-9$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s11277-018-5301-9$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,780,784,27924,27925,41488,42557,51319</link.rule.ids></links><search><creatorcontrib>Zhao, Wenbin</creatorcontrib><creatorcontrib>Fan, Tongrang</creatorcontrib><creatorcontrib>Nie, Yongchuan</creatorcontrib><creatorcontrib>Wu, Feng</creatorcontrib><creatorcontrib>Wen, Hou</creatorcontrib><title>Research on Attribute Dimension Partition Based on SVM Classifying and MapReduce</title><title>Wireless personal communications</title><addtitle>Wireless Pers Commun</addtitle><description>The data analysis is closely related to data attribute dimension. The traditional extraction and partition of data attribute dimension is so manual and inefficiency as to not meet the needs of analysing big data. This paper proposed an attribute dimension partition scheme based on SVM classifying and MapReduce for analysing big data. This scheme improve traditional SVM classifying method by combining Euclidean distance theory for overcoming its disadvantages, and adopts punish coefficient to reduce the unbalance of data distribution. With the improved SVM classifying method, the implementation of attribute dimension partition take MapReduce model of Hadoop as process engine, use TF–IDF vector to save the extracted attribute dimension, and use
k
-means clustering algorithm to clustering partition. The experiment result shows that the execution efficiency of the proposed method is enhanced, and while the rationality of partition is guaranteed, the increasing of data attributes does not significantly increase the execution time.</description><subject>Algorithms</subject><subject>Classification</subject><subject>Cluster analysis</subject><subject>Clustering</subject><subject>Communications Engineering</subject><subject>Computer Communication Networks</subject><subject>Data analysis</subject><subject>Engineering</subject><subject>Euclidean geometry</subject><subject>Networks</subject><subject>Partitions</subject><subject>Signal,Image and Speech Processing</subject><subject>Vector quantization</subject><issn>0929-6212</issn><issn>1572-834X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2018</creationdate><recordtype>article</recordtype><recordid>eNp1kMtOwzAQRS0EEqXwAewisTZ47MSPZSlPqRVVeYid5cZOSdUmxXYW_XscBYkVqxmNzp3RHIQugVwDIeImAFAhMAGJC0YAqyM0gkJQLFn-eYxGRFGFOQV6is5C2BCSUoqO0GLpgjO-_MraJpvE6OtVF112V-9cE-o0Wxgf69h3tyY422OvH_NsujUh1NWhbtaZaWw2N_uls13pztFJZbbBXfzWMXp_uH-bPuHZy-PzdDLDJQMe8YpLC0wKowS3xlhDTEU5Y8qsnKDWVIrbSipwTFDKoSpcXkiZpyeZlULlbIyuhr173353LkS9aTvfpJOaAmGCM5GLRMFAlb4NwbtK7329M_6ggehenB7E6SRO9-K0Shk6ZEJim7Xzf5v_D_0AJBZvaA</recordid><startdate>20181001</startdate><enddate>20181001</enddate><creator>Zhao, Wenbin</creator><creator>Fan, Tongrang</creator><creator>Nie, Yongchuan</creator><creator>Wu, Feng</creator><creator>Wen, Hou</creator><general>Springer US</general><general>Springer Nature B.V</general><scope>AAYXX</scope><scope>CITATION</scope></search><sort><creationdate>20181001</creationdate><title>Research on Attribute Dimension Partition Based on SVM Classifying and MapReduce</title><author>Zhao, Wenbin ; Fan, Tongrang ; Nie, Yongchuan ; Wu, Feng ; Wen, Hou</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c316t-b68d1387a976daada0af26339abe72daf96df891e372261f5e458841273d87943</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2018</creationdate><topic>Algorithms</topic><topic>Classification</topic><topic>Cluster analysis</topic><topic>Clustering</topic><topic>Communications Engineering</topic><topic>Computer Communication Networks</topic><topic>Data analysis</topic><topic>Engineering</topic><topic>Euclidean geometry</topic><topic>Networks</topic><topic>Partitions</topic><topic>Signal,Image and Speech Processing</topic><topic>Vector quantization</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhao, Wenbin</creatorcontrib><creatorcontrib>Fan, Tongrang</creatorcontrib><creatorcontrib>Nie, Yongchuan</creatorcontrib><creatorcontrib>Wu, Feng</creatorcontrib><creatorcontrib>Wen, Hou</creatorcontrib><collection>CrossRef</collection><jtitle>Wireless personal communications</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Zhao, Wenbin</au><au>Fan, Tongrang</au><au>Nie, Yongchuan</au><au>Wu, Feng</au><au>Wen, Hou</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Research on Attribute Dimension Partition Based on SVM Classifying and MapReduce</atitle><jtitle>Wireless personal communications</jtitle><stitle>Wireless Pers Commun</stitle><date>2018-10-01</date><risdate>2018</risdate><volume>102</volume><issue>4</issue><spage>2759</spage><epage>2774</epage><pages>2759-2774</pages><issn>0929-6212</issn><eissn>1572-834X</eissn><abstract>The data analysis is closely related to data attribute dimension. The traditional extraction and partition of data attribute dimension is so manual and inefficiency as to not meet the needs of analysing big data. This paper proposed an attribute dimension partition scheme based on SVM classifying and MapReduce for analysing big data. This scheme improve traditional SVM classifying method by combining Euclidean distance theory for overcoming its disadvantages, and adopts punish coefficient to reduce the unbalance of data distribution. With the improved SVM classifying method, the implementation of attribute dimension partition take MapReduce model of Hadoop as process engine, use TF–IDF vector to save the extracted attribute dimension, and use
k
-means clustering algorithm to clustering partition. The experiment result shows that the execution efficiency of the proposed method is enhanced, and while the rationality of partition is guaranteed, the increasing of data attributes does not significantly increase the execution time.</abstract><cop>New York</cop><pub>Springer US</pub><doi>10.1007/s11277-018-5301-9</doi><tpages>16</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0929-6212 |
| ispartof | Wireless personal communications, 2018-10, Vol.102 (4), p.2759-2774 |
| issn | 0929-6212 1572-834X |
| language | eng |
| recordid | cdi_proquest_journals_2103763747 |
| source | Springer Online Journals Complete |
| subjects | Algorithms Classification Cluster analysis Clustering Communications Engineering Computer Communication Networks Data analysis Engineering Euclidean geometry Networks Partitions Signal,Image and Speech Processing Vector quantization |
| title | Research on Attribute Dimension Partition Based on SVM Classifying and MapReduce |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-05T16%3A11%3A05IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Research%20on%20Attribute%20Dimension%20Partition%20Based%20on%20SVM%20Classifying%20and%20MapReduce&rft.jtitle=Wireless%20personal%20communications&rft.au=Zhao,%20Wenbin&rft.date=2018-10-01&rft.volume=102&rft.issue=4&rft.spage=2759&rft.epage=2774&rft.pages=2759-2774&rft.issn=0929-6212&rft.eissn=1572-834X&rft_id=info:doi/10.1007/s11277-018-5301-9&rft_dat=%3Cproquest_cross%3E2103763747%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2103763747&rft_id=info:pmid/&rfr_iscdi=true |