Distributed optimization of multi-class SVMs
Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, whic...
Gespeichert in:
| Veröffentlicht in: | PloS one 2017-06, Vol.12 (6), p.e0178161-e0178161 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | e0178161 |
|---|---|
| container_issue | 6 |
| container_start_page | e0178161 |
| container_title | PloS one |
| container_volume | 12 |
| creator | Alber, Maximilian Zimmert, Julian Dogan, Urun Kloft, Marius |
| description | Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, which require solving a quadratic program of size quadratically in the number of classes. We develop distributed algorithms for two all-in-one SVM formulations (Lee et al. and Weston and Watkins) that parallelize the computation evenly over the number of classes. This allows us to compare these models to one-vs.-rest SVMs on unprecedented scale. The results indicate superior accuracy on text classification data. |
| doi_str_mv | 10.1371/journal.pone.0178161 |
| format | Article |
| fullrecord | <record><control><sourceid>gale_plos_</sourceid><recordid>TN_cdi_plos_journals_1904976042</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><galeid>A493876523</galeid><doaj_id>oai_doaj_org_article_061471f88a47404785ce126bd0a1fd7d</doaj_id><sourcerecordid>A493876523</sourcerecordid><originalsourceid>FETCH-LOGICAL-c692t-77567634fc487c5f2e4077958199ab1f161367c73cb7205a7dcf3c14167e62bc3</originalsourceid><addsrcrecordid>eNqNkl1rFDEUhgdRbK3-A9EFQRTcNZl8nMyNUOrXQqVgtbchk0l2s2Qm6yQj6q83252WHemF5CLh5DnvOSd5i-IpRgtMAL_dhKHvlF9sQ2cWCIPAHN8rjnFFyjkvEbl_cD4qHsW4QYgRwfnD4qgUDBAgcly8ee9i6l09JNPMwja51v1RyYVuFuysHXxyc-1VjLPLqy_xcfHAKh_Nk3E_Kb5__PDt7PP8_OLT8uz0fK55VaY5AOPACbWaCtDMloYigIoJXFWqxjY3SjhoILqGEjEFjbZEY4o5GF7WmpwUz_e6Wx-iHAeNEleIVsARLTOx3BNNUBu57V2r-t8yKCevA6FfSdUnp72RiGMK2AqhKFBEQTBtcMnrBilsG2iy1rux2lC3ptGmS73yE9HpTefWchV-SkYZoYJngVejQB9-DCYm2bqojfeqM2G47psBEZTv-n7xD3r3dCO1UnkA19mQ6-qdqDylFRHAWUkytbiDyqsxrdPZFdbl-CTh9SQhM8n8Sis1xCiXl1__n724mrIvD9i1UT6tY_DDzkZxCtI9qPsQY2_s7SNjJHemvnkNuTO1HE2d054dftBt0o2LyV-o6e6P</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1904976042</pqid></control><display><type>article</type><title>Distributed optimization of multi-class SVMs</title><source>MEDLINE</source><source>Full-Text Journals in Chemistry (Open access)</source><source>Public Library of Science</source><source>PubMed</source><source>DOAJ Directory of Open Access Journals</source><source>EZB Electronic Journals Library</source><creator>Alber, Maximilian ; Zimmert, Julian ; Dogan, Urun ; Kloft, Marius</creator><contributor>Zou, Quan</contributor><creatorcontrib>Alber, Maximilian ; Zimmert, Julian ; Dogan, Urun ; Kloft, Marius ; Zou, Quan</creatorcontrib><description>Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, which require solving a quadratic program of size quadratically in the number of classes. We develop distributed algorithms for two all-in-one SVM formulations (Lee et al. and Weston and Watkins) that parallelize the computation evenly over the number of classes. This allows us to compare these models to one-vs.-rest SVMs on unprecedented scale. The results indicate superior accuracy on text classification data.</description><identifier>ISSN: 1932-6203</identifier><identifier>EISSN: 1932-6203</identifier><identifier>DOI: 10.1371/journal.pone.0178161</identifier><identifier>PMID: 28570703</identifier><language>eng</language><publisher>United States: Public Library of Science</publisher><subject>Algorithms ; Analysis ; Artificial intelligence ; Bibliographic data bases ; Biology and Life Sciences ; Classification ; Computer and Information Sciences ; Computer applications ; Computer science ; Formulations ; Information processing ; Machine learning ; Mathematical models ; Methods ; Models, Theoretical ; Neural networks ; Optimization ; Parallel processing ; Physical Sciences ; Problems ; Research and Analysis Methods ; Scale (ratio) ; Support Vector Machine ; Training</subject><ispartof>PloS one, 2017-06, Vol.12 (6), p.e0178161-e0178161</ispartof><rights>COPYRIGHT 2017 Public Library of Science</rights><rights>2017 Alber et al. This is an open access article distributed under the terms of the Creative Commons Attribution License: http://creativecommons.org/licenses/by/4.0/ (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>2017 Alber et al 2017 Alber et al</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c692t-77567634fc487c5f2e4077958199ab1f161367c73cb7205a7dcf3c14167e62bc3</citedby><cites>FETCH-LOGICAL-c692t-77567634fc487c5f2e4077958199ab1f161367c73cb7205a7dcf3c14167e62bc3</cites><orcidid>0000-0002-4614-6297</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC5453486/pdf/$$EPDF$$P50$$Gpubmedcentral$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC5453486/$$EHTML$$P50$$Gpubmedcentral$$Hfree_for_read</linktohtml><link.rule.ids>230,314,727,780,784,864,885,2102,2928,23866,27924,27925,53791,53793,79472,79473</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/28570703$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><contributor>Zou, Quan</contributor><creatorcontrib>Alber, Maximilian</creatorcontrib><creatorcontrib>Zimmert, Julian</creatorcontrib><creatorcontrib>Dogan, Urun</creatorcontrib><creatorcontrib>Kloft, Marius</creatorcontrib><title>Distributed optimization of multi-class SVMs</title><title>PloS one</title><addtitle>PLoS One</addtitle><description>Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, which require solving a quadratic program of size quadratically in the number of classes. We develop distributed algorithms for two all-in-one SVM formulations (Lee et al. and Weston and Watkins) that parallelize the computation evenly over the number of classes. This allows us to compare these models to one-vs.-rest SVMs on unprecedented scale. The results indicate superior accuracy on text classification data.</description><subject>Algorithms</subject><subject>Analysis</subject><subject>Artificial intelligence</subject><subject>Bibliographic data bases</subject><subject>Biology and Life Sciences</subject><subject>Classification</subject><subject>Computer and Information Sciences</subject><subject>Computer applications</subject><subject>Computer science</subject><subject>Formulations</subject><subject>Information processing</subject><subject>Machine learning</subject><subject>Mathematical models</subject><subject>Methods</subject><subject>Models, Theoretical</subject><subject>Neural networks</subject><subject>Optimization</subject><subject>Parallel processing</subject><subject>Physical Sciences</subject><subject>Problems</subject><subject>Research and Analysis Methods</subject><subject>Scale (ratio)</subject><subject>Support Vector Machine</subject><subject>Training</subject><issn>1932-6203</issn><issn>1932-6203</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>DOA</sourceid><recordid>eNqNkl1rFDEUhgdRbK3-A9EFQRTcNZl8nMyNUOrXQqVgtbchk0l2s2Qm6yQj6q83252WHemF5CLh5DnvOSd5i-IpRgtMAL_dhKHvlF9sQ2cWCIPAHN8rjnFFyjkvEbl_cD4qHsW4QYgRwfnD4qgUDBAgcly8ee9i6l09JNPMwja51v1RyYVuFuysHXxyc-1VjLPLqy_xcfHAKh_Nk3E_Kb5__PDt7PP8_OLT8uz0fK55VaY5AOPACbWaCtDMloYigIoJXFWqxjY3SjhoILqGEjEFjbZEY4o5GF7WmpwUz_e6Wx-iHAeNEleIVsARLTOx3BNNUBu57V2r-t8yKCevA6FfSdUnp72RiGMK2AqhKFBEQTBtcMnrBilsG2iy1rux2lC3ptGmS73yE9HpTefWchV-SkYZoYJngVejQB9-DCYm2bqojfeqM2G47psBEZTv-n7xD3r3dCO1UnkA19mQ6-qdqDylFRHAWUkytbiDyqsxrdPZFdbl-CTh9SQhM8n8Sis1xCiXl1__n724mrIvD9i1UT6tY_DDzkZxCtI9qPsQY2_s7SNjJHemvnkNuTO1HE2d054dftBt0o2LyV-o6e6P</recordid><startdate>20170601</startdate><enddate>20170601</enddate><creator>Alber, Maximilian</creator><creator>Zimmert, Julian</creator><creator>Dogan, Urun</creator><creator>Kloft, Marius</creator><general>Public Library of Science</general><general>Public Library of Science (PLoS)</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>IOV</scope><scope>ISR</scope><scope>3V.</scope><scope>7QG</scope><scope>7QL</scope><scope>7QO</scope><scope>7RV</scope><scope>7SN</scope><scope>7SS</scope><scope>7T5</scope><scope>7TG</scope><scope>7TM</scope><scope>7U9</scope><scope>7X2</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>8AO</scope><scope>8C1</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FH</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AEUYN</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>ATCPS</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>BHPHI</scope><scope>C1K</scope><scope>CCPQU</scope><scope>D1I</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>H94</scope><scope>HCIFZ</scope><scope>K9.</scope><scope>KB.</scope><scope>KB0</scope><scope>KL.</scope><scope>L6V</scope><scope>LK8</scope><scope>M0K</scope><scope>M0S</scope><scope>M1P</scope><scope>M7N</scope><scope>M7P</scope><scope>M7S</scope><scope>NAPCQ</scope><scope>P5Z</scope><scope>P62</scope><scope>P64</scope><scope>PATMY</scope><scope>PDBOC</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope><scope>PYCSY</scope><scope>RC3</scope><scope>7X8</scope><scope>5PM</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0002-4614-6297</orcidid></search><sort><creationdate>20170601</creationdate><title>Distributed optimization of multi-class SVMs</title><author>Alber, Maximilian ; Zimmert, Julian ; Dogan, Urun ; Kloft, Marius</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c692t-77567634fc487c5f2e4077958199ab1f161367c73cb7205a7dcf3c14167e62bc3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Algorithms</topic><topic>Analysis</topic><topic>Artificial intelligence</topic><topic>Bibliographic data bases</topic><topic>Biology and Life Sciences</topic><topic>Classification</topic><topic>Computer and Information Sciences</topic><topic>Computer applications</topic><topic>Computer science</topic><topic>Formulations</topic><topic>Information processing</topic><topic>Machine learning</topic><topic>Mathematical models</topic><topic>Methods</topic><topic>Models, Theoretical</topic><topic>Neural networks</topic><topic>Optimization</topic><topic>Parallel processing</topic><topic>Physical Sciences</topic><topic>Problems</topic><topic>Research and Analysis Methods</topic><topic>Scale (ratio)</topic><topic>Support Vector Machine</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Alber, Maximilian</creatorcontrib><creatorcontrib>Zimmert, Julian</creatorcontrib><creatorcontrib>Dogan, Urun</creatorcontrib><creatorcontrib>Kloft, Marius</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Gale in Context : Opposing Viewpoints</collection><collection>Gale In Context: Science</collection><collection>ProQuest Central (Corporate)</collection><collection>Animal Behavior Abstracts</collection><collection>Bacteriology Abstracts (Microbiology B)</collection><collection>Biotechnology Research Abstracts</collection><collection>ProQuest Nursing & Allied Health Database</collection><collection>Ecology Abstracts</collection><collection>Entomology Abstracts (Full archive)</collection><collection>Immunology Abstracts</collection><collection>Meteorological & Geoastrophysical Abstracts</collection><collection>Nucleic Acids Abstracts</collection><collection>Virology and AIDS Abstracts</collection><collection>Agricultural Science Collection</collection><collection>ProQuest Health & Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>ProQuest Pharma Collection</collection><collection>ProQuest Public Health Database</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest One Sustainability</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>Agricultural & Environmental Science Collection</collection><collection>ProQuest Central Essentials</collection><collection>Biological Science Collection</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Environmental Sciences and Pollution Management</collection><collection>ProQuest One Community College</collection><collection>ProQuest Materials Science Collection</collection><collection>ProQuest Central</collection><collection>Engineering Research Database</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>AIDS and Cancer Research Abstracts</collection><collection>SciTech Premium Collection (Proquest) (PQ_SDU_P3)</collection><collection>ProQuest Health & Medical Complete (Alumni)</collection><collection>Materials Science Database</collection><collection>Nursing & Allied Health Database (Alumni Edition)</collection><collection>Meteorological & Geoastrophysical Abstracts - Academic</collection><collection>ProQuest Engineering Collection</collection><collection>Biological Sciences</collection><collection>Agriculture Science Database</collection><collection>Health & Medical Collection (Alumni Edition)</collection><collection>PML(ProQuest Medical Library)</collection><collection>Algology Mycology and Protozoology Abstracts (Microbiology C)</collection><collection>Biological Science Database</collection><collection>Engineering Database</collection><collection>Nursing & Allied Health Premium</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>Environmental Science Database</collection><collection>Materials Science Collection</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection><collection>Environmental Science Collection</collection><collection>Genetics Abstracts</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>PloS one</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Alber, Maximilian</au><au>Zimmert, Julian</au><au>Dogan, Urun</au><au>Kloft, Marius</au><au>Zou, Quan</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Distributed optimization of multi-class SVMs</atitle><jtitle>PloS one</jtitle><addtitle>PLoS One</addtitle><date>2017-06-01</date><risdate>2017</risdate><volume>12</volume><issue>6</issue><spage>e0178161</spage><epage>e0178161</epage><pages>e0178161-e0178161</pages><issn>1932-6203</issn><eissn>1932-6203</eissn><abstract>Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, which require solving a quadratic program of size quadratically in the number of classes. We develop distributed algorithms for two all-in-one SVM formulations (Lee et al. and Weston and Watkins) that parallelize the computation evenly over the number of classes. This allows us to compare these models to one-vs.-rest SVMs on unprecedented scale. The results indicate superior accuracy on text classification data.</abstract><cop>United States</cop><pub>Public Library of Science</pub><pmid>28570703</pmid><doi>10.1371/journal.pone.0178161</doi><tpages>e0178161</tpages><orcidid>https://orcid.org/0000-0002-4614-6297</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 1932-6203 |
| ispartof | PloS one, 2017-06, Vol.12 (6), p.e0178161-e0178161 |
| issn | 1932-6203 1932-6203 |
| language | eng |
| recordid | cdi_plos_journals_1904976042 |
| source | MEDLINE; Full-Text Journals in Chemistry (Open access); Public Library of Science; PubMed; DOAJ Directory of Open Access Journals; EZB Electronic Journals Library |
| subjects | Algorithms Analysis Artificial intelligence Bibliographic data bases Biology and Life Sciences Classification Computer and Information Sciences Computer applications Computer science Formulations Information processing Machine learning Mathematical models Methods Models, Theoretical Neural networks Optimization Parallel processing Physical Sciences Problems Research and Analysis Methods Scale (ratio) Support Vector Machine Training |
| title | Distributed optimization of multi-class SVMs |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-07T17%3A09%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-gale_plos_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Distributed%20optimization%20of%20multi-class%20SVMs&rft.jtitle=PloS%20one&rft.au=Alber,%20Maximilian&rft.date=2017-06-01&rft.volume=12&rft.issue=6&rft.spage=e0178161&rft.epage=e0178161&rft.pages=e0178161-e0178161&rft.issn=1932-6203&rft.eissn=1932-6203&rft_id=info:doi/10.1371/journal.pone.0178161&rft_dat=%3Cgale_plos_%3EA493876523%3C/gale_plos_%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1904976042&rft_id=info:pmid/28570703&rft_galeid=A493876523&rft_doaj_id=oai_doaj_org_article_061471f88a47404785ce126bd0a1fd7d&rfr_iscdi=true |