Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network

Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs significant number of pixellevel annotated data, which is often unavailable. To address this lack, in this paper, we leverage, on one hand, massive amount of...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Souly, Nasim, Spampinato, Concetto, Shah, Mubarak
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Souly, Nasim
Spampinato, Concetto
Shah, Mubarak
description Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs significant number of pixellevel annotated data, which is often unavailable. To address this lack, in this paper, we leverage, on one hand, massive amount of available unlabeled or weakly labeled data, and on the other hand, non-real images created through Generative Adversarial Networks. In particular, we propose a semi-supervised framework ,based on Generative Adversarial Networks (GANs), which consists of a generator network to provide extra training examples to a multi-class classifier, acting as discriminator in the GAN framework, that assigns sample a label y from the K possible classes or marks it as a fake sample (extra class). The underlying idea is that adding large fake visual data forces real samples to be close in the feature space, enabling a bottom-up clustering process, which, in turn, improves multiclass pixel classification. To ensure higher quality of generated images for GANs with consequent improved pixel classification, we extend the above framework by adding weakly annotated data, i.e., we provide class level information to the generator. We tested our approaches on several challenging benchmarking visual datasets, i.e. PASCAL, SiftFLow, Stanford and CamVid, achieving competitive performance also compared to state-of-the-art semantic segmentation method
doi_str_mv 10.48550/arxiv.1703.09695
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_1703_09695</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1703_09695</sourcerecordid><originalsourceid>FETCH-LOGICAL-a675-e1e443b6fd17846c0add9e85ed7f842018acb547590c8cce95a95b8eab2368c03</originalsourceid><addsrcrecordid>eNotj8tOwzAURL1hgQofwAr_QIJdv5dVBQWpgkVbsYxu7JvKauJWTgj07wmlqxnNkUY6hDxwVkqrFHuC_BPHkhsmSua0U7dkt8EuUkiBfiIc2jPdfJ0wj7HHQCcEaYh-KvsO0wBDPCa662Pa0xUmzNMwIl2EEXMPOUJL33H4PubDHblpoO3x_pozsn153i5fi_XH6m25WBegjSqQo5Si1k3gxkrtGYTg0CoMprFyzrgFXytplGPeeo9OgVO1RajnQlvPxIw8_t9evKpTjh3kc_XnV138xC9oCExi</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network</title><source>arXiv.org</source><creator>Souly, Nasim ; Spampinato, Concetto ; Shah, Mubarak</creator><creatorcontrib>Souly, Nasim ; Spampinato, Concetto ; Shah, Mubarak</creatorcontrib><description>Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs significant number of pixellevel annotated data, which is often unavailable. To address this lack, in this paper, we leverage, on one hand, massive amount of available unlabeled or weakly labeled data, and on the other hand, non-real images created through Generative Adversarial Networks. In particular, we propose a semi-supervised framework ,based on Generative Adversarial Networks (GANs), which consists of a generator network to provide extra training examples to a multi-class classifier, acting as discriminator in the GAN framework, that assigns sample a label y from the K possible classes or marks it as a fake sample (extra class). The underlying idea is that adding large fake visual data forces real samples to be close in the feature space, enabling a bottom-up clustering process, which, in turn, improves multiclass pixel classification. To ensure higher quality of generated images for GANs with consequent improved pixel classification, we extend the above framework by adding weakly annotated data, i.e., we provide class level information to the generator. We tested our approaches on several challenging benchmarking visual datasets, i.e. PASCAL, SiftFLow, Stanford and CamVid, achieving competitive performance also compared to state-of-the-art semantic segmentation method</description><identifier>DOI: 10.48550/arxiv.1703.09695</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2017-03</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/1703.09695$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.1703.09695$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Souly, Nasim</creatorcontrib><creatorcontrib>Spampinato, Concetto</creatorcontrib><creatorcontrib>Shah, Mubarak</creatorcontrib><title>Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network</title><description>Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs significant number of pixellevel annotated data, which is often unavailable. To address this lack, in this paper, we leverage, on one hand, massive amount of available unlabeled or weakly labeled data, and on the other hand, non-real images created through Generative Adversarial Networks. In particular, we propose a semi-supervised framework ,based on Generative Adversarial Networks (GANs), which consists of a generator network to provide extra training examples to a multi-class classifier, acting as discriminator in the GAN framework, that assigns sample a label y from the K possible classes or marks it as a fake sample (extra class). The underlying idea is that adding large fake visual data forces real samples to be close in the feature space, enabling a bottom-up clustering process, which, in turn, improves multiclass pixel classification. To ensure higher quality of generated images for GANs with consequent improved pixel classification, we extend the above framework by adding weakly annotated data, i.e., we provide class level information to the generator. We tested our approaches on several challenging benchmarking visual datasets, i.e. PASCAL, SiftFLow, Stanford and CamVid, achieving competitive performance also compared to state-of-the-art semantic segmentation method</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj8tOwzAURL1hgQofwAr_QIJdv5dVBQWpgkVbsYxu7JvKauJWTgj07wmlqxnNkUY6hDxwVkqrFHuC_BPHkhsmSua0U7dkt8EuUkiBfiIc2jPdfJ0wj7HHQCcEaYh-KvsO0wBDPCa662Pa0xUmzNMwIl2EEXMPOUJL33H4PubDHblpoO3x_pozsn153i5fi_XH6m25WBegjSqQo5Si1k3gxkrtGYTg0CoMprFyzrgFXytplGPeeo9OgVO1RajnQlvPxIw8_t9evKpTjh3kc_XnV138xC9oCExi</recordid><startdate>20170328</startdate><enddate>20170328</enddate><creator>Souly, Nasim</creator><creator>Spampinato, Concetto</creator><creator>Shah, Mubarak</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20170328</creationdate><title>Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network</title><author>Souly, Nasim ; Spampinato, Concetto ; Shah, Mubarak</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a675-e1e443b6fd17846c0add9e85ed7f842018acb547590c8cce95a95b8eab2368c03</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Souly, Nasim</creatorcontrib><creatorcontrib>Spampinato, Concetto</creatorcontrib><creatorcontrib>Shah, Mubarak</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Souly, Nasim</au><au>Spampinato, Concetto</au><au>Shah, Mubarak</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network</atitle><date>2017-03-28</date><risdate>2017</risdate><abstract>Semantic segmentation has been a long standing challenging task in computer vision. It aims at assigning a label to each image pixel and needs significant number of pixellevel annotated data, which is often unavailable. To address this lack, in this paper, we leverage, on one hand, massive amount of available unlabeled or weakly labeled data, and on the other hand, non-real images created through Generative Adversarial Networks. In particular, we propose a semi-supervised framework ,based on Generative Adversarial Networks (GANs), which consists of a generator network to provide extra training examples to a multi-class classifier, acting as discriminator in the GAN framework, that assigns sample a label y from the K possible classes or marks it as a fake sample (extra class). The underlying idea is that adding large fake visual data forces real samples to be close in the feature space, enabling a bottom-up clustering process, which, in turn, improves multiclass pixel classification. To ensure higher quality of generated images for GANs with consequent improved pixel classification, we extend the above framework by adding weakly annotated data, i.e., we provide class level information to the generator. We tested our approaches on several challenging benchmarking visual datasets, i.e. PASCAL, SiftFLow, Stanford and CamVid, achieving competitive performance also compared to state-of-the-art semantic segmentation method</abstract><doi>10.48550/arxiv.1703.09695</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.1703.09695
ispartof
issn
language eng
recordid cdi_arxiv_primary_1703_09695
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T15%3A34%3A51IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Semi%20and%20Weakly%20Supervised%20Semantic%20Segmentation%20Using%20Generative%20Adversarial%20Network&rft.au=Souly,%20Nasim&rft.date=2017-03-28&rft_id=info:doi/10.48550/arxiv.1703.09695&rft_dat=%3Carxiv_GOX%3E1703_09695%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true