PMSGAN: Parallel Multistage GANs for Face Image Translation

In this article, we address the face image translation task, which aims to translate a face image of a source domain to a target domain. Although significant progress has been made by recent studies, face image translation is still a challenging task because it has more strict requirements for textu...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transaction on neural networks and learning systems 2024-07, Vol.35 (7), p.9352-9365
Hauptverfasser: Liang, Changcheng, Zhu, Mingrui, Wang, Nannan, Yang, Heng, Gao, Xinbo
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 9365
container_issue 7
container_start_page 9352
container_title IEEE transaction on neural networks and learning systems
container_volume 35
creator Liang, Changcheng
Zhu, Mingrui
Wang, Nannan
Yang, Heng
Gao, Xinbo
description In this article, we address the face image translation task, which aims to translate a face image of a source domain to a target domain. Although significant progress has been made by recent studies, face image translation is still a challenging task because it has more strict requirements for texture details: even a few artifacts will greatly affect the impression of generated face images. Targeting to synthesize high-quality face images with admirable visual appearance, we revisit the coarse-to-fine strategy and propose a novel p arallel m ultistage architecture on the basis of g enerative a dversarial n etworks (PMSGAN). More specifically, PMSGAN progressively learns the translation function by disintegrating the general synthesis process into multiple parallel stages that take images with gradually decreasing spatial resolution as inputs. To prompt the information exchange between various stages, a cross-stage atrous spatial pyramid (CSASP) structure is specially designed to receive and fuse the contextual information from other stages. At the end of the parallel model, we introduce a novel attention-based module that leverages multistage decoded outputs as in situ supervised attention to refine the final activations and yield the target image. Extensive experiments on several face image translation benchmarks show that PMSGAN performs considerably better than state-of-the-art approaches.
doi_str_mv 10.1109/TNNLS.2022.3233025
format Article
fullrecord <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_crossref_primary_10_1109_TNNLS_2022_3233025</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10014017</ieee_id><sourcerecordid>2797147005</sourcerecordid><originalsourceid>FETCH-LOGICAL-c282t-ad516d86687b79cd2e5123b31f04db3993ca20f5eb319659410faeb35e34b7083</originalsourceid><addsrcrecordid>eNpdkEFLwzAUx4Mobsx9AREpePHS-ZI0SaOnMdwcbHOwCd5C2qbS0bUzaQ9-ezM3h5hL8l5-78_jh9A1hgHGIB_Wi8VsNSBAyIASSoGwM9QlmJOQ0Dg-P73Fewf1nduAPxwYj-Ql6lABOOYR66Kn5Xw1GS4eg6W2uixNGczbsilcoz9M4D9ckNc2GOvUBNPtvre2unKlboq6ukIXuS6d6R_vHnobP69HL-HsdTIdDWdhSmLShDpjmGcx57FIhEwzYhgmNKE4hyhLqJQ01QRyZnxLciYjDLn2BTM0SgTEtIfuD7k7W3-2xjVqW7jUlKWuTN06RYQUOBIAzKN3_9BN3drKb6coiBgkk5R7ihyo1NbOWZOrnS222n4pDGpvV_3YVXu76mjXD90eo9tka7LTyK9LD9wcgMIY8ycRcARY0G-HKXp7</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3078095936</pqid></control><display><type>article</type><title>PMSGAN: Parallel Multistage GANs for Face Image Translation</title><source>IEEE Electronic Library (IEL)</source><creator>Liang, Changcheng ; Zhu, Mingrui ; Wang, Nannan ; Yang, Heng ; Gao, Xinbo</creator><creatorcontrib>Liang, Changcheng ; Zhu, Mingrui ; Wang, Nannan ; Yang, Heng ; Gao, Xinbo</creatorcontrib><description>In this article, we address the face image translation task, which aims to translate a face image of a source domain to a target domain. Although significant progress has been made by recent studies, face image translation is still a challenging task because it has more strict requirements for texture details: even a few artifacts will greatly affect the impression of generated face images. Targeting to synthesize high-quality face images with admirable visual appearance, we revisit the coarse-to-fine strategy and propose a novel p arallel m ultistage architecture on the basis of g enerative a dversarial n etworks (PMSGAN). More specifically, PMSGAN progressively learns the translation function by disintegrating the general synthesis process into multiple parallel stages that take images with gradually decreasing spatial resolution as inputs. To prompt the information exchange between various stages, a cross-stage atrous spatial pyramid (CSASP) structure is specially designed to receive and fuse the contextual information from other stages. At the end of the parallel model, we introduce a novel attention-based module that leverages multistage decoded outputs as in situ supervised attention to refine the final activations and yield the target image. Extensive experiments on several face image translation benchmarks show that PMSGAN performs considerably better than state-of-the-art approaches.</description><identifier>ISSN: 2162-237X</identifier><identifier>ISSN: 2162-2388</identifier><identifier>EISSN: 2162-2388</identifier><identifier>DOI: 10.1109/TNNLS.2022.3233025</identifier><identifier>PMID: 37018645</identifier><identifier>CODEN: ITNNAL</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>Atrous spatial pyramid ; Benchmarks ; Decoding ; Disintegration ; Face ; Face detection ; face image translation ; Generative adversarial networks ; Image analysis ; Image quality ; Information processing ; parallel multistage ; Spatial discrimination ; Spatial resolution ; Training ; Translation ; Visual tasks</subject><ispartof>IEEE transaction on neural networks and learning systems, 2024-07, Vol.35 (7), p.9352-9365</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024</rights><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c282t-ad516d86687b79cd2e5123b31f04db3993ca20f5eb319659410faeb35e34b7083</citedby><cites>FETCH-LOGICAL-c282t-ad516d86687b79cd2e5123b31f04db3993ca20f5eb319659410faeb35e34b7083</cites><orcidid>0000-0002-4179-7701 ; 0000-0002-4695-6134 ; 0000-0002-7985-0037</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10014017$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/10014017$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/37018645$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Liang, Changcheng</creatorcontrib><creatorcontrib>Zhu, Mingrui</creatorcontrib><creatorcontrib>Wang, Nannan</creatorcontrib><creatorcontrib>Yang, Heng</creatorcontrib><creatorcontrib>Gao, Xinbo</creatorcontrib><title>PMSGAN: Parallel Multistage GANs for Face Image Translation</title><title>IEEE transaction on neural networks and learning systems</title><addtitle>TNNLS</addtitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><description>In this article, we address the face image translation task, which aims to translate a face image of a source domain to a target domain. Although significant progress has been made by recent studies, face image translation is still a challenging task because it has more strict requirements for texture details: even a few artifacts will greatly affect the impression of generated face images. Targeting to synthesize high-quality face images with admirable visual appearance, we revisit the coarse-to-fine strategy and propose a novel p arallel m ultistage architecture on the basis of g enerative a dversarial n etworks (PMSGAN). More specifically, PMSGAN progressively learns the translation function by disintegrating the general synthesis process into multiple parallel stages that take images with gradually decreasing spatial resolution as inputs. To prompt the information exchange between various stages, a cross-stage atrous spatial pyramid (CSASP) structure is specially designed to receive and fuse the contextual information from other stages. At the end of the parallel model, we introduce a novel attention-based module that leverages multistage decoded outputs as in situ supervised attention to refine the final activations and yield the target image. Extensive experiments on several face image translation benchmarks show that PMSGAN performs considerably better than state-of-the-art approaches.</description><subject>Atrous spatial pyramid</subject><subject>Benchmarks</subject><subject>Decoding</subject><subject>Disintegration</subject><subject>Face</subject><subject>Face detection</subject><subject>face image translation</subject><subject>Generative adversarial networks</subject><subject>Image analysis</subject><subject>Image quality</subject><subject>Information processing</subject><subject>parallel multistage</subject><subject>Spatial discrimination</subject><subject>Spatial resolution</subject><subject>Training</subject><subject>Translation</subject><subject>Visual tasks</subject><issn>2162-237X</issn><issn>2162-2388</issn><issn>2162-2388</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpdkEFLwzAUx4Mobsx9AREpePHS-ZI0SaOnMdwcbHOwCd5C2qbS0bUzaQ9-ezM3h5hL8l5-78_jh9A1hgHGIB_Wi8VsNSBAyIASSoGwM9QlmJOQ0Dg-P73Fewf1nduAPxwYj-Ql6lABOOYR66Kn5Xw1GS4eg6W2uixNGczbsilcoz9M4D9ckNc2GOvUBNPtvre2unKlboq6ukIXuS6d6R_vHnobP69HL-HsdTIdDWdhSmLShDpjmGcx57FIhEwzYhgmNKE4hyhLqJQ01QRyZnxLciYjDLn2BTM0SgTEtIfuD7k7W3-2xjVqW7jUlKWuTN06RYQUOBIAzKN3_9BN3drKb6coiBgkk5R7ihyo1NbOWZOrnS222n4pDGpvV_3YVXu76mjXD90eo9tka7LTyK9LD9wcgMIY8ycRcARY0G-HKXp7</recordid><startdate>20240701</startdate><enddate>20240701</enddate><creator>Liang, Changcheng</creator><creator>Zhu, Mingrui</creator><creator>Wang, Nannan</creator><creator>Yang, Heng</creator><creator>Gao, Xinbo</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QF</scope><scope>7QO</scope><scope>7QP</scope><scope>7QQ</scope><scope>7QR</scope><scope>7SC</scope><scope>7SE</scope><scope>7SP</scope><scope>7SR</scope><scope>7TA</scope><scope>7TB</scope><scope>7TK</scope><scope>7U5</scope><scope>8BQ</scope><scope>8FD</scope><scope>F28</scope><scope>FR3</scope><scope>H8D</scope><scope>JG9</scope><scope>JQ2</scope><scope>KR7</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>P64</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-4179-7701</orcidid><orcidid>https://orcid.org/0000-0002-4695-6134</orcidid><orcidid>https://orcid.org/0000-0002-7985-0037</orcidid></search><sort><creationdate>20240701</creationdate><title>PMSGAN: Parallel Multistage GANs for Face Image Translation</title><author>Liang, Changcheng ; Zhu, Mingrui ; Wang, Nannan ; Yang, Heng ; Gao, Xinbo</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c282t-ad516d86687b79cd2e5123b31f04db3993ca20f5eb319659410faeb35e34b7083</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Atrous spatial pyramid</topic><topic>Benchmarks</topic><topic>Decoding</topic><topic>Disintegration</topic><topic>Face</topic><topic>Face detection</topic><topic>face image translation</topic><topic>Generative adversarial networks</topic><topic>Image analysis</topic><topic>Image quality</topic><topic>Information processing</topic><topic>parallel multistage</topic><topic>Spatial discrimination</topic><topic>Spatial resolution</topic><topic>Training</topic><topic>Translation</topic><topic>Visual tasks</topic><toplevel>online_resources</toplevel><creatorcontrib>Liang, Changcheng</creatorcontrib><creatorcontrib>Zhu, Mingrui</creatorcontrib><creatorcontrib>Wang, Nannan</creatorcontrib><creatorcontrib>Yang, Heng</creatorcontrib><creatorcontrib>Gao, Xinbo</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Aluminium Industry Abstracts</collection><collection>Biotechnology Research Abstracts</collection><collection>Calcium &amp; Calcified Tissue Abstracts</collection><collection>Ceramic Abstracts</collection><collection>Chemoreception Abstracts</collection><collection>Computer and Information Systems Abstracts</collection><collection>Corrosion Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>Materials Business File</collection><collection>Mechanical &amp; Transportation Engineering Abstracts</collection><collection>Neurosciences Abstracts</collection><collection>Solid State and Superconductivity Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>ANTE: Abstracts in New Technology &amp; Engineering</collection><collection>Engineering Research Database</collection><collection>Aerospace Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Civil Engineering Abstracts</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transaction on neural networks and learning systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Liang, Changcheng</au><au>Zhu, Mingrui</au><au>Wang, Nannan</au><au>Yang, Heng</au><au>Gao, Xinbo</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>PMSGAN: Parallel Multistage GANs for Face Image Translation</atitle><jtitle>IEEE transaction on neural networks and learning systems</jtitle><stitle>TNNLS</stitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><date>2024-07-01</date><risdate>2024</risdate><volume>35</volume><issue>7</issue><spage>9352</spage><epage>9365</epage><pages>9352-9365</pages><issn>2162-237X</issn><issn>2162-2388</issn><eissn>2162-2388</eissn><coden>ITNNAL</coden><abstract>In this article, we address the face image translation task, which aims to translate a face image of a source domain to a target domain. Although significant progress has been made by recent studies, face image translation is still a challenging task because it has more strict requirements for texture details: even a few artifacts will greatly affect the impression of generated face images. Targeting to synthesize high-quality face images with admirable visual appearance, we revisit the coarse-to-fine strategy and propose a novel p arallel m ultistage architecture on the basis of g enerative a dversarial n etworks (PMSGAN). More specifically, PMSGAN progressively learns the translation function by disintegrating the general synthesis process into multiple parallel stages that take images with gradually decreasing spatial resolution as inputs. To prompt the information exchange between various stages, a cross-stage atrous spatial pyramid (CSASP) structure is specially designed to receive and fuse the contextual information from other stages. At the end of the parallel model, we introduce a novel attention-based module that leverages multistage decoded outputs as in situ supervised attention to refine the final activations and yield the target image. Extensive experiments on several face image translation benchmarks show that PMSGAN performs considerably better than state-of-the-art approaches.</abstract><cop>United States</cop><pub>IEEE</pub><pmid>37018645</pmid><doi>10.1109/TNNLS.2022.3233025</doi><tpages>14</tpages><orcidid>https://orcid.org/0000-0002-4179-7701</orcidid><orcidid>https://orcid.org/0000-0002-4695-6134</orcidid><orcidid>https://orcid.org/0000-0002-7985-0037</orcidid></addata></record>
fulltext fulltext_linktorsrc
identifier ISSN: 2162-237X
ispartof IEEE transaction on neural networks and learning systems, 2024-07, Vol.35 (7), p.9352-9365
issn 2162-237X
2162-2388
2162-2388
language eng
recordid cdi_crossref_primary_10_1109_TNNLS_2022_3233025
source IEEE Electronic Library (IEL)
subjects Atrous spatial pyramid
Benchmarks
Decoding
Disintegration
Face
Face detection
face image translation
Generative adversarial networks
Image analysis
Image quality
Information processing
parallel multistage
Spatial discrimination
Spatial resolution
Training
Translation
Visual tasks
title PMSGAN: Parallel Multistage GANs for Face Image Translation
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-21T03%3A49%3A12IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=PMSGAN:%20Parallel%20Multistage%20GANs%20for%20Face%20Image%20Translation&rft.jtitle=IEEE%20transaction%20on%20neural%20networks%20and%20learning%20systems&rft.au=Liang,%20Changcheng&rft.date=2024-07-01&rft.volume=35&rft.issue=7&rft.spage=9352&rft.epage=9365&rft.pages=9352-9365&rft.issn=2162-237X&rft.eissn=2162-2388&rft.coden=ITNNAL&rft_id=info:doi/10.1109/TNNLS.2022.3233025&rft_dat=%3Cproquest_RIE%3E2797147005%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3078095936&rft_id=info:pmid/37018645&rft_ieee_id=10014017&rfr_iscdi=true