Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression

Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression

Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this p...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Guo, Jia, Yu, Jinke, Lattas, Alexandros, Deng, Jiankang
Format: Artikel
Sprache:eng
Schlagworte:
Computer Science - Computer Vision and Pattern Recognition
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Guo, Jia
Yu, Jinke
Lattas, Alexandros
Deng, Jiankang
description Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this paper, we propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by the PnP solver to represent perspective projection. Our approach achieves 1st place on the leader-board of the ECCV 2022 WCPA challenge and our model is visually robust under different identities, expressions and poses. The training code and models are released to facilitate future research.
doi_str_mv 10.48550/arxiv.2208.07142
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2208_07142</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2208_07142</sourcerecordid><originalsourceid>FETCH-LOGICAL-a672-d9ce6cc64cf10dfc5ce2dea4c74c4d39dbfc8cd0287d0a6db0e005469be616653</originalsourceid><addsrcrecordid>eNotj8FOwzAQRH3hgFo-gBP-gaQbx3aSY1W1FBRUhHqPnPUaIqhT2WlF_x5TuMxoDjOax9h9AbmslYKFCd_DORcC6hyqQopbtnulEI-E03Am_kY4-jiFU4qj56Pj29PBeL4xSJH3F_48Dn7iLxQ_uPGWt0kOJnym4nugGFNpzm6c-Yp09-8ztt-s96tt1u4en1bLNjO6EpltkDSilugKsA4VkrBkJFYSpS0b2zus0YKoKwtG2x4IQEnd9KQLrVU5Yw9_s1ei7hiG9OPS_ZJ1V7LyB2AYSlw</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression</title><source>arXiv.org</source><creator>Guo, Jia ; Yu, Jinke ; Lattas, Alexandros ; Deng, Jiankang</creator><creatorcontrib>Guo, Jia ; Yu, Jinke ; Lattas, Alexandros ; Deng, Jiankang</creatorcontrib><description>Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this paper, we propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by the PnP solver to represent perspective projection. Our approach achieves 1st place on the leader-board of the ECCV 2022 WCPA challenge and our model is visually robust under different identities, expressions and poses. The training code and models are released to facilitate future research.</description><identifier>DOI: 10.48550/arxiv.2208.07142</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2022-08</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2208.07142$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2208.07142$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Guo, Jia</creatorcontrib><creatorcontrib>Yu, Jinke</creatorcontrib><creatorcontrib>Lattas, Alexandros</creatorcontrib><creatorcontrib>Deng, Jiankang</creatorcontrib><title>Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression</title><description>Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this paper, we propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by the PnP solver to represent perspective projection. Our approach achieves 1st place on the leader-board of the ECCV 2022 WCPA challenge and our model is visually robust under different identities, expressions and poses. The training code and models are released to facilitate future research.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj8FOwzAQRH3hgFo-gBP-gaQbx3aSY1W1FBRUhHqPnPUaIqhT2WlF_x5TuMxoDjOax9h9AbmslYKFCd_DORcC6hyqQopbtnulEI-E03Am_kY4-jiFU4qj56Pj29PBeL4xSJH3F_48Dn7iLxQ_uPGWt0kOJnym4nugGFNpzm6c-Yp09-8ztt-s96tt1u4en1bLNjO6EpltkDSilugKsA4VkrBkJFYSpS0b2zus0YKoKwtG2x4IQEnd9KQLrVU5Yw9_s1ei7hiG9OPS_ZJ1V7LyB2AYSlw</recordid><startdate>20220815</startdate><enddate>20220815</enddate><creator>Guo, Jia</creator><creator>Yu, Jinke</creator><creator>Lattas, Alexandros</creator><creator>Deng, Jiankang</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20220815</creationdate><title>Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression</title><author>Guo, Jia ; Yu, Jinke ; Lattas, Alexandros ; Deng, Jiankang</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a672-d9ce6cc64cf10dfc5ce2dea4c74c4d39dbfc8cd0287d0a6db0e005469be616653</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Guo, Jia</creatorcontrib><creatorcontrib>Yu, Jinke</creatorcontrib><creatorcontrib>Lattas, Alexandros</creatorcontrib><creatorcontrib>Deng, Jiankang</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Guo, Jia</au><au>Yu, Jinke</au><au>Lattas, Alexandros</au><au>Deng, Jiankang</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression</atitle><date>2022-08-15</date><risdate>2022</risdate><abstract>Even though 3D face reconstruction has achieved impressive progress, most orthogonal projection-based face reconstruction methods can not achieve accurate and consistent reconstruction results when the face is very close to the camera due to the distortion under the perspective projection. In this paper, we propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane to address the problem of perspective 3D face reconstruction. Based on the predicted 3D vertices and 2D landmarks, the 6DoF (6 Degrees of Freedom) face pose can be easily estimated by the PnP solver to represent perspective projection. Our approach achieves 1st place on the leader-board of the ECCV 2022 WCPA challenge and our model is visually robust under different identities, expressions and poses. The training code and models are released to facilitate future research.</abstract><doi>10.48550/arxiv.2208.07142</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2208.07142
ispartof
issn
language eng
recordid cdi_arxiv_primary_2208_07142
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T09%3A54%3A15IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Perspective%20Reconstruction%20of%20Human%20Faces%20by%20Joint%20Mesh%20and%20Landmark%20Regression&rft.au=Guo,%20Jia&rft.date=2022-08-15&rft_id=info:doi/10.48550/arxiv.2208.07142&rft_dat=%3Carxiv_GOX%3E2208_07142%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true