Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions

We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently int...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE access 2023-01, Vol.11, p.1-1
Hauptverfasser: Burkov, Egor, Rakhimov, Ruslan, Safin, Aleksandr, Burnaev, Evgeny, Lempitsky, Victor
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 1
container_issue
container_start_page 1
container_title IEEE access
container_volume 11
creator Burkov, Egor
Rakhimov, Ruslan
Safin, Aleksandr
Burnaev, Evgeny
Lempitsky, Victor
description We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.
doi_str_mv 10.1109/ACCESS.2023.3309412
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2862609825</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10233007</ieee_id><doaj_id>oai_doaj_org_article_6ebc59c628a6465dae4803e736f52dbf</doaj_id><sourcerecordid>2862609825</sourcerecordid><originalsourceid>FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</originalsourceid><addsrcrecordid>eNpNkVtrGzEQhZfSQkOSX9A-CPq8ri4rrdS34MaJIZeC22cxqx25MmvL1WoJ-feRuyFkXjQcvnNGcKrqC6MLxqj5frVcXm82C065WAhBTcP4h-qMM2VqIYX6-G7_XF2O446W0UWS7Vn1cD8NOdQPOG1-EPGT3CL05FdMOUHII_Ep7skmHLYDkvUetkieQv5LCp5gKMpxCC5kspoOLod4GC-qTx6GES9f3_Pqz-r69_K2vnu8WS-v7mrXUJNrbySDHjoAzjolBNPGCN4yplyjW82copoLySSC59Rr7Tug6AvWIVW9F-fVes7tI-zsMYU9pGcbIdj_QkxbCykHN6BV2DlpnOIaVKNkD9hoKrAVykved6esb3PWMcV_E47Z7uKUDuX7lmvFFTWay0KJmXIpjmNC_3aVUXvqwc492FMP9rWH4vo6uwIivnPwAtBWvADKnYIH</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2862609825</pqid></control><display><type>article</type><title>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</title><source>IEEE Open Access Journals</source><source>DOAJ Directory of Open Access Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><creator>Burkov, Egor ; Rakhimov, Ruslan ; Safin, Aleksandr ; Burnaev, Evgeny ; Lempitsky, Victor</creator><creatorcontrib>Burkov, Egor ; Rakhimov, Ruslan ; Safin, Aleksandr ; Burnaev, Evgeny ; Lempitsky, Victor</creatorcontrib><description>We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.</description><identifier>ISSN: 2169-3536</identifier><identifier>EISSN: 2169-3536</identifier><identifier>DOI: 10.1109/ACCESS.2023.3309412</identifier><identifier>CODEN: IAECCG</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>3D portraits ; 3D reconstruction ; Algorithms ; few-shot ; Finite element method ; Head ; head reconstruction ; Image reconstruction ; meta-learning ; Metalearning ; Neural engineering ; neural implicit functions ; Neural networks ; Solid modeling ; Three-dimensional displays ; Training</subject><ispartof>IEEE access, 2023-01, Vol.11, p.1-1</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</citedby><cites>FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</cites><orcidid>0000-0002-6763-3510 ; 0000-0001-8424-0690 ; 0000-0003-2072-8093 ; 0000-0003-4118-710X ; 0000-0002-5453-1101</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10233007$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>314,776,780,860,2095,27612,27903,27904,54912</link.rule.ids></links><search><creatorcontrib>Burkov, Egor</creatorcontrib><creatorcontrib>Rakhimov, Ruslan</creatorcontrib><creatorcontrib>Safin, Aleksandr</creatorcontrib><creatorcontrib>Burnaev, Evgeny</creatorcontrib><creatorcontrib>Lempitsky, Victor</creatorcontrib><title>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</title><title>IEEE access</title><addtitle>Access</addtitle><description>We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.</description><subject>3D portraits</subject><subject>3D reconstruction</subject><subject>Algorithms</subject><subject>few-shot</subject><subject>Finite element method</subject><subject>Head</subject><subject>head reconstruction</subject><subject>Image reconstruction</subject><subject>meta-learning</subject><subject>Metalearning</subject><subject>Neural engineering</subject><subject>neural implicit functions</subject><subject>Neural networks</subject><subject>Solid modeling</subject><subject>Three-dimensional displays</subject><subject>Training</subject><issn>2169-3536</issn><issn>2169-3536</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>DOA</sourceid><recordid>eNpNkVtrGzEQhZfSQkOSX9A-CPq8ri4rrdS34MaJIZeC22cxqx25MmvL1WoJ-feRuyFkXjQcvnNGcKrqC6MLxqj5frVcXm82C065WAhBTcP4h-qMM2VqIYX6-G7_XF2O446W0UWS7Vn1cD8NOdQPOG1-EPGT3CL05FdMOUHII_Ep7skmHLYDkvUetkieQv5LCp5gKMpxCC5kspoOLod4GC-qTx6GES9f3_Pqz-r69_K2vnu8WS-v7mrXUJNrbySDHjoAzjolBNPGCN4yplyjW82copoLySSC59Rr7Tug6AvWIVW9F-fVes7tI-zsMYU9pGcbIdj_QkxbCykHN6BV2DlpnOIaVKNkD9hoKrAVykved6esb3PWMcV_E47Z7uKUDuX7lmvFFTWay0KJmXIpjmNC_3aVUXvqwc492FMP9rWH4vo6uwIivnPwAtBWvADKnYIH</recordid><startdate>20230101</startdate><enddate>20230101</enddate><creator>Burkov, Egor</creator><creator>Rakhimov, Ruslan</creator><creator>Safin, Aleksandr</creator><creator>Burnaev, Evgeny</creator><creator>Lempitsky, Victor</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7SR</scope><scope>8BQ</scope><scope>8FD</scope><scope>JG9</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0002-6763-3510</orcidid><orcidid>https://orcid.org/0000-0001-8424-0690</orcidid><orcidid>https://orcid.org/0000-0003-2072-8093</orcidid><orcidid>https://orcid.org/0000-0003-4118-710X</orcidid><orcidid>https://orcid.org/0000-0002-5453-1101</orcidid></search><sort><creationdate>20230101</creationdate><title>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</title><author>Burkov, Egor ; Rakhimov, Ruslan ; Safin, Aleksandr ; Burnaev, Evgeny ; Lempitsky, Victor</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>3D portraits</topic><topic>3D reconstruction</topic><topic>Algorithms</topic><topic>few-shot</topic><topic>Finite element method</topic><topic>Head</topic><topic>head reconstruction</topic><topic>Image reconstruction</topic><topic>meta-learning</topic><topic>Metalearning</topic><topic>Neural engineering</topic><topic>neural implicit functions</topic><topic>Neural networks</topic><topic>Solid modeling</topic><topic>Three-dimensional displays</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Burkov, Egor</creatorcontrib><creatorcontrib>Rakhimov, Ruslan</creatorcontrib><creatorcontrib>Safin, Aleksandr</creatorcontrib><creatorcontrib>Burnaev, Evgeny</creatorcontrib><creatorcontrib>Lempitsky, Victor</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>IEEE access</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Burkov, Egor</au><au>Rakhimov, Ruslan</au><au>Safin, Aleksandr</au><au>Burnaev, Evgeny</au><au>Lempitsky, Victor</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</atitle><jtitle>IEEE access</jtitle><stitle>Access</stitle><date>2023-01-01</date><risdate>2023</risdate><volume>11</volume><spage>1</spage><epage>1</epage><pages>1-1</pages><issn>2169-3536</issn><eissn>2169-3536</eissn><coden>IAECCG</coden><abstract>We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/ACCESS.2023.3309412</doi><tpages>1</tpages><orcidid>https://orcid.org/0000-0002-6763-3510</orcidid><orcidid>https://orcid.org/0000-0001-8424-0690</orcidid><orcidid>https://orcid.org/0000-0003-2072-8093</orcidid><orcidid>https://orcid.org/0000-0003-4118-710X</orcidid><orcidid>https://orcid.org/0000-0002-5453-1101</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2169-3536
ispartof IEEE access, 2023-01, Vol.11, p.1-1
issn 2169-3536
2169-3536
language eng
recordid cdi_proquest_journals_2862609825
source IEEE Open Access Journals; DOAJ Directory of Open Access Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals
subjects 3D portraits
3D reconstruction
Algorithms
few-shot
Finite element method
Head
head reconstruction
Image reconstruction
meta-learning
Metalearning
Neural engineering
neural implicit functions
Neural networks
Solid modeling
Three-dimensional displays
Training
title Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-23T04%3A00%3A52IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Multi-NeuS:%203D%20Head%20Portraits%20from%20Single%20Image%20with%20Neural%20Implicit%20Functions&rft.jtitle=IEEE%20access&rft.au=Burkov,%20Egor&rft.date=2023-01-01&rft.volume=11&rft.spage=1&rft.epage=1&rft.pages=1-1&rft.issn=2169-3536&rft.eissn=2169-3536&rft.coden=IAECCG&rft_id=info:doi/10.1109/ACCESS.2023.3309412&rft_dat=%3Cproquest_cross%3E2862609825%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2862609825&rft_id=info:pmid/&rft_ieee_id=10233007&rft_doaj_id=oai_doaj_org_article_6ebc59c628a6465dae4803e736f52dbf&rfr_iscdi=true