Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions
We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently int...
Gespeichert in:
Veröffentlicht in: | IEEE access 2023-01, Vol.11, p.1-1 |
---|---|
Hauptverfasser: | , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 1 |
---|---|
container_issue | |
container_start_page | 1 |
container_title | IEEE access |
container_volume | 11 |
creator | Burkov, Egor Rakhimov, Ruslan Safin, Aleksandr Burnaev, Evgeny Lempitsky, Victor |
description | We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results. |
doi_str_mv | 10.1109/ACCESS.2023.3309412 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2862609825</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>10233007</ieee_id><doaj_id>oai_doaj_org_article_6ebc59c628a6465dae4803e736f52dbf</doaj_id><sourcerecordid>2862609825</sourcerecordid><originalsourceid>FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</originalsourceid><addsrcrecordid>eNpNkVtrGzEQhZfSQkOSX9A-CPq8ri4rrdS34MaJIZeC22cxqx25MmvL1WoJ-feRuyFkXjQcvnNGcKrqC6MLxqj5frVcXm82C065WAhBTcP4h-qMM2VqIYX6-G7_XF2O446W0UWS7Vn1cD8NOdQPOG1-EPGT3CL05FdMOUHII_Ep7skmHLYDkvUetkieQv5LCp5gKMpxCC5kspoOLod4GC-qTx6GES9f3_Pqz-r69_K2vnu8WS-v7mrXUJNrbySDHjoAzjolBNPGCN4yplyjW82copoLySSC59Rr7Tug6AvWIVW9F-fVes7tI-zsMYU9pGcbIdj_QkxbCykHN6BV2DlpnOIaVKNkD9hoKrAVykved6esb3PWMcV_E47Z7uKUDuX7lmvFFTWay0KJmXIpjmNC_3aVUXvqwc492FMP9rWH4vo6uwIivnPwAtBWvADKnYIH</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2862609825</pqid></control><display><type>article</type><title>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</title><source>IEEE Open Access Journals</source><source>DOAJ Directory of Open Access Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><creator>Burkov, Egor ; Rakhimov, Ruslan ; Safin, Aleksandr ; Burnaev, Evgeny ; Lempitsky, Victor</creator><creatorcontrib>Burkov, Egor ; Rakhimov, Ruslan ; Safin, Aleksandr ; Burnaev, Evgeny ; Lempitsky, Victor</creatorcontrib><description>We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.</description><identifier>ISSN: 2169-3536</identifier><identifier>EISSN: 2169-3536</identifier><identifier>DOI: 10.1109/ACCESS.2023.3309412</identifier><identifier>CODEN: IAECCG</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>3D portraits ; 3D reconstruction ; Algorithms ; few-shot ; Finite element method ; Head ; head reconstruction ; Image reconstruction ; meta-learning ; Metalearning ; Neural engineering ; neural implicit functions ; Neural networks ; Solid modeling ; Three-dimensional displays ; Training</subject><ispartof>IEEE access, 2023-01, Vol.11, p.1-1</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</citedby><cites>FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</cites><orcidid>0000-0002-6763-3510 ; 0000-0001-8424-0690 ; 0000-0003-2072-8093 ; 0000-0003-4118-710X ; 0000-0002-5453-1101</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/10233007$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>314,776,780,860,2095,27612,27903,27904,54912</link.rule.ids></links><search><creatorcontrib>Burkov, Egor</creatorcontrib><creatorcontrib>Rakhimov, Ruslan</creatorcontrib><creatorcontrib>Safin, Aleksandr</creatorcontrib><creatorcontrib>Burnaev, Evgeny</creatorcontrib><creatorcontrib>Lempitsky, Victor</creatorcontrib><title>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</title><title>IEEE access</title><addtitle>Access</addtitle><description>We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.</description><subject>3D portraits</subject><subject>3D reconstruction</subject><subject>Algorithms</subject><subject>few-shot</subject><subject>Finite element method</subject><subject>Head</subject><subject>head reconstruction</subject><subject>Image reconstruction</subject><subject>meta-learning</subject><subject>Metalearning</subject><subject>Neural engineering</subject><subject>neural implicit functions</subject><subject>Neural networks</subject><subject>Solid modeling</subject><subject>Three-dimensional displays</subject><subject>Training</subject><issn>2169-3536</issn><issn>2169-3536</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>DOA</sourceid><recordid>eNpNkVtrGzEQhZfSQkOSX9A-CPq8ri4rrdS34MaJIZeC22cxqx25MmvL1WoJ-feRuyFkXjQcvnNGcKrqC6MLxqj5frVcXm82C065WAhBTcP4h-qMM2VqIYX6-G7_XF2O446W0UWS7Vn1cD8NOdQPOG1-EPGT3CL05FdMOUHII_Ep7skmHLYDkvUetkieQv5LCp5gKMpxCC5kspoOLod4GC-qTx6GES9f3_Pqz-r69_K2vnu8WS-v7mrXUJNrbySDHjoAzjolBNPGCN4yplyjW82copoLySSC59Rr7Tug6AvWIVW9F-fVes7tI-zsMYU9pGcbIdj_QkxbCykHN6BV2DlpnOIaVKNkD9hoKrAVykved6esb3PWMcV_E47Z7uKUDuX7lmvFFTWay0KJmXIpjmNC_3aVUXvqwc492FMP9rWH4vo6uwIivnPwAtBWvADKnYIH</recordid><startdate>20230101</startdate><enddate>20230101</enddate><creator>Burkov, Egor</creator><creator>Rakhimov, Ruslan</creator><creator>Safin, Aleksandr</creator><creator>Burnaev, Evgeny</creator><creator>Lempitsky, Victor</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7SR</scope><scope>8BQ</scope><scope>8FD</scope><scope>JG9</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0002-6763-3510</orcidid><orcidid>https://orcid.org/0000-0001-8424-0690</orcidid><orcidid>https://orcid.org/0000-0003-2072-8093</orcidid><orcidid>https://orcid.org/0000-0003-4118-710X</orcidid><orcidid>https://orcid.org/0000-0002-5453-1101</orcidid></search><sort><creationdate>20230101</creationdate><title>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</title><author>Burkov, Egor ; Rakhimov, Ruslan ; Safin, Aleksandr ; Burnaev, Evgeny ; Lempitsky, Victor</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c409t-f951adabaa21b6331899327116c48781c60823515eaf20f88fba0ef189be06df3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>3D portraits</topic><topic>3D reconstruction</topic><topic>Algorithms</topic><topic>few-shot</topic><topic>Finite element method</topic><topic>Head</topic><topic>head reconstruction</topic><topic>Image reconstruction</topic><topic>meta-learning</topic><topic>Metalearning</topic><topic>Neural engineering</topic><topic>neural implicit functions</topic><topic>Neural networks</topic><topic>Solid modeling</topic><topic>Three-dimensional displays</topic><topic>Training</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Burkov, Egor</creatorcontrib><creatorcontrib>Rakhimov, Ruslan</creatorcontrib><creatorcontrib>Safin, Aleksandr</creatorcontrib><creatorcontrib>Burnaev, Evgeny</creatorcontrib><creatorcontrib>Lempitsky, Victor</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>IEEE access</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Burkov, Egor</au><au>Rakhimov, Ruslan</au><au>Safin, Aleksandr</au><au>Burnaev, Evgeny</au><au>Lempitsky, Victor</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions</atitle><jtitle>IEEE access</jtitle><stitle>Access</stitle><date>2023-01-01</date><risdate>2023</risdate><volume>11</volume><spage>1</spage><epage>1</epage><pages>1-1</pages><issn>2169-3536</issn><eissn>2169-3536</eissn><coden>IAECCG</coden><abstract>We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation-neural implicit functions-which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/ACCESS.2023.3309412</doi><tpages>1</tpages><orcidid>https://orcid.org/0000-0002-6763-3510</orcidid><orcidid>https://orcid.org/0000-0001-8424-0690</orcidid><orcidid>https://orcid.org/0000-0003-2072-8093</orcidid><orcidid>https://orcid.org/0000-0003-4118-710X</orcidid><orcidid>https://orcid.org/0000-0002-5453-1101</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 2169-3536 |
ispartof | IEEE access, 2023-01, Vol.11, p.1-1 |
issn | 2169-3536 2169-3536 |
language | eng |
recordid | cdi_proquest_journals_2862609825 |
source | IEEE Open Access Journals; DOAJ Directory of Open Access Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals |
subjects | 3D portraits 3D reconstruction Algorithms few-shot Finite element method Head head reconstruction Image reconstruction meta-learning Metalearning Neural engineering neural implicit functions Neural networks Solid modeling Three-dimensional displays Training |
title | Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-23T04%3A00%3A52IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Multi-NeuS:%203D%20Head%20Portraits%20from%20Single%20Image%20with%20Neural%20Implicit%20Functions&rft.jtitle=IEEE%20access&rft.au=Burkov,%20Egor&rft.date=2023-01-01&rft.volume=11&rft.spage=1&rft.epage=1&rft.pages=1-1&rft.issn=2169-3536&rft.eissn=2169-3536&rft.coden=IAECCG&rft_id=info:doi/10.1109/ACCESS.2023.3309412&rft_dat=%3Cproquest_cross%3E2862609825%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2862609825&rft_id=info:pmid/&rft_ieee_id=10233007&rft_doaj_id=oai_doaj_org_article_6ebc59c628a6465dae4803e736f52dbf&rfr_iscdi=true |