NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs

NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs

A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target ge...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Fischer, Michael, Li, Zhengqin, Nguyen-Phuoc, Thu, Bozic, Aljaz, Dong, Zhao, Marshall, Carl, Ritschel, Tobias
Format: Artikel
Sprache:eng
Schlagworte:
Computer Science - Computer Vision and Pattern Recognition
Computer Science - Graphics
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Fischer, Michael
Li, Zhengqin
Nguyen-Phuoc, Thu
Bozic, Aljaz
Dong, Zhao
Marshall, Carl
Ritschel, Tobias
description A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic image analogies from 2D images to NeRFs. We leverage correspondence transfer along semantic affinity that is driven by semantic features from large, pre-trained 2D image models to achieve multi-view consistent appearance transfer. Our method allows exploring the mix-and-match product space of 3D geometry and appearance. We show that our method outperforms traditional stylization-based methods and that a large majority of users prefer our method over several typical baselines.
doi_str_mv 10.48550/arxiv.2402.08622
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2402_08622</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2402_08622</sourcerecordid><originalsourceid>FETCH-LOGICAL-a672-c58cab6831599b72cf95a4b4f61ff969250847b8848d631d56dea700a0a8a0af3</originalsourceid><addsrcrecordid>eNotz8uKwjAYBeBsZjE4PsCszAu0pmmS_pldR9QRREGK2_KnTSRQLyRV9O0dL4vDWZ0DHyHfGUsFSMnGGK7-knLBeMpAcf5JypXdzGh5wO648zb-0OkV96fOJr8YbUu3Pp6xo2XfB2_OvaVVwEN0NlB3DPSxjV_kw2EX7fDdA1LNptXkL1mu54tJuUxQFTxpJDRoFOSZ1NoUvHFaojDCqcw5rTSXDERhAAS0Ks9aqVqLBWPIEP7j8gEZvW6fhvoU_B7DrX5Y6qclvwOWnELf</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs</title><source>arXiv.org</source><creator>Fischer, Michael ; Li, Zhengqin ; Nguyen-Phuoc, Thu ; Bozic, Aljaz ; Dong, Zhao ; Marshall, Carl ; Ritschel, Tobias</creator><creatorcontrib>Fischer, Michael ; Li, Zhengqin ; Nguyen-Phuoc, Thu ; Bozic, Aljaz ; Dong, Zhao ; Marshall, Carl ; Ritschel, Tobias</creatorcontrib><description>A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic image analogies from 2D images to NeRFs. We leverage correspondence transfer along semantic affinity that is driven by semantic features from large, pre-trained 2D image models to achieve multi-view consistent appearance transfer. Our method allows exploring the mix-and-match product space of 3D geometry and appearance. We show that our method outperforms traditional stylization-based methods and that a large majority of users prefer our method over several typical baselines.</description><identifier>DOI: 10.48550/arxiv.2402.08622</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition ; Computer Science - Graphics</subject><creationdate>2024-02</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,776,881</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2402.08622$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2402.08622$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Fischer, Michael</creatorcontrib><creatorcontrib>Li, Zhengqin</creatorcontrib><creatorcontrib>Nguyen-Phuoc, Thu</creatorcontrib><creatorcontrib>Bozic, Aljaz</creatorcontrib><creatorcontrib>Dong, Zhao</creatorcontrib><creatorcontrib>Marshall, Carl</creatorcontrib><creatorcontrib>Ritschel, Tobias</creatorcontrib><title>NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs</title><description>A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic image analogies from 2D images to NeRFs. We leverage correspondence transfer along semantic affinity that is driven by semantic features from large, pre-trained 2D image models to achieve multi-view consistent appearance transfer. Our method allows exploring the mix-and-match product space of 3D geometry and appearance. We show that our method outperforms traditional stylization-based methods and that a large majority of users prefer our method over several typical baselines.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><subject>Computer Science - Graphics</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotz8uKwjAYBeBsZjE4PsCszAu0pmmS_pldR9QRREGK2_KnTSRQLyRV9O0dL4vDWZ0DHyHfGUsFSMnGGK7-knLBeMpAcf5JypXdzGh5wO648zb-0OkV96fOJr8YbUu3Pp6xo2XfB2_OvaVVwEN0NlB3DPSxjV_kw2EX7fDdA1LNptXkL1mu54tJuUxQFTxpJDRoFOSZ1NoUvHFaojDCqcw5rTSXDERhAAS0Ks9aqVqLBWPIEP7j8gEZvW6fhvoU_B7DrX5Y6qclvwOWnELf</recordid><startdate>20240213</startdate><enddate>20240213</enddate><creator>Fischer, Michael</creator><creator>Li, Zhengqin</creator><creator>Nguyen-Phuoc, Thu</creator><creator>Bozic, Aljaz</creator><creator>Dong, Zhao</creator><creator>Marshall, Carl</creator><creator>Ritschel, Tobias</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240213</creationdate><title>NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs</title><author>Fischer, Michael ; Li, Zhengqin ; Nguyen-Phuoc, Thu ; Bozic, Aljaz ; Dong, Zhao ; Marshall, Carl ; Ritschel, Tobias</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a672-c58cab6831599b72cf95a4b4f61ff969250847b8848d631d56dea700a0a8a0af3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><topic>Computer Science - Graphics</topic><toplevel>online_resources</toplevel><creatorcontrib>Fischer, Michael</creatorcontrib><creatorcontrib>Li, Zhengqin</creatorcontrib><creatorcontrib>Nguyen-Phuoc, Thu</creatorcontrib><creatorcontrib>Bozic, Aljaz</creatorcontrib><creatorcontrib>Dong, Zhao</creatorcontrib><creatorcontrib>Marshall, Carl</creatorcontrib><creatorcontrib>Ritschel, Tobias</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Fischer, Michael</au><au>Li, Zhengqin</au><au>Nguyen-Phuoc, Thu</au><au>Bozic, Aljaz</au><au>Dong, Zhao</au><au>Marshall, Carl</au><au>Ritschel, Tobias</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs</atitle><date>2024-02-13</date><risdate>2024</risdate><abstract>A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic image analogies from 2D images to NeRFs. We leverage correspondence transfer along semantic affinity that is driven by semantic features from large, pre-trained 2D image models to achieve multi-view consistent appearance transfer. Our method allows exploring the mix-and-match product space of 3D geometry and appearance. We show that our method outperforms traditional stylization-based methods and that a large majority of users prefer our method over several typical baselines.</abstract><doi>10.48550/arxiv.2402.08622</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2402.08622
ispartof
issn
language eng
recordid cdi_arxiv_primary_2402_08622
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
Computer Science - Graphics
title NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-02T21%3A06%3A35IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=NeRF%20Analogies:%20Example-Based%20Visual%20Attribute%20Transfer%20for%20NeRFs&rft.au=Fischer,%20Michael&rft.date=2024-02-13&rft_id=info:doi/10.48550/arxiv.2402.08622&rft_dat=%3Carxiv_GOX%3E2402_08622%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true