Real-Time High-Resolution Background Matting

Real-Time High-Resolution Background Matting

We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Lin, Shanchuan, Ryabtsev, Andrey, Sengupta, Soumyadip, Curless, Brian, Seitz, Steve, Kemelmacher-Shlizerman, Ira
Format: Artikel
Sprache:eng
Schlagworte:
Computer Science - Computer Vision and Pattern Recognition
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Lin, Shanchuan
Ryabtsev, Andrey
Sengupta, Soumyadip
Curless, Brian
Seitz, Steve
Kemelmacher-Shlizerman, Ira
description We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the foreground layer. The main challenge is to compute a high-quality alpha matte, preserving strand-level hair details, while processing high-resolution images in real-time. To achieve this goal, we employ two neural networks; a base network computes a low-resolution result which is refined by a second network operating at high-resolution on selective patches. We introduce two largescale video and image matting datasets: VideoMatte240K and PhotoMatte13K/85. Our approach yields higher quality results compared to the previous state-of-the-art in background matting, while simultaneously yielding a dramatic boost in both speed and resolution.
doi_str_mv 10.48550/arxiv.2012.07810
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2012_07810</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2012_07810</sourcerecordid><originalsourceid>FETCH-LOGICAL-a670-669aecff6e0c2c6b1019c78d2f34a517a0b8c9d054bb09304b2aafe76cb0dde83</originalsourceid><addsrcrecordid>eNotzs2OgjAUQOFuXBidB3AlD2DxtkBblmp0NHFiYtiT2z9sRDCIE-ftjTqrszv5CJkwiFOVZTDH7hF-Yw6MxyAVgyGZHR3WtAgXF21DdaJHd2vrex_aJlqiOVdde29s9IN9H5pqTAYe65v7-u-IFJt1sdrS_eF7t1rsKQoJVIgcnfFeODDcCM2A5UYqy32SYsYkglYmt5ClWkOeQKo5ondSGA3WOpWMyPSzfXPLaxcu2P2VL3b5ZidP9jQ8zw</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Real-Time High-Resolution Background Matting</title><source>arXiv.org</source><creator>Lin, Shanchuan ; Ryabtsev, Andrey ; Sengupta, Soumyadip ; Curless, Brian ; Seitz, Steve ; Kemelmacher-Shlizerman, Ira</creator><creatorcontrib>Lin, Shanchuan ; Ryabtsev, Andrey ; Sengupta, Soumyadip ; Curless, Brian ; Seitz, Steve ; Kemelmacher-Shlizerman, Ira</creatorcontrib><description>We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the foreground layer. The main challenge is to compute a high-quality alpha matte, preserving strand-level hair details, while processing high-resolution images in real-time. To achieve this goal, we employ two neural networks; a base network computes a low-resolution result which is refined by a second network operating at high-resolution on selective patches. We introduce two largescale video and image matting datasets: VideoMatte240K and PhotoMatte13K/85. Our approach yields higher quality results compared to the previous state-of-the-art in background matting, while simultaneously yielding a dramatic boost in both speed and resolution.</description><identifier>DOI: 10.48550/arxiv.2012.07810</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2020-12</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,782,887</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2012.07810$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2012.07810$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Lin, Shanchuan</creatorcontrib><creatorcontrib>Ryabtsev, Andrey</creatorcontrib><creatorcontrib>Sengupta, Soumyadip</creatorcontrib><creatorcontrib>Curless, Brian</creatorcontrib><creatorcontrib>Seitz, Steve</creatorcontrib><creatorcontrib>Kemelmacher-Shlizerman, Ira</creatorcontrib><title>Real-Time High-Resolution Background Matting</title><description>We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the foreground layer. The main challenge is to compute a high-quality alpha matte, preserving strand-level hair details, while processing high-resolution images in real-time. To achieve this goal, we employ two neural networks; a base network computes a low-resolution result which is refined by a second network operating at high-resolution on selective patches. We introduce two largescale video and image matting datasets: VideoMatte240K and PhotoMatte13K/85. Our approach yields higher quality results compared to the previous state-of-the-art in background matting, while simultaneously yielding a dramatic boost in both speed and resolution.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotzs2OgjAUQOFuXBidB3AlD2DxtkBblmp0NHFiYtiT2z9sRDCIE-ftjTqrszv5CJkwiFOVZTDH7hF-Yw6MxyAVgyGZHR3WtAgXF21DdaJHd2vrex_aJlqiOVdde29s9IN9H5pqTAYe65v7-u-IFJt1sdrS_eF7t1rsKQoJVIgcnfFeODDcCM2A5UYqy32SYsYkglYmt5ClWkOeQKo5ondSGA3WOpWMyPSzfXPLaxcu2P2VL3b5ZidP9jQ8zw</recordid><startdate>20201214</startdate><enddate>20201214</enddate><creator>Lin, Shanchuan</creator><creator>Ryabtsev, Andrey</creator><creator>Sengupta, Soumyadip</creator><creator>Curless, Brian</creator><creator>Seitz, Steve</creator><creator>Kemelmacher-Shlizerman, Ira</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20201214</creationdate><title>Real-Time High-Resolution Background Matting</title><author>Lin, Shanchuan ; Ryabtsev, Andrey ; Sengupta, Soumyadip ; Curless, Brian ; Seitz, Steve ; Kemelmacher-Shlizerman, Ira</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a670-669aecff6e0c2c6b1019c78d2f34a517a0b8c9d054bb09304b2aafe76cb0dde83</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Lin, Shanchuan</creatorcontrib><creatorcontrib>Ryabtsev, Andrey</creatorcontrib><creatorcontrib>Sengupta, Soumyadip</creatorcontrib><creatorcontrib>Curless, Brian</creatorcontrib><creatorcontrib>Seitz, Steve</creatorcontrib><creatorcontrib>Kemelmacher-Shlizerman, Ira</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Lin, Shanchuan</au><au>Ryabtsev, Andrey</au><au>Sengupta, Soumyadip</au><au>Curless, Brian</au><au>Seitz, Steve</au><au>Kemelmacher-Shlizerman, Ira</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Real-Time High-Resolution Background Matting</atitle><date>2020-12-14</date><risdate>2020</risdate><abstract>We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional frame of the background is captured and used in recovering the alpha matte and the foreground layer. The main challenge is to compute a high-quality alpha matte, preserving strand-level hair details, while processing high-resolution images in real-time. To achieve this goal, we employ two neural networks; a base network computes a low-resolution result which is refined by a second network operating at high-resolution on selective patches. We introduce two largescale video and image matting datasets: VideoMatte240K and PhotoMatte13K/85. Our approach yields higher quality results compared to the previous state-of-the-art in background matting, while simultaneously yielding a dramatic boost in both speed and resolution.</abstract><doi>10.48550/arxiv.2012.07810</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2012.07810
ispartof
issn
language eng
recordid cdi_arxiv_primary_2012_07810
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title Real-Time High-Resolution Background Matting
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-03T23%3A04%3A18IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Real-Time%20High-Resolution%20Background%20Matting&rft.au=Lin,%20Shanchuan&rft.date=2020-12-14&rft_id=info:doi/10.48550/arxiv.2012.07810&rft_dat=%3Carxiv_GOX%3E2012_07810%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true