1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation

Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving. In this report, we present further improvements to the SOTA VIS method, DVIS. First, we introduce a denoising training strategy for th...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Zhang, Tao, Tian, Xingye, Zhou, Yikang, Wu, Yu, Ji, Shunping, Yan, Cilin, Wang, Xuebo, Tao, Xin, Zhang, Yuan, Wan, Pengfei
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Zhang, Tao
Tian, Xingye
Zhou, Yikang
Wu, Yu
Ji, Shunping
Yan, Cilin
Wang, Xuebo
Tao, Xin
Zhang, Yuan
Wan, Pengfei
description Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving. In this report, we present further improvements to the SOTA VIS method, DVIS. First, we introduce a denoising training strategy for the trainable tracker, allowing it to achieve more stable and accurate object tracking in complex and long videos. Additionally, we explore the role of visual foundation models in video instance segmentation. By utilizing a frozen VIT-L model pre-trained by DINO v2, DVIS demonstrates remarkable performance improvements. With these enhancements, our method achieves 57.9 AP and 56.0 AP in the development and test phases, respectively, and ultimately ranked 1st in the VIS track of the 5th LSVOS Challenge. The code will be available at https://github.com/zhang-tao-whu/DVIS.
doi_str_mv 10.48550/arxiv.2308.14392
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2308_14392</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2308_14392</sourcerecordid><originalsourceid>FETCH-LOGICAL-a672-e3ab7a132280c37f46e67111717236724cf85958b11ad0e16f338c9729009fae3</originalsourceid><addsrcrecordid>eNotj71OwzAURr0woMIDMOEXSPC149hmQxE_FZGKlKprdJteN5FcByUGwdtDCtO3fOdIh7EbEHlhtRZ3OH0Nn7lUwuZQKCcv2SvMib8F7Ig3Y_hIwxi5HyeeeuI69bxudpuGVz2GQPFI93w3HGjk6zgnjAtExxPFhAt4xS48hpmu_3fFtk-P2-olqzfP6-qhzrA0MiOFe4OgpLSiU8YXJZUGAAwYqX4PReetdtruAfAgCEqvlO2ckU4I55HUit3-ac817fs0nHD6bpeq9lylfgBRnEVT</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation</title><source>arXiv.org</source><creator>Zhang, Tao ; Tian, Xingye ; Zhou, Yikang ; Wu, Yu ; Ji, Shunping ; Yan, Cilin ; Wang, Xuebo ; Tao, Xin ; Zhang, Yuan ; Wan, Pengfei</creator><creatorcontrib>Zhang, Tao ; Tian, Xingye ; Zhou, Yikang ; Wu, Yu ; Ji, Shunping ; Yan, Cilin ; Wang, Xuebo ; Tao, Xin ; Zhang, Yuan ; Wan, Pengfei</creatorcontrib><description>Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving. In this report, we present further improvements to the SOTA VIS method, DVIS. First, we introduce a denoising training strategy for the trainable tracker, allowing it to achieve more stable and accurate object tracking in complex and long videos. Additionally, we explore the role of visual foundation models in video instance segmentation. By utilizing a frozen VIT-L model pre-trained by DINO v2, DVIS demonstrates remarkable performance improvements. With these enhancements, our method achieves 57.9 AP and 56.0 AP in the development and test phases, respectively, and ultimately ranked 1st in the VIS track of the 5th LSVOS Challenge. The code will be available at https://github.com/zhang-tao-whu/DVIS.</description><identifier>DOI: 10.48550/arxiv.2308.14392</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2023-08</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,777,882</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2308.14392$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2308.14392$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Zhang, Tao</creatorcontrib><creatorcontrib>Tian, Xingye</creatorcontrib><creatorcontrib>Zhou, Yikang</creatorcontrib><creatorcontrib>Wu, Yu</creatorcontrib><creatorcontrib>Ji, Shunping</creatorcontrib><creatorcontrib>Yan, Cilin</creatorcontrib><creatorcontrib>Wang, Xuebo</creatorcontrib><creatorcontrib>Tao, Xin</creatorcontrib><creatorcontrib>Zhang, Yuan</creatorcontrib><creatorcontrib>Wan, Pengfei</creatorcontrib><title>1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation</title><description>Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving. In this report, we present further improvements to the SOTA VIS method, DVIS. First, we introduce a denoising training strategy for the trainable tracker, allowing it to achieve more stable and accurate object tracking in complex and long videos. Additionally, we explore the role of visual foundation models in video instance segmentation. By utilizing a frozen VIT-L model pre-trained by DINO v2, DVIS demonstrates remarkable performance improvements. With these enhancements, our method achieves 57.9 AP and 56.0 AP in the development and test phases, respectively, and ultimately ranked 1st in the VIS track of the 5th LSVOS Challenge. The code will be available at https://github.com/zhang-tao-whu/DVIS.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj71OwzAURr0woMIDMOEXSPC149hmQxE_FZGKlKprdJteN5FcByUGwdtDCtO3fOdIh7EbEHlhtRZ3OH0Nn7lUwuZQKCcv2SvMib8F7Ig3Y_hIwxi5HyeeeuI69bxudpuGVz2GQPFI93w3HGjk6zgnjAtExxPFhAt4xS48hpmu_3fFtk-P2-olqzfP6-qhzrA0MiOFe4OgpLSiU8YXJZUGAAwYqX4PReetdtruAfAgCEqvlO2ckU4I55HUit3-ac817fs0nHD6bpeq9lylfgBRnEVT</recordid><startdate>20230828</startdate><enddate>20230828</enddate><creator>Zhang, Tao</creator><creator>Tian, Xingye</creator><creator>Zhou, Yikang</creator><creator>Wu, Yu</creator><creator>Ji, Shunping</creator><creator>Yan, Cilin</creator><creator>Wang, Xuebo</creator><creator>Tao, Xin</creator><creator>Zhang, Yuan</creator><creator>Wan, Pengfei</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20230828</creationdate><title>1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation</title><author>Zhang, Tao ; Tian, Xingye ; Zhou, Yikang ; Wu, Yu ; Ji, Shunping ; Yan, Cilin ; Wang, Xuebo ; Tao, Xin ; Zhang, Yuan ; Wan, Pengfei</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a672-e3ab7a132280c37f46e67111717236724cf85958b11ad0e16f338c9729009fae3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Zhang, Tao</creatorcontrib><creatorcontrib>Tian, Xingye</creatorcontrib><creatorcontrib>Zhou, Yikang</creatorcontrib><creatorcontrib>Wu, Yu</creatorcontrib><creatorcontrib>Ji, Shunping</creatorcontrib><creatorcontrib>Yan, Cilin</creatorcontrib><creatorcontrib>Wang, Xuebo</creatorcontrib><creatorcontrib>Tao, Xin</creatorcontrib><creatorcontrib>Zhang, Yuan</creatorcontrib><creatorcontrib>Wan, Pengfei</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Zhang, Tao</au><au>Tian, Xingye</au><au>Zhou, Yikang</au><au>Wu, Yu</au><au>Ji, Shunping</au><au>Yan, Cilin</au><au>Wang, Xuebo</au><au>Tao, Xin</au><au>Zhang, Yuan</au><au>Wan, Pengfei</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation</atitle><date>2023-08-28</date><risdate>2023</risdate><abstract>Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving. In this report, we present further improvements to the SOTA VIS method, DVIS. First, we introduce a denoising training strategy for the trainable tracker, allowing it to achieve more stable and accurate object tracking in complex and long videos. Additionally, we explore the role of visual foundation models in video instance segmentation. By utilizing a frozen VIT-L model pre-trained by DINO v2, DVIS demonstrates remarkable performance improvements. With these enhancements, our method achieves 57.9 AP and 56.0 AP in the development and test phases, respectively, and ultimately ranked 1st in the VIS track of the 5th LSVOS Challenge. The code will be available at https://github.com/zhang-tao-whu/DVIS.</abstract><doi>10.48550/arxiv.2308.14392</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2308.14392
ispartof
issn
language eng
recordid cdi_arxiv_primary_2308_14392
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title 1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-20T06%3A18%3A43IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=1st%20Place%20Solution%20for%20the%205th%20LSVOS%20Challenge:%20Video%20Instance%20Segmentation&rft.au=Zhang,%20Tao&rft.date=2023-08-28&rft_id=info:doi/10.48550/arxiv.2308.14392&rft_dat=%3Carxiv_GOX%3E2308_14392%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true