StableDrag: Stable Dragging for Point-based Image Editing

Point-based image editing has attracted remarkable attention since the emergence of DragGAN. Recently, DragDiffusion further pushes forward the generative quality via adapting this dragging technique to diffusion models. Despite these great success, this dragging scheme exhibits two major drawbacks,...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Cui, Yutao, Zhao, Xiaotong, Zhang, Guozhen, Cao, Shengming, Ma, Kai, Wang, Limin
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Cui, Yutao
Zhao, Xiaotong
Zhang, Guozhen
Cao, Shengming
Ma, Kai
Wang, Limin
description Point-based image editing has attracted remarkable attention since the emergence of DragGAN. Recently, DragDiffusion further pushes forward the generative quality via adapting this dragging technique to diffusion models. Despite these great success, this dragging scheme exhibits two major drawbacks, namely inaccurate point tracking and incomplete motion supervision, which may result in unsatisfactory dragging outcomes. To tackle these issues, we build a stable and precise drag-based editing framework, coined as StableDrag, by designing a discirminative point tracking method and a confidence-based latent enhancement strategy for motion supervision. The former allows us to precisely locate the updated handle points, thereby boosting the stability of long-range manipulation, while the latter is responsible for guaranteeing the optimized latent as high-quality as possible across all the manipulation steps. Thanks to these unique designs, we instantiate two types of image editing models including StableDrag-GAN and StableDrag-Diff, which attains more stable dragging performance, through extensive qualitative experiments and quantitative assessment on DragBench.
doi_str_mv 10.48550/arxiv.2403.04437
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2403_04437</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2403_04437</sourcerecordid><originalsourceid>FETCH-LOGICAL-a677-a3dc881eca4eb853b8fb92e81e914b2f5cbbc36a53acb8764dba0e67aa9beb7e3</originalsourceid><addsrcrecordid>eNotT8kKwjAUzMWDqB_gyfxAa9okTepN3KGgoPfyXvpaAm7EIvr3rqfZYJhhbJiIWFmtxRjCw9_jVAkZC6Wk6bJ83wIeaR6gmfAf5x_R-HPD60vgu4s_txHCjSq-OUFDfFH59p32WaeG440Gf-yxw3JxmK2jYrvazKZFBJkxEcjKWZuQA0VotURbY57S28kThWmtHaKTGWgJDq3JVIUgKDMAORIakj02-tV-t5fX4E8QnuXnQ_n9IF-vQEHd</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>StableDrag: Stable Dragging for Point-based Image Editing</title><source>arXiv.org</source><creator>Cui, Yutao ; Zhao, Xiaotong ; Zhang, Guozhen ; Cao, Shengming ; Ma, Kai ; Wang, Limin</creator><creatorcontrib>Cui, Yutao ; Zhao, Xiaotong ; Zhang, Guozhen ; Cao, Shengming ; Ma, Kai ; Wang, Limin</creatorcontrib><description>Point-based image editing has attracted remarkable attention since the emergence of DragGAN. Recently, DragDiffusion further pushes forward the generative quality via adapting this dragging technique to diffusion models. Despite these great success, this dragging scheme exhibits two major drawbacks, namely inaccurate point tracking and incomplete motion supervision, which may result in unsatisfactory dragging outcomes. To tackle these issues, we build a stable and precise drag-based editing framework, coined as StableDrag, by designing a discirminative point tracking method and a confidence-based latent enhancement strategy for motion supervision. The former allows us to precisely locate the updated handle points, thereby boosting the stability of long-range manipulation, while the latter is responsible for guaranteeing the optimized latent as high-quality as possible across all the manipulation steps. Thanks to these unique designs, we instantiate two types of image editing models including StableDrag-GAN and StableDrag-Diff, which attains more stable dragging performance, through extensive qualitative experiments and quantitative assessment on DragBench.</description><identifier>DOI: 10.48550/arxiv.2403.04437</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2024-03</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,781,886</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2403.04437$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2403.04437$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Cui, Yutao</creatorcontrib><creatorcontrib>Zhao, Xiaotong</creatorcontrib><creatorcontrib>Zhang, Guozhen</creatorcontrib><creatorcontrib>Cao, Shengming</creatorcontrib><creatorcontrib>Ma, Kai</creatorcontrib><creatorcontrib>Wang, Limin</creatorcontrib><title>StableDrag: Stable Dragging for Point-based Image Editing</title><description>Point-based image editing has attracted remarkable attention since the emergence of DragGAN. Recently, DragDiffusion further pushes forward the generative quality via adapting this dragging technique to diffusion models. Despite these great success, this dragging scheme exhibits two major drawbacks, namely inaccurate point tracking and incomplete motion supervision, which may result in unsatisfactory dragging outcomes. To tackle these issues, we build a stable and precise drag-based editing framework, coined as StableDrag, by designing a discirminative point tracking method and a confidence-based latent enhancement strategy for motion supervision. The former allows us to precisely locate the updated handle points, thereby boosting the stability of long-range manipulation, while the latter is responsible for guaranteeing the optimized latent as high-quality as possible across all the manipulation steps. Thanks to these unique designs, we instantiate two types of image editing models including StableDrag-GAN and StableDrag-Diff, which attains more stable dragging performance, through extensive qualitative experiments and quantitative assessment on DragBench.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotT8kKwjAUzMWDqB_gyfxAa9okTepN3KGgoPfyXvpaAm7EIvr3rqfZYJhhbJiIWFmtxRjCw9_jVAkZC6Wk6bJ83wIeaR6gmfAf5x_R-HPD60vgu4s_txHCjSq-OUFDfFH59p32WaeG440Gf-yxw3JxmK2jYrvazKZFBJkxEcjKWZuQA0VotURbY57S28kThWmtHaKTGWgJDq3JVIUgKDMAORIakj02-tV-t5fX4E8QnuXnQ_n9IF-vQEHd</recordid><startdate>20240307</startdate><enddate>20240307</enddate><creator>Cui, Yutao</creator><creator>Zhao, Xiaotong</creator><creator>Zhang, Guozhen</creator><creator>Cao, Shengming</creator><creator>Ma, Kai</creator><creator>Wang, Limin</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240307</creationdate><title>StableDrag: Stable Dragging for Point-based Image Editing</title><author>Cui, Yutao ; Zhao, Xiaotong ; Zhang, Guozhen ; Cao, Shengming ; Ma, Kai ; Wang, Limin</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a677-a3dc881eca4eb853b8fb92e81e914b2f5cbbc36a53acb8764dba0e67aa9beb7e3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Cui, Yutao</creatorcontrib><creatorcontrib>Zhao, Xiaotong</creatorcontrib><creatorcontrib>Zhang, Guozhen</creatorcontrib><creatorcontrib>Cao, Shengming</creatorcontrib><creatorcontrib>Ma, Kai</creatorcontrib><creatorcontrib>Wang, Limin</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Cui, Yutao</au><au>Zhao, Xiaotong</au><au>Zhang, Guozhen</au><au>Cao, Shengming</au><au>Ma, Kai</au><au>Wang, Limin</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>StableDrag: Stable Dragging for Point-based Image Editing</atitle><date>2024-03-07</date><risdate>2024</risdate><abstract>Point-based image editing has attracted remarkable attention since the emergence of DragGAN. Recently, DragDiffusion further pushes forward the generative quality via adapting this dragging technique to diffusion models. Despite these great success, this dragging scheme exhibits two major drawbacks, namely inaccurate point tracking and incomplete motion supervision, which may result in unsatisfactory dragging outcomes. To tackle these issues, we build a stable and precise drag-based editing framework, coined as StableDrag, by designing a discirminative point tracking method and a confidence-based latent enhancement strategy for motion supervision. The former allows us to precisely locate the updated handle points, thereby boosting the stability of long-range manipulation, while the latter is responsible for guaranteeing the optimized latent as high-quality as possible across all the manipulation steps. Thanks to these unique designs, we instantiate two types of image editing models including StableDrag-GAN and StableDrag-Diff, which attains more stable dragging performance, through extensive qualitative experiments and quantitative assessment on DragBench.</abstract><doi>10.48550/arxiv.2403.04437</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2403.04437
ispartof
issn
language eng
recordid cdi_arxiv_primary_2403_04437
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title StableDrag: Stable Dragging for Point-based Image Editing
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-17T17%3A52%3A03IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=StableDrag:%20Stable%20Dragging%20for%20Point-based%20Image%20Editing&rft.au=Cui,%20Yutao&rft.date=2024-03-07&rft_id=info:doi/10.48550/arxiv.2403.04437&rft_dat=%3Carxiv_GOX%3E2403_04437%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true