AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results
Video quality assessment (VQA) is a crucial task in the development of video compression standards, as it directly impacts the viewer experience. This paper presents the results of the Compressed Video Quality Assessment challenge, held in conjunction with the Advances in Image Manipulation (AIM) wo...
Gespeichert in:
| Veröffentlicht in: | arXiv.org 2024-10 |
|---|---|
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | arXiv.org |
| container_volume | |
| creator | Smirnov, Maksim Gushchin, Aleksandr Antsiferova, Anastasia Vatolin, Dmitry Timofte, Radu Jia, Ziheng Zhang, Zicheng Sun, Wei Qian, Jiaying Cao, Yuqin Sun, Yinan Zhu, Yuxin Xiongkuo Min Zhai, Guangtao De, Kanjar Luo, Qing Ao-Xiang Zhang Zhang, Peng Lei, Haibo Jiang, Linyan Li, Yaqing Meng, Wenhui Chen, Zhenzhong Cheng, Zhengxue Xiao, Jiahao Xu, Jun He, Chenlong Zheng, Qi Zhu, Ruoxi Li, Min Fan, Yibo Tu, Zhengzhong |
| description | Video quality assessment (VQA) is a crucial task in the development of video compression standards, as it directly impacts the viewer experience. This paper presents the results of the Compressed Video Quality Assessment challenge, held in conjunction with the Advances in Image Manipulation (AIM) workshop at ECCV 2024. The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H.264, HEVC/H.265, AV1, and VVC/H.266) and containing a comprehensive collection of compression artifacts. To measure the methods performance, we employed traditional correlation coefficients between their predictions and subjective scores, which were collected via large-scale crowdsourced pairwise human comparisons. For training purposes, participants were provided with the Compressed Video Quality Assessment Dataset (CVQAD), a previously developed dataset of 1022 videos. Up to 30 participating teams registered for the challenge, while we report the results of 6 teams, which submitted valid final solutions and code for reproducing the results. Moreover, we calculated and present the performance of state-of-the-art VQA methods on the developed dataset, providing a comprehensive benchmark for future research. The dataset, results, and online leaderboard are publicly available at https://challenges.videoprocessing.ai/challenges/compressedvideo-quality-assessment.html. |
| format | Article |
| fullrecord | <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_3096438504</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3096438504</sourcerecordid><originalsourceid>FETCH-proquest_journals_30964385043</originalsourceid><addsrcrecordid>eNqNisEKgkAUAJcgSMp_eNBZ2HbVrJtIUQeDIrrKwr5SWXfNtx76-zz0AZ0GZmbGAiHlJspiIRYsJGo55yLdiiSRAbvk5xIEFzEUtTIG7QvBWShc1w9IhBoejUYH11GZxn8gnxxRh9bvoURfO02grIYb0mg8rdj8qQxh-OOSrY-He3GK-sG9RyRftW4c7JQqyXdpLLOEx_K_6wvZbTy0</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3096438504</pqid></control><display><type>article</type><title>AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results</title><source>Free E- Journals</source><creator>Smirnov, Maksim ; Gushchin, Aleksandr ; Antsiferova, Anastasia ; Vatolin, Dmitry ; Timofte, Radu ; Jia, Ziheng ; Zhang, Zicheng ; Sun, Wei ; Qian, Jiaying ; Cao, Yuqin ; Sun, Yinan ; Zhu, Yuxin ; Xiongkuo Min ; Zhai, Guangtao ; De, Kanjar ; Luo, Qing ; Ao-Xiang Zhang ; Zhang, Peng ; Lei, Haibo ; Jiang, Linyan ; Li, Yaqing ; Meng, Wenhui ; Chen, Zhenzhong ; Cheng, Zhengxue ; Xiao, Jiahao ; Xu, Jun ; He, Chenlong ; Zheng, Qi ; Zhu, Ruoxi ; Li, Min ; Fan, Yibo ; Tu, Zhengzhong</creator><creatorcontrib>Smirnov, Maksim ; Gushchin, Aleksandr ; Antsiferova, Anastasia ; Vatolin, Dmitry ; Timofte, Radu ; Jia, Ziheng ; Zhang, Zicheng ; Sun, Wei ; Qian, Jiaying ; Cao, Yuqin ; Sun, Yinan ; Zhu, Yuxin ; Xiongkuo Min ; Zhai, Guangtao ; De, Kanjar ; Luo, Qing ; Ao-Xiang Zhang ; Zhang, Peng ; Lei, Haibo ; Jiang, Linyan ; Li, Yaqing ; Meng, Wenhui ; Chen, Zhenzhong ; Cheng, Zhengxue ; Xiao, Jiahao ; Xu, Jun ; He, Chenlong ; Zheng, Qi ; Zhu, Ruoxi ; Li, Min ; Fan, Yibo ; Tu, Zhengzhong</creatorcontrib><description>Video quality assessment (VQA) is a crucial task in the development of video compression standards, as it directly impacts the viewer experience. This paper presents the results of the Compressed Video Quality Assessment challenge, held in conjunction with the Advances in Image Manipulation (AIM) workshop at ECCV 2024. The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H.264, HEVC/H.265, AV1, and VVC/H.266) and containing a comprehensive collection of compression artifacts. To measure the methods performance, we employed traditional correlation coefficients between their predictions and subjective scores, which were collected via large-scale crowdsourced pairwise human comparisons. For training purposes, participants were provided with the Compressed Video Quality Assessment Dataset (CVQAD), a previously developed dataset of 1022 videos. Up to 30 participating teams registered for the challenge, while we report the results of 6 teams, which submitted valid final solutions and code for reproducing the results. Moreover, we calculated and present the performance of state-of-the-art VQA methods on the developed dataset, providing a comprehensive benchmark for future research. The dataset, results, and online leaderboard are publicly available at https://challenges.videoprocessing.ai/challenges/compressedvideo-quality-assessment.html.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Codec ; Correlation coefficients ; Datasets ; Image compression ; Image manipulation ; Image quality ; Performance evaluation ; Quality assessment ; Teams ; Video compression</subject><ispartof>arXiv.org, 2024-10</ispartof><rights>2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>776,780</link.rule.ids></links><search><creatorcontrib>Smirnov, Maksim</creatorcontrib><creatorcontrib>Gushchin, Aleksandr</creatorcontrib><creatorcontrib>Antsiferova, Anastasia</creatorcontrib><creatorcontrib>Vatolin, Dmitry</creatorcontrib><creatorcontrib>Timofte, Radu</creatorcontrib><creatorcontrib>Jia, Ziheng</creatorcontrib><creatorcontrib>Zhang, Zicheng</creatorcontrib><creatorcontrib>Sun, Wei</creatorcontrib><creatorcontrib>Qian, Jiaying</creatorcontrib><creatorcontrib>Cao, Yuqin</creatorcontrib><creatorcontrib>Sun, Yinan</creatorcontrib><creatorcontrib>Zhu, Yuxin</creatorcontrib><creatorcontrib>Xiongkuo Min</creatorcontrib><creatorcontrib>Zhai, Guangtao</creatorcontrib><creatorcontrib>De, Kanjar</creatorcontrib><creatorcontrib>Luo, Qing</creatorcontrib><creatorcontrib>Ao-Xiang Zhang</creatorcontrib><creatorcontrib>Zhang, Peng</creatorcontrib><creatorcontrib>Lei, Haibo</creatorcontrib><creatorcontrib>Jiang, Linyan</creatorcontrib><creatorcontrib>Li, Yaqing</creatorcontrib><creatorcontrib>Meng, Wenhui</creatorcontrib><creatorcontrib>Chen, Zhenzhong</creatorcontrib><creatorcontrib>Cheng, Zhengxue</creatorcontrib><creatorcontrib>Xiao, Jiahao</creatorcontrib><creatorcontrib>Xu, Jun</creatorcontrib><creatorcontrib>He, Chenlong</creatorcontrib><creatorcontrib>Zheng, Qi</creatorcontrib><creatorcontrib>Zhu, Ruoxi</creatorcontrib><creatorcontrib>Li, Min</creatorcontrib><creatorcontrib>Fan, Yibo</creatorcontrib><creatorcontrib>Tu, Zhengzhong</creatorcontrib><title>AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results</title><title>arXiv.org</title><description>Video quality assessment (VQA) is a crucial task in the development of video compression standards, as it directly impacts the viewer experience. This paper presents the results of the Compressed Video Quality Assessment challenge, held in conjunction with the Advances in Image Manipulation (AIM) workshop at ECCV 2024. The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H.264, HEVC/H.265, AV1, and VVC/H.266) and containing a comprehensive collection of compression artifacts. To measure the methods performance, we employed traditional correlation coefficients between their predictions and subjective scores, which were collected via large-scale crowdsourced pairwise human comparisons. For training purposes, participants were provided with the Compressed Video Quality Assessment Dataset (CVQAD), a previously developed dataset of 1022 videos. Up to 30 participating teams registered for the challenge, while we report the results of 6 teams, which submitted valid final solutions and code for reproducing the results. Moreover, we calculated and present the performance of state-of-the-art VQA methods on the developed dataset, providing a comprehensive benchmark for future research. The dataset, results, and online leaderboard are publicly available at https://challenges.videoprocessing.ai/challenges/compressedvideo-quality-assessment.html.</description><subject>Codec</subject><subject>Correlation coefficients</subject><subject>Datasets</subject><subject>Image compression</subject><subject>Image manipulation</subject><subject>Image quality</subject><subject>Performance evaluation</subject><subject>Quality assessment</subject><subject>Teams</subject><subject>Video compression</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>BENPR</sourceid><recordid>eNqNisEKgkAUAJcgSMp_eNBZ2HbVrJtIUQeDIrrKwr5SWXfNtx76-zz0AZ0GZmbGAiHlJspiIRYsJGo55yLdiiSRAbvk5xIEFzEUtTIG7QvBWShc1w9IhBoejUYH11GZxn8gnxxRh9bvoURfO02grIYb0mg8rdj8qQxh-OOSrY-He3GK-sG9RyRftW4c7JQqyXdpLLOEx_K_6wvZbTy0</recordid><startdate>20241022</startdate><enddate>20241022</enddate><creator>Smirnov, Maksim</creator><creator>Gushchin, Aleksandr</creator><creator>Antsiferova, Anastasia</creator><creator>Vatolin, Dmitry</creator><creator>Timofte, Radu</creator><creator>Jia, Ziheng</creator><creator>Zhang, Zicheng</creator><creator>Sun, Wei</creator><creator>Qian, Jiaying</creator><creator>Cao, Yuqin</creator><creator>Sun, Yinan</creator><creator>Zhu, Yuxin</creator><creator>Xiongkuo Min</creator><creator>Zhai, Guangtao</creator><creator>De, Kanjar</creator><creator>Luo, Qing</creator><creator>Ao-Xiang Zhang</creator><creator>Zhang, Peng</creator><creator>Lei, Haibo</creator><creator>Jiang, Linyan</creator><creator>Li, Yaqing</creator><creator>Meng, Wenhui</creator><creator>Chen, Zhenzhong</creator><creator>Cheng, Zhengxue</creator><creator>Xiao, Jiahao</creator><creator>Xu, Jun</creator><creator>He, Chenlong</creator><creator>Zheng, Qi</creator><creator>Zhu, Ruoxi</creator><creator>Li, Min</creator><creator>Fan, Yibo</creator><creator>Tu, Zhengzhong</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20241022</creationdate><title>AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results</title><author>Smirnov, Maksim ; Gushchin, Aleksandr ; Antsiferova, Anastasia ; Vatolin, Dmitry ; Timofte, Radu ; Jia, Ziheng ; Zhang, Zicheng ; Sun, Wei ; Qian, Jiaying ; Cao, Yuqin ; Sun, Yinan ; Zhu, Yuxin ; Xiongkuo Min ; Zhai, Guangtao ; De, Kanjar ; Luo, Qing ; Ao-Xiang Zhang ; Zhang, Peng ; Lei, Haibo ; Jiang, Linyan ; Li, Yaqing ; Meng, Wenhui ; Chen, Zhenzhong ; Cheng, Zhengxue ; Xiao, Jiahao ; Xu, Jun ; He, Chenlong ; Zheng, Qi ; Zhu, Ruoxi ; Li, Min ; Fan, Yibo ; Tu, Zhengzhong</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_30964385043</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Codec</topic><topic>Correlation coefficients</topic><topic>Datasets</topic><topic>Image compression</topic><topic>Image manipulation</topic><topic>Image quality</topic><topic>Performance evaluation</topic><topic>Quality assessment</topic><topic>Teams</topic><topic>Video compression</topic><toplevel>online_resources</toplevel><creatorcontrib>Smirnov, Maksim</creatorcontrib><creatorcontrib>Gushchin, Aleksandr</creatorcontrib><creatorcontrib>Antsiferova, Anastasia</creatorcontrib><creatorcontrib>Vatolin, Dmitry</creatorcontrib><creatorcontrib>Timofte, Radu</creatorcontrib><creatorcontrib>Jia, Ziheng</creatorcontrib><creatorcontrib>Zhang, Zicheng</creatorcontrib><creatorcontrib>Sun, Wei</creatorcontrib><creatorcontrib>Qian, Jiaying</creatorcontrib><creatorcontrib>Cao, Yuqin</creatorcontrib><creatorcontrib>Sun, Yinan</creatorcontrib><creatorcontrib>Zhu, Yuxin</creatorcontrib><creatorcontrib>Xiongkuo Min</creatorcontrib><creatorcontrib>Zhai, Guangtao</creatorcontrib><creatorcontrib>De, Kanjar</creatorcontrib><creatorcontrib>Luo, Qing</creatorcontrib><creatorcontrib>Ao-Xiang Zhang</creatorcontrib><creatorcontrib>Zhang, Peng</creatorcontrib><creatorcontrib>Lei, Haibo</creatorcontrib><creatorcontrib>Jiang, Linyan</creatorcontrib><creatorcontrib>Li, Yaqing</creatorcontrib><creatorcontrib>Meng, Wenhui</creatorcontrib><creatorcontrib>Chen, Zhenzhong</creatorcontrib><creatorcontrib>Cheng, Zhengxue</creatorcontrib><creatorcontrib>Xiao, Jiahao</creatorcontrib><creatorcontrib>Xu, Jun</creatorcontrib><creatorcontrib>He, Chenlong</creatorcontrib><creatorcontrib>Zheng, Qi</creatorcontrib><creatorcontrib>Zhu, Ruoxi</creatorcontrib><creatorcontrib>Li, Min</creatorcontrib><creatorcontrib>Fan, Yibo</creatorcontrib><creatorcontrib>Tu, Zhengzhong</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Smirnov, Maksim</au><au>Gushchin, Aleksandr</au><au>Antsiferova, Anastasia</au><au>Vatolin, Dmitry</au><au>Timofte, Radu</au><au>Jia, Ziheng</au><au>Zhang, Zicheng</au><au>Sun, Wei</au><au>Qian, Jiaying</au><au>Cao, Yuqin</au><au>Sun, Yinan</au><au>Zhu, Yuxin</au><au>Xiongkuo Min</au><au>Zhai, Guangtao</au><au>De, Kanjar</au><au>Luo, Qing</au><au>Ao-Xiang Zhang</au><au>Zhang, Peng</au><au>Lei, Haibo</au><au>Jiang, Linyan</au><au>Li, Yaqing</au><au>Meng, Wenhui</au><au>Chen, Zhenzhong</au><au>Cheng, Zhengxue</au><au>Xiao, Jiahao</au><au>Xu, Jun</au><au>He, Chenlong</au><au>Zheng, Qi</au><au>Zhu, Ruoxi</au><au>Li, Min</au><au>Fan, Yibo</au><au>Tu, Zhengzhong</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results</atitle><jtitle>arXiv.org</jtitle><date>2024-10-22</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>Video quality assessment (VQA) is a crucial task in the development of video compression standards, as it directly impacts the viewer experience. This paper presents the results of the Compressed Video Quality Assessment challenge, held in conjunction with the Advances in Image Manipulation (AIM) workshop at ECCV 2024. The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H.264, HEVC/H.265, AV1, and VVC/H.266) and containing a comprehensive collection of compression artifacts. To measure the methods performance, we employed traditional correlation coefficients between their predictions and subjective scores, which were collected via large-scale crowdsourced pairwise human comparisons. For training purposes, participants were provided with the Compressed Video Quality Assessment Dataset (CVQAD), a previously developed dataset of 1022 videos. Up to 30 participating teams registered for the challenge, while we report the results of 6 teams, which submitted valid final solutions and code for reproducing the results. Moreover, we calculated and present the performance of state-of-the-art VQA methods on the developed dataset, providing a comprehensive benchmark for future research. The dataset, results, and online leaderboard are publicly available at https://challenges.videoprocessing.ai/challenges/compressedvideo-quality-assessment.html.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | EISSN: 2331-8422 |
| ispartof | arXiv.org, 2024-10 |
| issn | 2331-8422 |
| language | eng |
| recordid | cdi_proquest_journals_3096438504 |
| source | Free E- Journals |
| subjects | Codec Correlation coefficients Datasets Image compression Image manipulation Image quality Performance evaluation Quality assessment Teams Video compression |
| title | AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-30T08%3A16%3A28IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=AIM%202024%20Challenge%20on%20Compressed%20Video%20Quality%20Assessment:%20Methods%20and%20Results&rft.jtitle=arXiv.org&rft.au=Smirnov,%20Maksim&rft.date=2024-10-22&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E3096438504%3C/proquest%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3096438504&rft_id=info:pmid/&rfr_iscdi=true |