Offset-Based In-Loop Filtering With a Deep Network in HEVC
With the great flexibility and performance of deep learning technology, there have been many attempts to replace existing functions inside video codecs such as High-Efficiency Video Coding (HEVC) with deep-learning-based solutions. One of the most researched approaches is adopting a deep network as...
Gespeichert in:
Veröffentlicht in: | IEEE access 2020, Vol.8, p.213958-213967 |
---|---|
Hauptverfasser: | , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | 213967 |
---|---|
container_issue | |
container_start_page | 213958 |
container_title | IEEE access |
container_volume | 8 |
creator | Lee, So Yoon Yang, Yoonmo Kim, Dongsin Cho, Seunghyun Oh, Byung Tae |
description | With the great flexibility and performance of deep learning technology, there have been many attempts to replace existing functions inside video codecs such as High-Efficiency Video Coding (HEVC) with deep-learning-based solutions. One of the most researched approaches is adopting a deep network as an image restoration filter to recover distorted compressed frames. In this paper, instead, we introduce a novel idea for using a deep network, in which it chooses and transmits the side information according to the type of errors and contents, inspired by the sample adaptive offset filter in HEVC. A part of the network computes the optimal offset values while another part estimates the type of error and contents simultaneously. The combination of two subnetworks can address the estimation of highly nonlinear and complicated errors compared to conventional deep- learning-based schemes. Experimental results show that the proposed system yields an average bit-rate saving of 4.2% and 2.8% for the low-delay P and random access modes, respectively, compared to the conventional HEVC. Moreover, the performance improvement is up to 6.3% and 3.9% for higher-resolution sequences. |
doi_str_mv | 10.1109/ACCESS.2020.3040751 |
format | Article |
fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1109_ACCESS_2020_3040751</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9272307</ieee_id><doaj_id>oai_doaj_org_article_01594e7b69594215a84bb00424a09613</doaj_id><sourcerecordid>2468761414</sourcerecordid><originalsourceid>FETCH-LOGICAL-c408t-ce823accc3263affec9adb292a6a6ddd0ad03bca980c9abed9fc01b3919c482e3</originalsourceid><addsrcrecordid>eNpNUE1PwzAMrRBITLBfsEslzh3OR9OG2ygbmzSxw_g4Rmnqjo7RlLQT4t-T0WnCF1vPfs_2C4IRgTEhIG8nWTZdr8cUKIwZcEhichYMKBEyYjET5__qy2DYtlvwkXooTgbB3aosW-yie91iES7qaGltE86qXYeuqjfhW9W9hzp8QGzCJ-y-rfsIqzqcT1-z6-Ci1LsWh8d8FbzMps_ZPFquHhfZZBkZDmkXGUwp08YYRgXTZYlG6iKnkmqhRVEUoAtgudEyBd_JsZClAZIzSaThKUV2FSx63cLqrWpc9andj7K6Un-AdRulXVeZHSogseSY5P43ySmJdcrzHIBTrkEKwrzWTa_VOPu1x7ZTW7t3tT9fUS7SRBBOuJ9i_ZRxtm0dlqetBNTBc9V7rg6eq6PnnjXqWRUinhiSJpRBwn4BBHJ6ZQ</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2468761414</pqid></control><display><type>article</type><title>Offset-Based In-Loop Filtering With a Deep Network in HEVC</title><source>IEEE Open Access Journals</source><source>DOAJ Directory of Open Access Journals</source><source>EZB-FREE-00999 freely available EZB journals</source><creator>Lee, So Yoon ; Yang, Yoonmo ; Kim, Dongsin ; Cho, Seunghyun ; Oh, Byung Tae</creator><creatorcontrib>Lee, So Yoon ; Yang, Yoonmo ; Kim, Dongsin ; Cho, Seunghyun ; Oh, Byung Tae</creatorcontrib><description>With the great flexibility and performance of deep learning technology, there have been many attempts to replace existing functions inside video codecs such as High-Efficiency Video Coding (HEVC) with deep-learning-based solutions. One of the most researched approaches is adopting a deep network as an image restoration filter to recover distorted compressed frames. In this paper, instead, we introduce a novel idea for using a deep network, in which it chooses and transmits the side information according to the type of errors and contents, inspired by the sample adaptive offset filter in HEVC. A part of the network computes the optimal offset values while another part estimates the type of error and contents simultaneously. The combination of two subnetworks can address the estimation of highly nonlinear and complicated errors compared to conventional deep- learning-based schemes. Experimental results show that the proposed system yields an average bit-rate saving of 4.2% and 2.8% for the low-delay P and random access modes, respectively, compared to the conventional HEVC. Moreover, the performance improvement is up to 6.3% and 3.9% for higher-resolution sequences.</description><identifier>ISSN: 2169-3536</identifier><identifier>EISSN: 2169-3536</identifier><identifier>DOI: 10.1109/ACCESS.2020.3040751</identifier><identifier>CODEN: IAECCG</identifier><language>eng</language><publisher>Piscataway: IEEE</publisher><subject>Adaptive filters ; Adaptive sampling ; Codec ; Convolutional codes ; Convolutional neural network (CNN) ; Convolutional neural networks ; Decoding ; deep learning ; Encoding ; HEVC ; Image coding ; Image filters ; Image restoration ; in-loop filter ; Information filters ; Random access ; sample adaptive offset (SAO) ; Video compression</subject><ispartof>IEEE access, 2020, Vol.8, p.213958-213967</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c408t-ce823accc3263affec9adb292a6a6ddd0ad03bca980c9abed9fc01b3919c482e3</citedby><cites>FETCH-LOGICAL-c408t-ce823accc3263affec9adb292a6a6ddd0ad03bca980c9abed9fc01b3919c482e3</cites><orcidid>0000-0003-1985-4420 ; 0000-0003-2816-1685 ; 0000-0003-1437-2422</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9272307$$EHTML$$P50$$Gieee$$Hfree_for_read</linktohtml><link.rule.ids>314,780,784,864,2102,4024,27633,27923,27924,27925,54933</link.rule.ids></links><search><creatorcontrib>Lee, So Yoon</creatorcontrib><creatorcontrib>Yang, Yoonmo</creatorcontrib><creatorcontrib>Kim, Dongsin</creatorcontrib><creatorcontrib>Cho, Seunghyun</creatorcontrib><creatorcontrib>Oh, Byung Tae</creatorcontrib><title>Offset-Based In-Loop Filtering With a Deep Network in HEVC</title><title>IEEE access</title><addtitle>Access</addtitle><description>With the great flexibility and performance of deep learning technology, there have been many attempts to replace existing functions inside video codecs such as High-Efficiency Video Coding (HEVC) with deep-learning-based solutions. One of the most researched approaches is adopting a deep network as an image restoration filter to recover distorted compressed frames. In this paper, instead, we introduce a novel idea for using a deep network, in which it chooses and transmits the side information according to the type of errors and contents, inspired by the sample adaptive offset filter in HEVC. A part of the network computes the optimal offset values while another part estimates the type of error and contents simultaneously. The combination of two subnetworks can address the estimation of highly nonlinear and complicated errors compared to conventional deep- learning-based schemes. Experimental results show that the proposed system yields an average bit-rate saving of 4.2% and 2.8% for the low-delay P and random access modes, respectively, compared to the conventional HEVC. Moreover, the performance improvement is up to 6.3% and 3.9% for higher-resolution sequences.</description><subject>Adaptive filters</subject><subject>Adaptive sampling</subject><subject>Codec</subject><subject>Convolutional codes</subject><subject>Convolutional neural network (CNN)</subject><subject>Convolutional neural networks</subject><subject>Decoding</subject><subject>deep learning</subject><subject>Encoding</subject><subject>HEVC</subject><subject>Image coding</subject><subject>Image filters</subject><subject>Image restoration</subject><subject>in-loop filter</subject><subject>Information filters</subject><subject>Random access</subject><subject>sample adaptive offset (SAO)</subject><subject>Video compression</subject><issn>2169-3536</issn><issn>2169-3536</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2020</creationdate><recordtype>article</recordtype><sourceid>ESBDL</sourceid><sourceid>RIE</sourceid><sourceid>DOA</sourceid><recordid>eNpNUE1PwzAMrRBITLBfsEslzh3OR9OG2ygbmzSxw_g4Rmnqjo7RlLQT4t-T0WnCF1vPfs_2C4IRgTEhIG8nWTZdr8cUKIwZcEhichYMKBEyYjET5__qy2DYtlvwkXooTgbB3aosW-yie91iES7qaGltE86qXYeuqjfhW9W9hzp8QGzCJ-y-rfsIqzqcT1-z6-Ci1LsWh8d8FbzMps_ZPFquHhfZZBkZDmkXGUwp08YYRgXTZYlG6iKnkmqhRVEUoAtgudEyBd_JsZClAZIzSaThKUV2FSx63cLqrWpc9andj7K6Un-AdRulXVeZHSogseSY5P43ySmJdcrzHIBTrkEKwrzWTa_VOPu1x7ZTW7t3tT9fUS7SRBBOuJ9i_ZRxtm0dlqetBNTBc9V7rg6eq6PnnjXqWRUinhiSJpRBwn4BBHJ6ZQ</recordid><startdate>2020</startdate><enddate>2020</enddate><creator>Lee, So Yoon</creator><creator>Yang, Yoonmo</creator><creator>Kim, Dongsin</creator><creator>Cho, Seunghyun</creator><creator>Oh, Byung Tae</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>ESBDL</scope><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>7SR</scope><scope>8BQ</scope><scope>8FD</scope><scope>JG9</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0003-1985-4420</orcidid><orcidid>https://orcid.org/0000-0003-2816-1685</orcidid><orcidid>https://orcid.org/0000-0003-1437-2422</orcidid></search><sort><creationdate>2020</creationdate><title>Offset-Based In-Loop Filtering With a Deep Network in HEVC</title><author>Lee, So Yoon ; Yang, Yoonmo ; Kim, Dongsin ; Cho, Seunghyun ; Oh, Byung Tae</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c408t-ce823accc3263affec9adb292a6a6ddd0ad03bca980c9abed9fc01b3919c482e3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2020</creationdate><topic>Adaptive filters</topic><topic>Adaptive sampling</topic><topic>Codec</topic><topic>Convolutional codes</topic><topic>Convolutional neural network (CNN)</topic><topic>Convolutional neural networks</topic><topic>Decoding</topic><topic>deep learning</topic><topic>Encoding</topic><topic>HEVC</topic><topic>Image coding</topic><topic>Image filters</topic><topic>Image restoration</topic><topic>in-loop filter</topic><topic>Information filters</topic><topic>Random access</topic><topic>sample adaptive offset (SAO)</topic><topic>Video compression</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Lee, So Yoon</creatorcontrib><creatorcontrib>Yang, Yoonmo</creatorcontrib><creatorcontrib>Kim, Dongsin</creatorcontrib><creatorcontrib>Cho, Seunghyun</creatorcontrib><creatorcontrib>Oh, Byung Tae</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE Open Access Journals</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>IEEE access</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Lee, So Yoon</au><au>Yang, Yoonmo</au><au>Kim, Dongsin</au><au>Cho, Seunghyun</au><au>Oh, Byung Tae</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Offset-Based In-Loop Filtering With a Deep Network in HEVC</atitle><jtitle>IEEE access</jtitle><stitle>Access</stitle><date>2020</date><risdate>2020</risdate><volume>8</volume><spage>213958</spage><epage>213967</epage><pages>213958-213967</pages><issn>2169-3536</issn><eissn>2169-3536</eissn><coden>IAECCG</coden><abstract>With the great flexibility and performance of deep learning technology, there have been many attempts to replace existing functions inside video codecs such as High-Efficiency Video Coding (HEVC) with deep-learning-based solutions. One of the most researched approaches is adopting a deep network as an image restoration filter to recover distorted compressed frames. In this paper, instead, we introduce a novel idea for using a deep network, in which it chooses and transmits the side information according to the type of errors and contents, inspired by the sample adaptive offset filter in HEVC. A part of the network computes the optimal offset values while another part estimates the type of error and contents simultaneously. The combination of two subnetworks can address the estimation of highly nonlinear and complicated errors compared to conventional deep- learning-based schemes. Experimental results show that the proposed system yields an average bit-rate saving of 4.2% and 2.8% for the low-delay P and random access modes, respectively, compared to the conventional HEVC. Moreover, the performance improvement is up to 6.3% and 3.9% for higher-resolution sequences.</abstract><cop>Piscataway</cop><pub>IEEE</pub><doi>10.1109/ACCESS.2020.3040751</doi><tpages>10</tpages><orcidid>https://orcid.org/0000-0003-1985-4420</orcidid><orcidid>https://orcid.org/0000-0003-2816-1685</orcidid><orcidid>https://orcid.org/0000-0003-1437-2422</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 2169-3536 |
ispartof | IEEE access, 2020, Vol.8, p.213958-213967 |
issn | 2169-3536 2169-3536 |
language | eng |
recordid | cdi_crossref_primary_10_1109_ACCESS_2020_3040751 |
source | IEEE Open Access Journals; DOAJ Directory of Open Access Journals; EZB-FREE-00999 freely available EZB journals |
subjects | Adaptive filters Adaptive sampling Codec Convolutional codes Convolutional neural network (CNN) Convolutional neural networks Decoding deep learning Encoding HEVC Image coding Image filters Image restoration in-loop filter Information filters Random access sample adaptive offset (SAO) Video compression |
title | Offset-Based In-Loop Filtering With a Deep Network in HEVC |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T12%3A09%3A52IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Offset-Based%20In-Loop%20Filtering%20With%20a%20Deep%20Network%20in%20HEVC&rft.jtitle=IEEE%20access&rft.au=Lee,%20So%20Yoon&rft.date=2020&rft.volume=8&rft.spage=213958&rft.epage=213967&rft.pages=213958-213967&rft.issn=2169-3536&rft.eissn=2169-3536&rft.coden=IAECCG&rft_id=info:doi/10.1109/ACCESS.2020.3040751&rft_dat=%3Cproquest_cross%3E2468761414%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2468761414&rft_id=info:pmid/&rft_ieee_id=9272307&rft_doaj_id=oai_doaj_org_article_01594e7b69594215a84bb00424a09613&rfr_iscdi=true |