Stylized image denoising via noise style transfer and Quasi Siamese network
Neural style transfer has attracted wide attention for the surprising quality of synthetic images. However, the visual quality of existing methods usually suffers from style transfer noise (STN). Unlike the real-world noises contaminated in natural image capturing, the STN results from the stylized...
Gespeichert in:
Veröffentlicht in: | Signal processing. Image communication 2023-07, Vol.115, p.116958, Article 116958 |
---|---|
Hauptverfasser: | , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Neural style transfer has attracted wide attention for the surprising quality of synthetic images. However, the visual quality of existing methods usually suffers from style transfer noise (STN). Unlike the real-world noises contaminated in natural image capturing, the STN results from the stylized image synthesizing process. This difference leads to two major challenges for stylized image denoising: (1) Synthesized stylized images have no corresponding ground-truth images, which makes it challenging to obtain noisy-clean sample pairs for training denoising networks. (2) STNs are usually color noises or accompanied by color distortions, so color correction should be considered in denoising. To tackle the first challenge, we propose a novel strategy called noise style transfer to produce noisy-clean sample pairs. Specifically, the noise style transfer takes noises as special style textures and transfers them from the noisy stylized image onto the clean style image. To tackle the second challenge, we design a Quasi Siamese denoising network, which takes the noisy sample and its high saturation (HS) version as inputs of two network branches in training. Through mutual supervision between the two branches, the “photo developer” effect of HS images on color noise is conducive to simultaneous noise removal and color correction. Extensive experimental results on various stylized images substantially demonstrate the superior performance of our approach. With great adaptiveness to different style transfer methods, our approach successfully removes distinctive STNs while keeping the artistic texture and achieves a further reduction in style loss of up to 57.7%.
•We proposed the first promising solution for stylized image denoising.•We proposed Quasi Siamese network and noise style transfer to remove STN.•Experiments show our method removes STN while preserving details and style patterns. |
---|---|
ISSN: | 0923-5965 1879-2677 |
DOI: | 10.1016/j.image.2023.116958 |