Texture-aware re-parameterization to mitigate accuracy drop after quantization for 4K/8K image super-resolution

In this paper, we aim to improve super-resolution (SR) imaging quality on 4K/8K images with a negligible increase in computational cost and alleviate the accuracy drop after quantization. Experiments have discovered two phenomena: (1) the re-parameterization (Rep) technique has no apparent advantages in regions with smooth textures, and (2) the accuracy drop after quantization compared with SR methods based on No-Rep because the structure information of the image will be weakened caused by the multi-branch fusion in Rep technique. Inspired by the above phenomenon, we innovatively combine texture classification and Rep techniques to propose a generic TARepSR framework (consisting of TA-Module and VarRepSR-Module) to adjust the branching of Rep blocks texture-awarely. Specifically, the TA-Module is a lightweight classification network to classify textures in different regions. An existing SR network using texture-aware Rep techniques can be used as the VarRepSR-Module to super-resolute images with higher imaging quality without additional computational costs. Moreover, we propose a TC loss to avoid over-fitting caused by an unbalanced degree of the tendency of classification results to classify different textures better. Experiments show that our TARepSR can not only improve the imaging quality of most existing methods (e.g., FSRCNN, CARN, EDSR, XLSR) on 4K/8K images with negligible increase in computational cost but also improve the accuracy after quantization compared with the state-of-the-art Rep methods.