正则化权值自适应的相对全变分图像平滑
Adaptive regularization of the weighted relative total variation for image smoothing
- 2024年29卷第12期 页码:3578-3594
纸质出版日期: 2024-12-16
DOI: 10.11834/jig.230828
移动端阅览
浏览全部资源
扫码关注微信
纸质出版日期: 2024-12-16 ,
移动端阅览
崔鹏, 梁皓涵, 王志强, 刘婷婷. 2024. 正则化权值自适应的相对全变分图像平滑. 中国图象图形学报, 29(12):3578-3594
Cui Peng, Liang Haohan, Wang Zhiqiang, Liu Tingting. 2024. Adaptive regularization of the weighted relative total variation for image smoothing. Journal of Image and Graphics, 29(12):3578-3594
目的
2
针对目前已有的纹理结构滤波方法存在无法有效保证在滤除纹理的同时保持结构稳定的问题,提出一种正则化权值自适应的相对全变分图像平滑算法。
方法
2
首先,提出一种具有纹理抑制和结构保持的多尺度区间圆形梯度算子,其中引入了定向各向异性结构度量框架,提高了纹理—结构间的区分度。随后,利用高斯混合模型和EM(expectation maximization)算法实现纹理层和结构层的分离。最后,根据纹理和结构之间的差异性,对相对全变分模型中的正则化项进行自适应设置,使之可以在纹理区域利用大权重的正则化权值进行纹理抑制;在结构区域利用小权重的正则化权值进行结构保持。
结果
2
在视觉层面上,通过测试油画、十字绣、涂鸦、壁画和自然场景类型图像,并与已有的主流纹理结构滤波方法进行比较,本文算法不仅可以有效地抑制强梯度纹理,还可以保持弱梯度结构边缘的稳定;在定量度量方面,通过JPG格式图像压缩痕迹去除和高斯噪声图像平
滑,并与相对全变分、滚动引导图像滤波、双边纹理滤波、尺度感知纹理滤波和
<math id="M1"><msub><mrow><mi>L</mi></mrow><mrow><mn mathvariant="normal">0</mn></mrow></msub></math>
https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=70533348&type=
https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=70533346&type=
2.70933342
3.21733332
梯度最小化等方法进行关于峰值信噪比(peak signal-to-noise ratio,PSNR)和结构相似度(structural similarity index,SSIM)指标的比较,本文方法均取得最大值。此外,本文方法将所生成的滤波结果应用于图像风格化、细节增强和超像素分割,效果具有一定改进和提升。
结论
2
相较于已有的纹理结构滤波方法,本文方法在强梯度纹理抑制和精细结构保持方面更具优势,为后续图像处理奠定坚实的基础。
Objective
2
Texture shows different characteristics on different scales. On a smaller scale, the texture may appear more intricate and detailed, but on a larger scale, texture may present large structures and patterns. Therefore, texture patterns are complex and diverse and show various characteristics across patterns. For example, structural texture has clear geometric shape and arrangement, natural texture has randomness and complexity, and abstract texture presents a combination of different colors, lines, and patterns. While the human visual system can effectively distinguish an ordered structure from a disordered one, computers are generally unable to do so. Texture filtering is a basic and important tool in the fields of computer vision and computer graphics whose main purpose is to filter out unnecessary texture details and maintain the stability of the core structure. The mainstream texture filtering methods are mainly divided into local- and global-based methods. However, the existing texture filtering methods do not effectively guarantee the structural stability while filtering the texture. To address this problem, we propose an adaptive regularization of the weighted relative total variation for image smoothing algorithm.
Method
2
The main idea of this algorithm is to obtain a structure measure amplitude image with high texture structure discrimination and then use the relative total variation model to smooth this image according to the difference between the texture and structure. Our method implements texture filtering and structure preservation in three steps. First, we propose a multi-scale interval circular gradient operator that can effectively distinguish texture from structure. By inputting the intensity change information of the interval gradient in the horizontal and vertical directions (captured by the interval circular gradient operator) into the frame of directional anisotropic structure measurement (DASM), we generate a structure measure amplitude image with high contrast. In each iteration, we constantly adjust the scale radius of the interval circular gradient operator, where the scale radius of the interval circular gradient operator decreases as the number of iterations increases. On the one hand, this approach can capture the low-level semantic information of the texture structure in a large range at the initial stage of iteration and suppress the texture effectively. On the other hand, this approach can accurately capture the advanced semantic information of the texture structure at the end of the iteration to keep the structure stable. Second, given the high accuracy of the Gaussian mixture model in data classification, we separate the texture and structure layers of the structure measure amplitude image by using this model along with the EM algorithm. Before the separation operation, we conduct a morphological erosion operation on the image to refine the structure edge and shrink the structure area so as to improve the accuracy of the separation result. Finally, we adaptively assign regularization weights according to the structure measure amplitude image and the texture structure separation image. We assign a regularization term with high weight to the texture region for texture suppression, and we allocate a regularization term with a small weight in the structure area to maintain the stability of the fine structure and to ensure that the texture is filtered out in a large area to the greatest extent while maintaining the integrity of the structure.
Result
2
We ran our experiment on the Windows platform and implement our algorithm using Opencv and MATLAB. We defined three main parameters, including the maximum scale radius of the multi-scale interval circular gradient operator, the regular term of the texture region, and the regular term of the structure region. Maximum scale radius controls how much texture is suppressed. A larger regular term of the texture region corresponds to smoother filtering results, while a smaller regular term of the structure region corresponds to a better structure retention ability. On the visual level, by testing the images of oil paintings, cross embroideries, graffiti, murals, and natural scenes and comparing with the existing mainstream texture filtering methods, our proposed algorithm not only effectively suppresses the strong gradient texture but also maintains the stability of the edge of the weak gradient structure. In terms of quantitative measurement, by removing compressed traces of JPG images and smoothing Gaussian noise images, our proposed algorithm obtains the maximum peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) compared with the relative total variation, rolling guidance filtering, bilateral texture filtering, scale-aware texture filtering, and
<math id="M2"><msub><mrow><mi>L</mi></mrow><mrow><mn mathvariant="normal">0</mn></mrow></msub></math>
https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=70533359&type=
https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=70533350&type=
2.70933342
3.21733332
gradient minimization.
Conclusion
2
Compared with the existing texture filtering methods, the proposed algorithm achieves strong gradient texture suppression and fine structure preservation by using the adaptive allocation of regularization weights and completes the differentiated filtering operation between the texture and structure. Experiments show that our algorithm can maintain the main structure of the image and achieve gradient smoothing. This algorithm can be used to design powerful image preprocessing methods for image stylization, detail enhancement, HDR tone mapping, superpixel segmentation, and other fields sensitive to strong gradient texture.
图像平滑纹理滤波相对全变分多尺度正则项自适应
image smoothingtexture filteringrelative total variationmulti-scaleregularization term adaptation
Cho H, Lee H, Kang H and Lee S. 2014. Bilateral texture filtering. ACM Transactions on Graphics, 33(4): #128 [DOI: 10.1145/2601097.2601188http://dx.doi.org/10.1145/2601097.2601188]
Fang S, Yao Z J and Zhang J. 2019. Scale and gradient aware image smoothing. IEEE Access, 7: 166268-166281 [DOI: 10.1109/ACCESS.2019.2953550http://dx.doi.org/10.1109/ACCESS.2019.2953550]
Farbman Z, Fattal R, Lischinski D and Szeliski R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics, 27(3): 1-10 [DOI: 10.1145/1360612.1360666http://dx.doi.org/10.1145/1360612.1360666]
Jeon J, Lee H, Kang H and Lee S. 2016. Scale-aware structure-preserving texture filtering. Computer Graphics Forum, 35(7): 77-86 [DOI: 10.1111/cgf.13005http://dx.doi.org/10.1111/cgf.13005]
Jiang L X, Li X J and Wang Y D. 2024. Iterative unsupervised deep bilateral texture filtering. The Visual Computer, 40(5): 3055-3067 [DOI: 10.1007/s00371-023-03010-whttp://dx.doi.org/10.1007/s00371-023-03010-w]
Karacan L, Erdem E and Erdem A. 2013. Structure-preserving image smoothing via region covariances. ACM Transactions on Graphics, 32(6): #176 [DOI: 10.1145/2508363.2508403http://dx.doi.org/10.1145/2508363.2508403]
Lee H, Jeon J, Kim J and Lee S. 2017. Structure-texture decomposition of images with interval gradient. Computer Graphics Forum, 36(6): 262-274 [DOI: 10.1111/cgf.12875http://dx.doi.org/10.1111/cgf.12875]
Lin J Y, Liu C X, Zhang J K and Li H Y. 2022. Texture-smoothing-oriented directional filtering scales-predicting model. Journal of Image and Graphics, 27(8): 2506-2515
林俊彦, 刘春晓, 章金凯, 李泓易. 2022. 面向纹理平滑的方向性滤波尺度预测模型. 中国图象图形学报, 27(8): 2506-2515 [DOI: 10.11834/jig.210176http://dx.doi.org/10.11834/jig.210176]
Liu Y, Liu G D, Liu H L and Liu C Y. 2020. Structure-aware texture filtering based on local histogram operator. IEEE Access, 8: 43838-43849 [DOI: 10.1109/ACCESS.2020.2977408http://dx.doi.org/10.1109/ACCESS.2020.2977408]
Pradhan K and Patra S. 2022. Structure preserving semantic texture filtering//Proceedings of 2022 IEEE Calcutta Conference (CALCON). Kolkata, India: IEEE: 283-287 [DOI: 10.1109/CALCON56258.2022.10060620http://dx.doi.org/10.1109/CALCON56258.2022.10060620]
Pradhan K and Patra S. 2024. Semantic-aware structure-preserving median morpho-filtering. The Visual Computer, 40(2): 505-521 [DOI: 10.1007/s00371-023-02796-zhttp://dx.doi.org/10.1007/s00371-023-02796-z]
Shao H and Liu C X. 2018. Texture filtering by using texture gradient suppression and L0 gradient minimization. Journal of Image and Graphics, 23(11): 1666-1675
邵欢, 刘春晓. 2018. 结合纹理梯度抑制与L0梯度最小化的纹理滤波. 中国图象图形学报, 23(11): 1666-1675 [DOI: 10.11834/jig.180280http://dx.doi.org/10.11834/jig.180280]
Song Y and Sun W Y. 2021. Edge-preserving smoothing algorithm based on l0 gradient minimization of image-patch. Journal of Shenzhen University Science and Engineering (Science Engineering), 38(3): 307-314
宋昱, 孙文赟. 2021. 基于图像块l0梯度最小化的边缘保持平滑算法. 深圳大学学报(理工版), 38(3): 307-314 [DOI: 10.3724/SP.J.1249.2021.03307http://dx.doi.org/10.3724/SP.J.1249.2021.03307]
Xiao Y, Zhu X Y, He Y T and Zheng Y. 2019. Detail-aware texture filtering algorithm. Journal of Image and Graphics, 24(6): 969-978
肖懿, 朱贤益, 何仰提, 郑燕. 2019. 细节感知的纹理滤波算法. 中国图象图形学报, 24(6): 969-978 [DOI: 10.11834/jig.180521http://dx.doi.org/10.11834/jig.180521]
Xu L, Lu C W, Xu Y and Jia J Y. 2011. Image smoothing via L0 gradient minimization. ACM Transactions on Graphics, 30(6): 1-12 [DOI: 10.1145/2070781.2024208http://dx.doi.org/10.1145/2070781.2024208]
Xu L, Yan Q, Xia Y and Jia J Y. 2012. Structure extraction from texture via relative total variation. ACM Transactions on Graphics, 31(6): #139 [DOI: 10.1145/2366145.2366158http://dx.doi.org/10.1145/2366145.2366158]
Xu P P and Wang W C. 2019. Structure-aware window optimization for texture filtering. IEEE Transactions on Image Processing, 28(9): 4354-4363 [DOI: 10.1109/TIP.2019.2904847http://dx.doi.org/10.1109/TIP.2019.2904847]
Zang Y, Huang H and Zhang L. 2015. Guided adaptive image smoothing via directional anisotropic structure measurement. IEEE Transactions on Visualization and Computer Graphics, 21(9): 1015-1027 [DOI: 10.1109/TVCG.2015.2410296http://dx.doi.org/10.1109/TVCG.2015.2410296]
Zhang Q, Shen X Y, Xu L and Jia J Y. 2014. Rolling guidance filter//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland: Springer: 815-830 [DOI: 10.1007/978-3-319-10578-9_53http://dx.doi.org/10.1007/978-3-319-10578-9_53]
Zhao H L, Jiang L, Jin X G, Du H and Li X J. 2018. Constant time texture filtering. The Visual Computer, 34(1): 83-92 [DOI: 10.1007/s00371-016-1315-zhttp://dx.doi.org/10.1007/s00371-016-1315-z]
Zhou P C, Xue Y and Xue M G. 2023. Adaptive side window joint bilateral filter. The Visual Computer, 39(4): 1533-1555 [DOI: 10.1007/s00371-022-02427-zhttp://dx.doi.org/10.1007/s00371-022-02427-z]
Zhu D K, Wang W M, Xue X, Xie H R, Cheng G and Wang F L. 2023. Structure-preserving image smoothing via contrastive learning. The Visual Computer, 40: 5139-5153 [DOI: 10.1007/s00371-023-02897-9http://dx.doi.org/10.1007/s00371-023-02897-9]
相关作者
相关机构