Infrared and Visible Image Fusion Based on Contrast Enhancement and Multi-scale Edge-preserving Decomposition
ZHU Haoran① LIU Yunqing① ZHANG Wenying②③
①(School of Electronics and Information Engineering, Changchun University of Science and Technology, Changchun 130022, China) ②(Photoelectric Engineering College, Changchun University of Science and Technology, Changchun 130022, China) ③(Academy of Opto-electronics, Chinese Academy of Sciences, Beijing 100094, China)
Abstract：The visibility of the visible images is not good under the poor lighting condition. If the visible and infrared images are fused directly, the resolution of the fused images is not ideal. In order to solve this problem, a modified infrared and visible image fusion approach based on contrast enhancement and multi-scale edge-preserving is proposed. Firstly, an adaptive enhancement method based on the guided filter is adopted to enhance the visibility of dark region content in the visible image. Input images are then decomposed with a scale-aware edge-preserving filter. Subsequently, saliency maps of infrared and visible images are calculated on the basis of frequency-tuned filtering. Finally, the fused images are reconstructed with the weighting maps. Experiments show that the proposed scheme can not only make the detail information more prominent, but also suppress the artifacts effectively.
AKERMAN A. Pyramidal techniques for multi-sensor fusion[J]. SPIE, 1992, 1828: 124-131. doi: 10.1117/12.131644.
[2]
TOET A, VALETON J M, and VAN RUYEN L J. Merging thermal and visual images by a contrast pyramid[J]. Optical Engineering, 1989, 28(7): 789-792. doi: 10.1117/12.7977034.
[3]
SHAO Zhenfeng, LIU Jun, and CHENG Qimin. Fusion of infrared and visible images based on focus measure operators in the curvelet domain[J]. Applied Optics, 2012, 51(12): 1910-1921. doi: 10.1364/AO.51.001910.
[4]
LEWIS J J, O’CALLAGHAN R J, NIKOLOV S G, et al. Pixel-and region-based image fusion with complex wavelets[J]. Information Fusion, 2007, 8(2): 119-130. doi: 10.1016/j.inffus. 2005.09.006.
[5]
LI Shutao, KANG Xudong, and HU Jianwen. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864-2875. doi: 10.1109/TIP.2013. 2244222.
[6]
ZHANG Qiong and MALDAGUE X. An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing[J]. Infrared Physics & Technology, 2016, 74(1): 11-20. doi: 10.1016/j.infrared.2015.11.003.
[7]
RIZZI A, GATTA C, and MARINI D. A new algorithm for unsupervised global and local color correction[J]. Pattern Recognition Letters, 2003, 24(11): 1663-1677. doi: 10.1016/ S0167-8655(02)00323-9.
WEN Haibin, BI Duyan, MA Shiping, et al. Variational retinex algorithm for infrared image enhancement with staircase effect suppression and detail enhancement[J]. Acta Optica Sinica, 2016, 36(9): 122-131. doi: 10.3788/aos201636. 0911005.
[9]
TAO Li, NGO Hau, ZHANG Ming, et al. A multisensory image fusion and enhancement system for assisting drivers in poor lighting conditions[C]. Proceedings of the 34th Applied Imagery and Pattern Recognition Workshop, Washington, 2005: 106-113. doi: 10.1109/AIPR.2005.9.
[10]
LIU Zheng and LAGANIERE R. Context enhancement through infrared vision: A modified fusion scheme[J]. Signal Image and Video Processing, 2007, 1(4): 293-301. doi: 10.1007 /s11760-007-0025-4.
XIE Wei, ZHOU Yuqin, and YOU Min. Improved guided image filtering integrated with gradient information[J]. Journal of Image and Graphics, 2016, 21(9): 1119-1126. doi: 10.11834/jig.20160901.
SU Juan, LI Bing, and WANG Yanzhao. Infrared image enhancement based on PCNN segmentation and fuzzy set theory[J]. Acta Optica Sinica, 2016, 36(9): 82-90. doi: 10.3788 /aos201636.0910001.
LIU Feng, SHEN Tongsheng, MA Xinxing. Image fusion via cross bilateral filter and visual weight information[J]. Chinese Journal of Scientific Instrument, 2017, 38(4): 1005-1013. doi: 10.3969/j.issn.0254-3087.2017.04.027.
[14]
ZHANG Qi, SHEN Xiaoyong, XU Li, et al. Rolling guidance filter[C]. Proceedings of the 13th European Conference on Computer Vision, Berlin Heidelberg, 2014: 815-830. doi: 10.1007/978-3-319-10578-9_53.
[15]
ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency- tuned salient region detection[C]. Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Miami, USA, 2009: 1597-1604. doi: 10.1109/ CVPR.2009.5206596.
[16]
ZHAO Jufeng, FENG Huajun, XU Zhihai, et al. Detail enhanced multi-source fusion using visual weight map extraction based on multi scale edge preserving decomposition[J]. Optics Communications, 2013, 287(2): 45-52. doi: 10.1016/j.optcom.2012.08.070.
CHEN Zhen, YANG Xiaoping, ZHANG Congxuan, et al. Infrared and visible image fusion based on the compensation mechanism in NSCT domain[J]. Chinese Journal of Scientific Instrument, 37(4): 860-870. doi: 10.3969/j.issn.0254-3087. 2016.04.019.
[18]
LIU Yu, LIU Shupeng, and WANG Zengfu. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24(7): 147-164. doi: 10.1016/j.inffus.2014.09.004.
SUN Yanjing, YANG Yufen, LIU Donglin, et al. Multiple- scale structural similarity image quality assessment based on internal generative mechanism[J]. Journal of Electronics & Information Technology, 2016, 38(1): 127-134. doi: 10.11999 /JEIT150616.
