基于纹元森林和显著性先验的弱监督图像语义分割方法

doi:10.11999/JEIT170472

Abstract
Figure/Table
References (24)
Related Citation (9)

Download: PDF (16756 KB)
Export: BibTeX | EndNote (RIS)

Abstract Most previous weakly supervised semantic segmentation works utilize the labels of the whole training set and thereby need the construction of a relationship graph about image labels. This method lack of structure information in single image and suffer from enormous quantity parameters which result in expensive computation. In this study, a weakly-supervised semantic segmentation algorithm is proposed. Under Conditional Random Field (CRF) framework, an novel energy function expression is developed based on saliency priors as structure context relationship, which avoids the construction of a huge graph in whole training dataset. Specifically, a nonparametric random Semantic Texton Forest (STF) is obtained using weakly supervised training data and images saliency. Then STF feature is extracted from image superpixels and probability estimates of superpixels label is calculated by naive Bayesian method. Finally, a CRF based optimization algorithm is proposed which can efficiency solved by alpha expansion algorithm. Experiments on the MSRC-21 dataset show that the new algorithm outperforms some previous influential weakly-supervised segmentation algorithms with no building graph in whole training set.

Key words： Semantic segmentation Weakly supervised learning Saliency detection Semantic Texton Forest (STF) Conditional Random Fields (CRF)

Received: 17 May 2017 Published: 27 December 2017

PACS:

TP391.4

Fund:The Specialized Research Fund for the Doctoral Program of Higher Education (SRFDP20131201110001), The Applied Basic Research Programs of China National Textile and Apparel Council (J201509), The Scientific Studies Program of Higher Education of Inner Mongolia Municipality (NJZY237)

Corresponding Authors: XIAO Zhitao E-mail: xiaozhitao@tjpu.edu.cn

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	HAN Zheng
	XIAO Zhitao

Cite this article:

HAN Zheng,XIAO Zhitao. Weakly Supervised Semantic Segmentation Based on Semantic Texton Forest and Saliency Prior[J]. JEIT, 2018, 40(3): 610-617.

URL:

http://jeit.ie.ac.cn/EN/10.11999/JEIT170472 OR http://jeit.ie.ac.cn/EN/Y2018/V40/I3/610

[1]	KOHLI Pushmeet, LADICKY L, and TORR P H S. Robust higher order potentials for enforcing label consistency[J]. International Journal of Computer Vision, 2009, 82(3): 302-324. doi: 10.1007/s11263-008-0202-0.
[2]	ZHANG L, SONG M, LIU Z, et al. Probabilistic graphlet cut: Exploiting spatial structure cue for weakly supervised image segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013: 1908-1915. doi: 10.1109/CVPR.2013.249.
[3]	ZHANG Ke, ZHANG Wei, ZHENG Yingbin, et al. Sparse reconstruction for weakly supervised semantic segmentation [C]. International Joint Conference on Artificial Intelligence, Beijing, China, 2013: 1889-1895.
[4]	VEZHNEVETS A, FERRARIV, and BUHMANN J M. Weakly supervised structured output learning for semantic segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 2012: 845-852. doi: 10.1109/CVPR.2012.6247757.
[5]	VEZHNEVETS A and BUHMANN J M. Towards weakly supervised semantic segmentation by means of multiple instance and multitask learning[C]. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010: 3249-3256. doi: 10.1109/CVPR.2010. 5540060.
[6]	SHOTTON Jamie, JOHNSON Matthew, and CIPOLLA Roberto. Semantic texton forests for image categorization and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA, 2008: 1-8. doi: 10.1109/CVPR.2008.4587503.
[7]	WEI Yunchao, LIANG Xiaodan, CHEN Yunpeng, et al. STC: a simple to complex framework for weakly-supervised semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(11): 2314-2320. doi: 10.1109/TPAMI.2016.2636150.
[8]	VEZHNEVETS A, FERRARI V, and BUHMANN J. M. Weakly supervised semantic segmentation with a multi-image model[C]. IEEE International Conference on Computer Vision, Washington, DC, USA, 2011: 643-650. doi: 10.1109/ ICCV.2011.6126299.
[9]	ZENG Zinan, XIAO Shijie, JIA Kui, et al. Learning by associating ambiguously labeled images[C]. IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013: 708-715. doi: 10.1109/CVPR.2013.97.
[10]	VEZHNEVETS A, BUHMANN J M, and FERRARI V. Active learning for semantic segmentation with expected change[C]. IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 2012: 3162-3169. doi: 10.1109/CVPR.2012.6248050.
[11]	YING P, LIU J, and LU H. Dictionary learning based superpixels clustering for weakly-supervised semantic segmentation[C]. IEEE International Conference on Image Processing, Quebec City, QC, Canada, 2015: 4258-4262. doi: 10.1109/ICIP.2015.7351609.
[12]	OQUAB Maxime, BOTTOU Leon, LAPTEV Ivan, et al. Is object localization for free? Weakly-supervised learning with convolutional neural networks[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 685-694. doi: 10.1109/CVPR.2015.7298668.
[13]	PAPANDREOU George, CHEN Liang-chieh, MURPHY Kevin, et al. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation[C]. IEEE International Conference on Computer Vision, Santiago, Chile, 2015: 1742-1750, doi: 10.1109/ICCV.2015. 203.
[14]	PINHEIRO Pedro O and COLLOBERT Ronan. From image- level to pixel-level labeling with Convolutional Networks[C]. IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, USA, 2015: 1713-1721. doi: 10.1109/CVPR.2015.7298780.
[15]	XU Jia, SCHWING A G, and URTASUN R. Tell me what you see and i will show you where it is[C]. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014: 3190-3197. doi: 10.1109/CVPR.2014.408.
[16]	CABRAL R, TORRE F D L, COSTEIRA J P, et al. Matrix completion for weakly-supervised multi-label image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(1): 121-135. doi: 10.1109/ TPAMI.2014.2343234.
[17]	KOLMOGOROV Vladimir and ZABIH R. What energy functions can be minimized via graph cuts?[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(2): 147-159. doi: 10.1109/TPAMI.2004.1262177.
[18]	GEURTS Pierre, DAMIEN Ernst, and LOUIS Wehenkel. Extremely randomized trees[J]. Machine Learning, 2006, 63(1): 3-42. doi: 10.1007/s10994-006-6226-1.
[19]	JIANG Huaizu, WANG Jingdong, YUAN Zejian, et al. Salient object detection: A discriminative regional feature integration approach[J]. International Journal of Computer Vision, 2016, 9(4): 1-18. doi: 10.1007/s11263-016-0977-3.
[20]	GOFERMAN Stas, ZELNIK-MANOR Lihi, and TAL Ayellet. Context-aware saliency detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(10): 1915-1926. doi: 10.1109/TPAMI.2011.272.
[21]	SHOTTON Jamie, WINN John, ROTHER Carsten, et al. Texton Boost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation[C]. European Conference on Computer Vision, Graz, Austria, 2006: 1-15. doi: 10.1007/11744023-1.
[22]	LEVINSHTEIN A, STERE A, KUTULAKOS K N, et al. TurboPixels: fast superpixels using geometric flows[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(12): 2290-2297. doi: 10.1109/TPAMI.2009.96.
[23]	LADICKY L, RUSSELL C, KOHLI P, et al. Associative hierarchical random fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(6): 1056-1077. doi: 10.1109/TPAMI.2013.165.
[24]	VERBEEK J and TRIGGS B. Region classification with markov field aspect models[C]. IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 2007: 1-8. doi: 10.1109/CVPR.2007.383098.