|
|
|
| Moving Hand Segmentation Based on Multi-cues |
| RUAN Xiaogang LIN Jia YU Naigong ZHU Xiaoqing OUATTARA Sie |
(College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing 100124, China)
(Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China) |
|
|
|
|
Abstract For moving hand segmentation, in order not to use unreasonable assumptions and to solve the hand-face occlusion, a segmentation method based on skin color, grayscale, depth and motion cues is proposed. Firstly, according to the variance information of grayscale and depth optical flow, Motion Region of Interest (MRoI) is adaptively extracted to locate the moving body part. Then, corners which satisfy skin color and adaptive motion constraints are detected as skin seed points in the MRoI. Next, skin seed points are grown to [JL1]obtain candidate hand region utilizing skin color, depth and motion criterions. Finally, edge depth gradient, skeleton extraction and optimal path search are employed to segment moving hand region from candidate hand region. Experiment results show that the proposed method can effectively and accurately segment moving hand region under different circumstances, especially when the face is occluded by the hand.
|
|
Received: 08 July 2016
Published: 28 February 2017
|
|
|
| Fund: The National Natural Science Foundation of China (61375086), The Key Project of S&T Plan of Beijing Municipal Commission of Education (KZ201610005010) |
|
Corresponding Authors:
LIN Jia
E-mail: linjia.bjut@gmail.com
|
|
|
|
| [1] |
SHIEH Mingyuan, HSIEH Chungyu, and HSIEH Tsungmin. Fuzzy visual detection for human-robot interaction[J]. Engineering Computations, 2014, 31(8): 1709-1719. doi: 10.1108/EC-11-2012-0292.
|
| [2] |
张旭东, 杨静, 胡良梅, 等. 基于多层运动历史图像的飞行时间相机人体运动识别[J]. 电子与信息学报, 2014, 36(5): 1139-1144. doi: 10.3724/SP.J.1146.2013.01003.
|
|
ZHANG Xudong, YANG Jing, HU Liangmei, et al. Human activity recognition using Multi-Layered motion history images with Time-Of-Flight camera[J]. Journal of Electronic & Information Technology, 2014, 36(5): 1139-1144. doi: 10.3724/SP.J.1146.2013.01003.
|
| [3] |
WAN Jun, RUAN Qiuqi, LI Wei, et al. One-shot learning gesture recognition from RGB-D Data using Bag of Features [J]. Journal of Machine Learning Research, 2013, 14: 2549-2582.
|
| [4] |
WAN Jun, GUO Guodong, and LI S Z. Explore efficient local features from RGB-D data for one-shot learning gesture recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(8): 1626-1639. doi: 10.1109/ TPAMI.2015.251347.
|
| [5] |
HERNANDEZ-VELA A, BAUTISTA M A, PEREZ-SALA X, et al. Probability-based dynamic time warping and bag-of- visual-and-depth -words for human gesture recognition in RGB-D[J]. Pattern Recognition Letters, 2014, 50[JL2]: 112-121. doi: 10.1016/j.patrec.2013.09.009.
|
| [6] |
MONNIER C, GERMAN S, and OST A. A multi-scale boosted detector for efficient and robust gesture recognition [C]. Proceedings of the European Conference on Computer Vision Workshops, Zurich, 2014: 491-502. doi: 10.1007/978- 3-319-16178-5_34.
|
| [7] |
PFISTER T, CHARLES J, and ZISSERMAN A. Domain- adaptive discriminative one-shot learning of gestures[C]. Proceedings of the European Conference on Computer Vision, Zurich, 2014: 814-829. doi: 10.1007/978-3-319-10599-4_52.
|
| [8] |
YANG H D and LEE S W. Simultaneous spotting of signs and fingerspellings based on hierarchical conditional random fields and boostmap embeddings[J]. Pattern Recognition, 2010, 43(8): 2858-2870. doi: 10.1016/ j.patcog.2010.03.007.
|
| [9] |
FANELLO S R, GORI I, METTA G, et al. Keep it simple and sparse: Real-time action recognition[J]. Journal of Machine Learning Research, 2013, 14: 2617-2640.
|
| [10] |
LUI Yuiman. Human gesture recognition on product manifolds[J]. Journal of Machine Learning Research, 2012, 13: 3297-3321.
|
| [11] |
KONECNY J and HAGARA M. One-shot-learning gesture recognition using HOG-HOF Features[J]. Journal of Machine Learning Research, 2014, 15: 2513-2532.
|
| [12] |
JADOOKI S, MOHAMAD D, TANZILA S, et al. Fused features mining for depth-based hand gesture recognition to classify blind human communication[J]. Neural Computing and Applications, 2016: 1-10. doi: 10.1007/s00521-016- 2244- 5.
|
| [13] |
DOMINIO F, DONADEO M, and ZANUTTIGH P. Combining multiple depth-based descriptors for hand gesture recognition[J]. Pattern Recognition Letters, 2014, 50: 101-111. doi: 10.1016/j.patrec.2013.10.010.
|
| [14] |
LIANG Hui, YUAN Junsong, and THALMANN D. 3D fingertip and palm tracking in depth image sequences[C]. Proceedings of the ACM International Conference on Multimedia, Nara, 2012: 785-788. doi: 10.1145/2393347. 2396312.
|
| [15] |
JIANG Hairong, DUERSTOCK B S, and WACHS J P. A machine vision-based gestural interface for people with upper extremity physical impairments[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2014, 44(5): 630-641. doi: 10.1109/TSMC.2013.2270226.
|
| [16] |
CHENG Hong, DAI Zhongjun, LIU Zicheng, et al. An image-to-class dynamic time warping approach for both 3D static and trajectory hand gesture recognition[J]. Pattern Recognition, 2016, 55: 137-147. doi: 10.1016/j.patcog.2016.01. 011.
|
| [17] |
WANG Chong, LIU Zhong, and CHAN Shingchow. Superpixel-based hand gesture recognition with Kinect depth camera[J]. IEEE Transactions on Multimedia, 2015, 17(1): 29-39. doi: 10.1109/TMM.2014.2374357.
|
| [18] |
CHEMYSHOV V and MESTETSKIY L. Real-time hand detection using continuous skeletons[J]. Pattern Recognition and Image Analysis, 2016, 26(2): 368-373. doi: 10.1134/ S1054661816020048.
|
| [19] |
南栋, 毕笃彦, 查宇飞, 等. 基于参数估计的无参考型图像质量评价算法[J]. 电子与信息学报, 2013, 35(9): 2066-2072. doi: 10.3724/SP.J.1146.2012.01652.
|
|
NAN Dong, BI Duyan, ZHA Yufei, et al. A no-reference image quality assessment method based on parameter estimation[J]. Journal of Electronic & Information Technology, 2013, 35(9): 2066-2072. doi: 10.3724/SP.J.1146. 2012.01652.
|
| [20] |
JONES M J and REHG J M. Statistical color models with application to skin detection[J]. International Journal of Computer Vision, 2002, 46(1): 81-96. doi: 10.1023/A: 1013200319198.
|
| [21] |
CHALEARN. ChaLearn Gesture Dataset (CGD2011) [DB/ OL]. http://gesture.chalearn.org/data, 2011.
|
| [22] |
STERGIOPOULOU E, SGOUROPOULOS K, NIKOLAOU N, et al. Real time hand detection in a complex background [J]. Engineering Applications of Artificial Intelligence, 2014, 35[JL3]: 54-70. doi: 10.1016/j.engappai.2014.06.006.
|
| [23] |
LIN Jia, RUAN Xiaogang, YU Naigong, et al. One-shot learning gesture recognition based on improved 3D SMoSIFT feature descriptor from RGB-D videos[C]. Proceedings of the Chinese Control and Decision Conference, Qingdao, 2015: 4911-4916. doi: 10.1109/CCDC.2015.7162803.
|
| [24] |
FARNEBACK G. Two-frame motion estimation based on polynomial expansion[C]. Proceedings of the Scandinavian Conference on Image Analysis, Halmstad, 2003: 363-370. doi: 10.1007/3-540-45103-X_50.
|
| [25] |
SHI J and TOMASI G. Good feature to track[C]. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, 1994: 593-600. doi: 10.1109/CVPR.1994.323794.
|
| [26] |
GONZALEZ R C and WOODS R E. Digital Image Processing[M]. Beijing: Publishing House of Electronic Industry, 2008: 543-545.
|
| [27] |
CALABI L and HARNETT W E. Shape recognition, prairie fires, convex deficiencies and skeletons[J]. The American Mathematical Monthly, 1968, 75(4): 335-342. doi: 10.2307/ 2313409.
|
| [28] |
DIJKSTRA E W. A note on two problems in connexion with graphs[J]. Numerische Mathematik, 1959, 1(1): 269-271. doi: 10.1007/BF01386390.
|
|
|
|
