Transfer learning usually focuses on dealing with small training set in target domain by sharing knowledge generated from source ones, in which one main challenge is divergence metric of distributed samples between training and test data. In order to deal with “negative transfer” problem caused by improper auxiliary sample selections in source domains, this paper presents a modified covariate-shift multi-source ensemble method with transferability criterion. Firstly, transferability metric of auxiliary samples is defined by joint density estimation in accordance with co-variant transfer principles from source to target, so that the coherency of data distributions is verified. After that, whether transfer learning occurs or not should be determined after evaluating transferability metric in different sources to boost accuracy. Finally, experiments on Caltech256 using GIST demonstrate effectiveness and efficiency in the proposed approach and discussions of performance under diverse selections from auxiliary samples and source domains are presented as well. Experimental results show that the proposed method can sufficiently hold back “negative transfer” for better learnability in transfer style.
Tommasi T, OrabonaF, and Caputo B. Learning categories from few examples with multi model knowledge transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 928-941.
[2]
Yao Y and Doretto G. Boosting for transfer learning with multiple sources[C]. Proceedings of Computer Vision and Pattern Recognition, San Francisco, 2010: 1855-1862.
[3]
Long M S, Wang J M, Ding G G, et al.. Adaptation regularization a general framework for transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5): 1076-1089.
[4]
Lin D, An X, and Zhang J. Double-bootstrapping source data selection for instance-based transfer learning[J]. Pattern Recognition Letters, 2013, 34(11): 1279-1285.
[5]
Kuzborskij I, Orabona F, and Caputo B. From N to N+1: multiclass transfer incremental learning[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 3358-3365.
[6]
Zhu Y, Chen Y Q, Lu Z Q, et al.. Heterogeneous transfer learning for image classification[C]. Proceedings of AAAI Conference on Artificial Intelligence, San Francisco, 2011: 1304-1309.
Zhang Qian, Li Ming, Wang Xue-song, et al.. Instance-based transfer learning for multi-source domains[J]. Acta Automatica Sinica, 2014, 40(6): 1176-1183.
[8]
Pang J, Huang Q, Yan S, et al.. Transferring boosted detectors towards viewpoint and scene adaptiveness[J]. IEEE Transactions on Image Processing, 2011, 20(5): 1388-1400.
[9]
Li G, Qin L, Huang Q, et al.. Treat samples differently: object tracking with semi-supervised online CovBoost[C]. Proceedings of International Conference on Computer Vision, Barcelona, 2011: 627-634.
[10]
Qi G J, Aggarwal C, Rui Y, et al.. Towards cross-category knowledge propagation for learning visual concepts[C]. Proceedings of Computer Vision and Pattern Recognition, Colorado Springs, 2011: 897-904.
[11]
Chu W S, Torre F D, and CohnJ F. Selective transfer machine for personalized facial action unit detection[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 3515-3522.
[12]
Yang S Z, Hou C P, Zhang C S, et al.. Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning[J]. Neural Computing and Applications, 2013, 23(2): 541-559.
Fang Yao-ning, Guo Yun-fei, Ding Xue-tao, et al.. An improved regularized singular value decomposition recommender algorithm based on tag transfer learning[J]. Journal of Electronics & Information Technology, 2013, 35(12): 3046-3050.
[14]
Gopalan R. Learning cross-domain information transfer for location recognition and clustering[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 731-738.
[15]
Luo Y, Liu T L, Tao D C, et al.. Decomposition-based transfer distance metric learning for image classification[J]. IEEE Transactions on Image Processing, 2014, 23(9): 3789-3801.
[16]
Long M S, Wang J M, Ding G G, et al.. Transfer learning with graph co-regularization[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 1805-1818.
Hong Jia-ming, Yin Jian, Huang Yun, et al.. TrSVM: a transfer learning algorithm using domain similarity[J]. Journal of Computer Research and Development, 2011, 48(10): 1823-1830.
[18]
Seah C W, Tsang I W, and Ong Y S. Transfer ordinal label learning[J]. IEEE Transactions on Neural Networkand Learning System, 2013, 24(11): 1863-1876.
Xu Min, Wang Shi-tong, and Shi Ying-zhong. A novel transfer-learning-oriented L2 kernel classifier[J]. Journal of Electronics & Information Technology, 2013, 35(9): 2059-2065.
[20]
Long M S, Wang J M, Ding G G, et al.. Transfer feature learning with joint distribution adaptation[C]. Proceedings of International Conference on Computer Vision, Sydney, 2013: 2200-2207.
[21]
Patricia N and Caputo B. Learning to learn, from transfer learning to domain adaptation: a unifying perspective[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 1442-1449.
[22]
Zhang B, Wang Y, Wang Y, et al.. Stable learning in coding space for multi-class decoding and its extension for multi-class hypothesis transfer learning[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 1075-1081.
[23]
Gretton A, Smola A, Huang J, et al.. Covariate Shift by Kernel Mean Matching[M]. Cambridge: MIT Press, 2009: 131-160.
[24]
Huang P P, Wang G, and Qin S Y. Boosting for transfer learning from multiple data sources[J]. Pattern Recognition Letters, 2012, 33(5): 568-579.
[25]
Nie Q F, Jin L Z, and Fei S M. Probability estimation for multi-class classification using AdaBoost[J]. Pattern Recognition, 2014, 47(12): 3931-3940.
[26]
Sugiyama M, Krauledat M, and Müller K R. Covariate shift adaptation by importance weighted cross validation[J]. The Journal of Machine Learning Research, 2007, 8(1): 985-1005.
[27]
Choi M J, Lim J J, Torralba A, et al.. Exploiting hierarchical context on a large database of object categories[C]. IEEE Conference on Computer Vision & Pattern Recognition, San Frencisco, CA, 2010: 129-136.
[28]
Han Y and Liu G. Biologically inspired task oriented gist model for scene classification[J]. Computer Vision and Image Understanding, 2013, 117(1): 76-95.