最小化类内距离和分类算法

doi:10.11999/JEIT150633

Abstract
Figure/Table
References (32)
Related Citation (15)

Download: PDF (690 KB)
Export: BibTeX | EndNote (RIS)

Abstract

Classification algorithm of Support Vector Machine (SVM) is introduced the penalty factor to adjust the overfit and nonlinear problem. The method is beneficial for seeking the optimal solution by allowing a part of samples error classified. But it also causes a problem that error classified samples distribute disorderedly and increase the burden of training. In order to solve the above problems, according to large margin classification thought, based on principles that the intraclass samples must be closer and the interclass samples must be looser, this research proposes a new classification algorithm called Intraclass-Distance-Sum-Minimization (IDSM) based classification algorithm. This algorithm constructs a training model by using principle of minimizing the sum of the intraclass distance and finds the optimal projection rule by analytical method. And then the optimal projection rule is used to samples’ projection transformation to achieve the effect that intraclass intervals are small and the interclass intervals are large. Accordingly, this research offers a kernel version of the algorithm to solve high-dimensional classification problems. Experiment results on a large number of UCI datasets and the Yale face database indicate the superiority of the proposed algorithm.

Key words： Support Vector Machine (SVM) Penalty factor Large margin classification thought Sum of intraclass distance Projection rule

Received: 27 May 2015 Published: 19 November 2015

PACS:

TP391.41

Fund:

The National Natural Science Foundation of China (61170122, 61272210)

Corresponding Authors: WANG Xiaochu E-mail: icnice@yeah.net

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	WANG Xiaochu
	WANG Shitong
	BAO Fang
	JIANG Yizhang

Cite this article:

WANG Xiaochu,WANG Shitong,BAO Fang等. Intraclass-Distance-Sum-Minimization Based Classification Algorithm[J]. JEIT, 2016, 38(3): 532-540.

URL:

http://jeit.ie.ac.cn/EN/10.11999/JEIT150633 OR http://jeit.ie.ac.cn/EN/Y2016/V38/I3/532

[1]	QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1): 81-106.
[2]	QUINLAN J R. Improved use of continuous attributes in C4.5[J]. Journal of Artificial Intelligence Research, 1996, 4(1): 77-90.
[3]	PENG F, SCHUURMANS D, and WANG S. Augmenting naive Bayes classifiers with statistical language models[J]. Information Retrieval, 2004, 7(3/4): 317-345.
[4]	CHENG J and GREINER R. Comparing Bayesian network classifiers[C]. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, San Francisco, USA, 1999: 101-108.
[5]	COVER T and HART P. Nearest neighbor pattern classification [J]. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.
[6]	BIJALWAN V, KUMAR V, KUMARI P, et al. KNN based machine learning approach for text and document mining[J]. International Journal of Database Theory and Application, 2014, 7(1): 61-70.
[7]	黄剑华, 丁建睿, 刘家锋, 等. 基于局部加权的Citation-kNN算法[J]. 电子与信息学报, 2013, 35(3): 627-632.
	HUANG Jianhua, DING Jianrui, LIU Jiafeng, et al. Citation- kNN algorithm based on locally-weighting[J]. Journal of Electronics & Information Technology, 2013, 35(3): 627-632.
[8]	WELLING M. Fisher linear discriminant analysis[J]. Department of Computer Science, 2008, 16(94): 237-280.
[9]	FUIN N, PEDEMONTE S, ARRIDGE S, et al. Efficient determination of the uncertainty for the optimization of SPECT system design: a subsampled fisher information matrix[J]. IEEE Transactions on Medical Imaging, 2014, 33(3): 618-635.
[10]	DUFRENOIS F. A one-class kernel fisher criterion for outlier detection[J]. IEEE Transactions on Neural Networks & Learning Systems, 2014, 26(5): 982-994.
[11]	VAN Ooyen A and NIENHUIS B. Improving the convergence of the back-propagation algorithm[J]. Neural Networks, 1992, 5(3): 465-471.
[12]	潘舟浩, 李道京, 刘波, 等. 基于BP算法和时变基线的机载InSAR数据处理方法研究[J]. 电子与信息学报, 2014, 36(7): 1585-1591.
	PAN Zhouhao, LI Daojing, LIU Bo, et al. Processing of the airborne InSAR data based on the BP algorithm and the time-varying baseline[J] Journal of Electronics & Information Technology, 2014, 36(7): 1585-1591.
[13]	SUYKENS J A K and VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3): 293-300.
[14]	胡文军, 王士同, 王娟, 等. 非线性分类的分割超平面快速集成方法[J]. 电子与信息学报, 2012, 34(3): 535-542.
	HU Wenjun, WANG Shitong, WANG Juan, et al. Fast ensemble of separating hyperplanes for nonlinear classification[J]. Journal of Electronics & Information Technology, 2012, 34(3): 535-542.
[15]	GAO X, LU T, LIU P, et al. A soil moisture classification model based on SVM used in agricultural WSN[C]. 2014 IEEE 7th Joint International, Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, 2014: 432-436.
[16]	RIES C X, RICHTER F, ROMBERG S, et al. Automatic object annotation from weakly labeled data with latent structured SVM[C]. 2014 12th IEEE International Workshop on Content-based Multimedia Indexing (CBMI), Klagenfurt, Austria, 2014: 1-4.
[17]	PLATT J. Fast training of support vector machines using sequential minimal optimization[J]. Advances in Kernel Methods: Support Vector Learning, 1999(3): 185-208.
[18]	JOACHIMS T. Making large scale SVM learning practical[R]. Universit?t Dortmund, 1999.
[19]	CHANG C C and LIN C J. LIBSVM: A library for support vector machines[J]. ACM Transactions on Intelligent Systems & Technology, 2011, 2(3): 389-396.
[20]	MANGASARIAN O L and MUSICANT D R. Lagrangian support vector machines[J]. The Journal of Machine Learning Research, 2001, 1(3): 161-177.
[21]	SEOK K. Semi-supervised regression based on support vector machine[J]. Computer Engineering & Applications, 2014, 25(2): 447-454.
[22]	LENG Y, XU X, and QI G. Combining active learning and semi-supervised learning to construct SVM classifier[J]. Knowledge-Based Systems, 2013, 44(1): 121-131.
[23]	CHEN W J, SHAO Y H, XU D K, et al. Manifold proximal support vector machine for semi-supervised classification[J]. Applied Intelligence, 2014, 40(4): 623-638.
[24]	李红莲, 王春花, 袁保宗. 一种改进的支持向量机NN- SVM[J]. 计算机学报, 2003, 26(8): 1015-1020.
	LI Honglian, WANG Chunhua, and YUAN Baozong. An improved SVM: NN-SVM[J]. Chinese Journal of Computers, 2003, 26(8): 1015-1020.
[25]	陈宝林. 最优化理论与算法[M]. 北京: 清华大学出版社, 2005: 281-322.
	CHEN Baolin. Optimization Theory and Algorithm[M]. Beijing, Tsinghua University Press, 2005: 281-322.
[26]	YOSHIYAMA K and SAKURAI A. Laplacian minimax probability machine[J]. Pattern Recognition Letters, 2014, 37: 192-200.
[27]	MIGLIORATI G. Adaptive polynomial approximation by means of random discrete least squares[J]. Lecture Notes in Computational Science & Engineering, 2013, 103: 547-554.
[28]	HUANG K, YANG H, KING I, et al. The minimum error minimax probability machine[J]. The Journal of Machine Learning Research, 2004(5): 1253-1286.
[29]	PLAN Y and VERSHYNIN R. Robust 1-bit compressed sensing and sparse logistic regression: A convex programming approach[J]. IEEE Transactions on Information Theory, 2013, 59(1): 482-494.
[30]	ALIZADEN F. Interior point methods in semidefinite programming with applications to combinatorial optimization[J]. SIAM Journal on Optimization, 1995, 5(1): 13-51.
[31]	BOYD S and VANDENBERGHE L. Convex Optimi- zation[M]. Cambridge University Press, 2009: 127- 214.
[32]	边肇祺, 张学工. 模式识别[M]. 北京: 清华大学出版社, 2001: 83-90.
	BIAN Zhaoqi and ZHANG Xuegong. Pattern Recognition[M]. Beijing, Tsinghua University Press, 2001: 83-90.