Intraclass-Distance-Sum-Minimization Based Classification Algorithm
WANG Xiaochu①② WANG Shitong① BAO Fang② JIANG Yizhang①
①(School of Digital Media, Jiangnan University, Wuxi 214122, China) ②(Information Fusion Software Engineering Research and Development Center of Jiangsu Province, Jiangyin 214405, China)
Classification algorithm of Support Vector Machine (SVM) is introduced the penalty factor to adjust the overfit and nonlinear problem. The method is beneficial for seeking the optimal solution by allowing a part of samples error classified. But it also causes a problem that error classified samples distribute disorderedly and increase the burden of training. In order to solve the above problems, according to large margin classification thought, based on principles that the intraclass samples must be closer and the interclass samples must be looser, this research proposes a new classification algorithm called Intraclass-Distance-Sum-Minimization (IDSM) based classification algorithm. This algorithm constructs a training model by using principle of minimizing the sum of the intraclass distance and finds the optimal projection rule by analytical method. And then the optimal projection rule is used to samples’ projection transformation to achieve the effect that intraclass intervals are small and the interclass intervals are large. Accordingly, this research offers a kernel version of the algorithm to solve high-dimensional classification problems. Experiment results on a large number of UCI datasets and the Yale face database indicate the superiority of the proposed algorithm.
QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1): 81-106.
[2]
QUINLAN J R. Improved use of continuous attributes in C4.5[J]. Journal of Artificial Intelligence Research, 1996, 4(1): 77-90.
[3]
PENG F, SCHUURMANS D, and WANG S. Augmenting naive Bayes classifiers with statistical language models[J]. Information Retrieval, 2004, 7(3/4): 317-345.
[4]
CHENG J and GREINER R. Comparing Bayesian network classifiers[C]. Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, San Francisco, USA, 1999: 101-108.
[5]
COVER T and HART P. Nearest neighbor pattern classification [J]. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.
[6]
BIJALWAN V, KUMAR V, KUMARI P, et al. KNN based machine learning approach for text and document mining[J]. International Journal of Database Theory and Application, 2014, 7(1): 61-70.
HUANG Jianhua, DING Jianrui, LIU Jiafeng, et al. Citation- kNN algorithm based on locally-weighting[J]. Journal of Electronics & Information Technology, 2013, 35(3): 627-632.
[8]
WELLING M. Fisher linear discriminant analysis[J]. Department of Computer Science, 2008, 16(94): 237-280.
[9]
FUIN N, PEDEMONTE S, ARRIDGE S, et al. Efficient determination of the uncertainty for the optimization of SPECT system design: a subsampled fisher information matrix[J]. IEEE Transactions on Medical Imaging, 2014, 33(3): 618-635.
[10]
DUFRENOIS F. A one-class kernel fisher criterion for outlier detection[J]. IEEE Transactions on Neural Networks & Learning Systems, 2014, 26(5): 982-994.
[11]
VAN Ooyen A and NIENHUIS B. Improving the convergence of the back-propagation algorithm[J]. Neural Networks, 1992, 5(3): 465-471.
PAN Zhouhao, LI Daojing, LIU Bo, et al. Processing of the airborne InSAR data based on the BP algorithm and the time-varying baseline[J] Journal of Electronics & Information Technology, 2014, 36(7): 1585-1591.
[13]
SUYKENS J A K and VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3): 293-300.
HU Wenjun, WANG Shitong, WANG Juan, et al. Fast ensemble of separating hyperplanes for nonlinear classification[J]. Journal of Electronics & Information Technology, 2012, 34(3): 535-542.
[15]
GAO X, LU T, LIU P, et al. A soil moisture classification model based on SVM used in agricultural WSN[C]. 2014 IEEE 7th Joint International, Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, 2014: 432-436.
[16]
RIES C X, RICHTER F, ROMBERG S, et al. Automatic object annotation from weakly labeled data with latent structured SVM[C]. 2014 12th IEEE International Workshop on Content-based Multimedia Indexing (CBMI), Klagenfurt, Austria, 2014: 1-4.
[17]
PLATT J. Fast training of support vector machines using sequential minimal optimization[J]. Advances in Kernel Methods: Support Vector Learning, 1999(3): 185-208.
[18]
JOACHIMS T. Making large scale SVM learning practical[R]. Universit?t Dortmund, 1999.
[19]
CHANG C C and LIN C J. LIBSVM: A library for support vector machines[J]. ACM Transactions on Intelligent Systems & Technology, 2011, 2(3): 389-396.
[20]
MANGASARIAN O L and MUSICANT D R. Lagrangian support vector machines[J]. The Journal of Machine Learning Research, 2001, 1(3): 161-177.
[21]
SEOK K. Semi-supervised regression based on support vector machine[J]. Computer Engineering & Applications, 2014, 25(2): 447-454.
[22]
LENG Y, XU X, and QI G. Combining active learning and semi-supervised learning to construct SVM classifier[J]. Knowledge-Based Systems, 2013, 44(1): 121-131.
[23]
CHEN W J, SHAO Y H, XU D K, et al. Manifold proximal support vector machine for semi-supervised classification[J]. Applied Intelligence, 2014, 40(4): 623-638.
LI Honglian, WANG Chunhua, and YUAN Baozong. An improved SVM: NN-SVM[J]. Chinese Journal of Computers, 2003, 26(8): 1015-1020.
[25]
陈宝林. 最优化理论与算法[M]. 北京: 清华大学出版社, 2005: 281-322.
CHEN Baolin. Optimization Theory and Algorithm[M]. Beijing, Tsinghua University Press, 2005: 281-322.
[26]
YOSHIYAMA K and SAKURAI A. Laplacian minimax probability machine[J]. Pattern Recognition Letters, 2014, 37: 192-200.
[27]
MIGLIORATI G. Adaptive polynomial approximation by means of random discrete least squares[J]. Lecture Notes in Computational Science & Engineering, 2013, 103: 547-554.
[28]
HUANG K, YANG H, KING I, et al. The minimum error minimax probability machine[J]. The Journal of Machine Learning Research, 2004(5): 1253-1286.
[29]
PLAN Y and VERSHYNIN R. Robust 1-bit compressed sensing and sparse logistic regression: A convex programming approach[J]. IEEE Transactions on Information Theory, 2013, 59(1): 482-494.
[30]
ALIZADEN F. Interior point methods in semidefinite programming with applications to combinatorial optimization[J]. SIAM Journal on Optimization, 1995, 5(1): 13-51.
[31]
BOYD S and VANDENBERGHE L. Convex Optimi- zation[M]. Cambridge University Press, 2009: 127- 214.
[32]
边肇祺, 张学工. 模式识别[M]. 北京: 清华大学出版社, 2001: 83-90.
BIAN Zhaoqi and ZHANG Xuegong. Pattern Recognition[M]. Beijing, Tsinghua University Press, 2001: 83-90.
