基于有序编码的核极限学习顺序回归模型

doi:10.11999/JEIT170765

Abstract
Figure/Table
References (0)
Related Citation (15)

Download: PDF (240 KB)
Export: BibTeX | EndNote (RIS)

Abstract Ordinal regression is one of the supervised learning issues, which resides between classification and regression in machine learning fields. There exist many real problems in practice, which can be modeled as ordinal regression problems due to the ordering information between labels. Therefore ordinal regression has received increasing interest by many researchers recently. The Extreme Learning Machine (ELM)-based algorithms are easy to train without iterative algorithm and they can avoid the local optimal solution; meanwhile they reduce the training time compared with other learning algorithms. However, the ELM-based algorithms which are applied to ordinal regression have not been exploited much. This paper proposes a new ordered code-based kernel extreme learning ordinal regression machine to fill this gap, which combines the kernel ELM and error correcting output codes effectively. The model overcomes the problems of how to get high quality feature mappings in ordinal regression and how to avoid setting the number of hidden nodes by manual. To validate the effectiveness of this model, numerous experiments are conducted on a lot of datasets. The experimental results show that the model can improve the accuracy by 10.8% on average compared with traditional ELM-based algorithms and achieve the state- of-the-art performance with the least time.

Key words： Error correct output code Ordinal regression Extreme learning machine Kernel function

Received: 28 July 2017 Published: 14 March 2018

PACS:

TP391

Fund:The National Natural Science Foundation of China (71110107026, 71331005, 91546201, 11671379, 111331012), The Grant of University of Chinese Academy of Sciences (Y55202LY00)

Corresponding Authors: SHI Yong E-mail: yshi@ucas.ac.cn

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	LI Peijia
	SHI Yong
	WANG Huadong
	NIU Lingfeng

Cite this article:

LI Peijia,SHI Yong,WANG Huadong等. Ordered Code-based Kernel Extreme Learning Machine for Ordinal Regression[J]. JEIT, 2018, 40(6): 1287-1293.

URL:

http://jeit.ie.ac.cn/EN/10.11999/JEIT170765 OR http://jeit.ie.ac.cn/EN/Y2018/V40/I6/1287

[1]	NAKOV P, RITTER A, ROSENTHAL S, et al. SemEval- 2016 task 4: Sentiment analysis in Twitter[C]. International Workshop on Semantic Evaluation, San Diego, USA, 2016: 1-18. doi: 10.18653/v1/S16-1028.
[2]	TIAN Q, CHEN S, and TAN X. Comparative study among three strategies of incorporating spatial structures to ordinal image regression[J]. Neurocomputing, 2014, 136: 152-161. doi: 10.1016/j.neucom.2014.01.017.
[3]	CORRENTE S, DOUMPOS M, GRECO S, et al. Multiple criteria hierarchy process for sorting problems based on ordinal regression with additive value functions[J]. Annals of Operations Research, 2017, 251(1/2): 117-139. doi: 10.1007/ s10479-015-1898-1.
[4]	GUTIÉRREZ P A, PÉREZ-ORTIZ M, SANCHEZ- MONEDERO J, et al. Ordinal regression methods: Survey and experimental study[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(1): 127-146. doi: 10.1109/ TKDE.2015.2457911.
[5]	HUANG G B, ZHU Q Y, and SIEW C K. Extreme learning machine: Theory and applications[J]. Neurocomputing, 2006, 70(1): 489-501. doi: 10.1016/j.neucom.2005.12.126.
[6]	RAJASEKARAN S and PAI G A V. Neural Networks, Fuzzy Systems and Evolutionary Algorithms: Synthesis and Applications[M]. Haryana, India: Rajkamal Electric Press, 2017: 151-168.
[7]	CHU W and KEERTHI S S. Support vector ordinal regression[J]. Neural Computation, 2007, 19(3): 792-815. doi: 10.1162/neco.2007.19.3.792.
[8]	HUANG G B, ZHOU H, DING X, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(2): 513-529. doi: 10.1109/TSMCB. 2011.2168604.
[9]	UCAR A, DEMIR Y, and GÜZELI C. A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering[J]. Neural Computing and Applications, 2016, 27(1): 131-142. doi: 10.1007/s00521-014-1569-1.
[10]	徐涛, 郭威, 吕宗磊. 基于快速极限学习机和差分进化的机场噪声预测模型[J]. 电子与信息学报, 2016, 38(6): 1512-1518. doi: 10.11999/JEIT150986.
	XU Tao, GUO Wei, and LÜ Zonglei. Prediction model of airport noise based on fast extreme learning machine and differential evolution[J]. Journal of Electronics & Information Technology, 2016, 38(6): 1512-1518. doi: 10.11999/JEIT 150986.
[11]	GOODFELLOW I, BENGIO Y, and COURVILLE A. Deep Learning[M]. Massachusetts, USA, MIT Press, 2016: 165-480. doi: 10.1038/nature14539.
[12]	DENG W Y, ZHENG Q H, LIAN S, et al. Ordinal extreme learning machine[J]. Neurocomputing, 2010, 74(1): 447-456. doi: 10.1016/j.neucom.2010.08.022.
[13]	RICCARDI A, FERNÁNDEZ-NAVARRO F, and CARLONI S. Cost-sensitive AdaBoost algorithm for ordinal regression based on extreme learning machine[J]. IEEE Transactions on Cybernetics, 2014, 44(10): 1898-1909. doi: 10.1109/TCYB. 2014.2299291.
[14]	HORNIK K, STINCHCOMBE M, and WHITE H. Multilayer feedforward networks are universal approximators[J]. Neural Networks, 1989, 2(5): 359-366. doi: 10.1016/0893-6080(89) 90020-8.
[15]	HUANG G B and BABRI H A. Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions[J]. IEEE Transactions on Neural Networks, 1998, 9(1): 224-229. doi: 10.1109/72.655045.
[16]	HUANG G B, CHEN L, and SIEW C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes[J]. IEEE Transactions on Neural Networks, 2006, 17(4): 879-892. doi: 10.1109/TNN. 2006.875977.
[17]	HUANG G B. Learning capability and storage capacity of two-hidden-layer feedforward networks[J]. IEEE Transactions on Neural Networks, 2003, 14(2): 274-281. doi: 10.1109/TNN.2003.809401.
[18]	BARTLETT P L. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network[J]. IEEE Transactions on Information Theory, 1998, 44(2): 525-536. doi: 10.1109/18.661502.
[19]	TANG J, DENG C, and HUANG G B. Extreme learning machine for multilayer perceptron[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 809-821. doi: 10.1109/TNNLS.2015.2424995.
[20]	HOERL A E and KENNARD R W. Ridge regression: Biased estimation for nonorthogonal problems[J]. Technometrics, 1970, 12(1): 55-67. doi: 10.1080/00401706.1970.10488634.
[21]	ALLWEIN E L, SCHAPIRE R E, and SINGER Y. Reducing multiclass to binary: A unifying approach for margin classifiers[J]. Journal of Machine Learning Research, 2000, 1(12): 113-141. doi: 10.1162/15324430152733133.
[22]	雷蕾, 王晓丹, 罗玺, 等. ECOC多类分类研究综述[J]. 电子学报, 2014, 42(9): 1794-1800. doi: 10.3969/j.issn.0372-2112. 2014.09.020.
	LEI Lei, WANG Xiaodan, LUO Xi, et al. An overview of multi-classification based on error-correcting output codes[J]. Acta Electronica Sinica, 2014, 42(9): 1794-1800. doi: 10.3969 /j.issn.0372-2112.2014.09.020.
[23]	HUANG G, HUANG G B, SONG S, et al. Trends in extreme learning machines: A review[J]. Neural Networks, 2015, 61: 32-48. doi: 10.1016/j.neunet.2014.10.001.
[24]	LIU Q, HE Q, and SHI Z. Extreme support vector machine classifier[C]. 12th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Osaka, Japan, 2008: 222-233. doi: 10.1007/978-3-540-68125-0_21.
[25]	FRÉNAY B and VERLEYSEN M. Using SVMs with randomised feature spaces: an extreme learning approach[C]. European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, 2010: 315-320.
[26]	HUANG G B, DING X, and ZHOU H. Optimization method based extreme learning machine for classification[J]. Neurocomputing, 2010, 74(1): 155-163. doi: 10.1016/j.neucom. 2010.02.019.
[27]	CHU W and GHAHRAMANI Z. Gaussian processes for ordinal regression[J]. Journal of Machine Learning Research, 2005, 6(7): 1019-1041.
[28]	BACCIANELLA S, ESULI A, and SEBASTIANI F. Evaluation measures for ordinal regression[C]. The Ninth International Conference on Intelligent Systems Design and Applications, Pisa, Italy, 2009: 283-287. doi: 10.1109/ISDA. 2009.230.