Design of An Exponential-like Kernel Function Based on Multi-scale Resampling
HU Zhanwei①② JIAO Liguo③ XU Shengjin① HUANG Yong②
①(Fluid Dynamics Institute, School of Aerospace, Tsinghua University, Beijing 100084, China) ②(China Aerodynamics Research & Development Center, Mianyang 621000, China) ③(Daqing Area Flood Control Engineering Management Office, Daqing 163311, China)
Based on multi-scale resampling, an Exponential-Like Kernel (ELK) function is designed, and evaluated with local feature extraction in kernel regression and Support Vector Machine (SVM) classification. The ELK is a one-parameter kernel, whose distribution is controlled only by the resolution of analysis. With block and Doppler noisy signals, Nadaraya-Watson regression with ELK mainly shows more noise and step error than with Gaussian kernel, it also has better precision and is more robust than LOcally WEighted Scatterplot Smoothing (LOWESS). Data sets from the UCI Machine Learning Repository used in SVM test demonstrate that, ELK has nearly equal classification accuracy as RBF does, and its locality results in more detailed margin hyperplanes, in consequence, a big number of support vectors in low classification accuracy situation. Moreover, the insensitivity?of ELK to the adjustive coefficient in kernel methods shows the potential to facilitate the parameter optimization progress. ELK, as a single parameter kernel with significant locality, is hopefully to be extensively used in relative kernel methods.
胡站伟,焦立国,徐胜金,黄勇. 基于多尺度重采样思想的类指数核函数构造[J]. 电子与信息学报, 2016, 38(7): 1689-1695.
HU Zhanwei, JIAO Liguo, XU Shengjin, HUANG Yong. Design of An Exponential-like Kernel Function Based on Multi-scale Resampling. JEIT, 2016, 38(7): 1689-1695.
SMOLA A J and SCHLKOPF B. On a kernel-based method for pattern recognition, regression, approximation, and operator inversion[J]. Lgorithmica, 1998, 22(1): 211-231. doi: 10.1007/PL00013831.
[2]
KOHLER M, SCHINDLER A, and SPERLICH S. A review and comparison of bandwidth selection methods for kernel regression [J]. International Statistical Review, 2014, 82(2): 243-274. doi: 10.1111/insr.12039.
[3]
DAS D, DEVI R, PRASANNA S, et al. Performance comparison of online handwriting recognition system for assamese language based on HMM and SVM modelling[J]. International Journal of Computer Science & Information Technology, 2014, 6(5): 87-95. doi: 10.5121/csit.2014.4717.
[4]
HASTIE T and LOADER C. Local regression: Automatic kernel carpentry[J]. Statistical Science, 1993, 8(2): 120-143. doi: 10.1214/ss/1177011002.
[5]
SCHÖLKOPF B, SMOLA A, and MÜLLER K R. Nonlinear component analysis as a kernel eigenvalue problem[J]. Neural Computation, 1998, 10(5): 1299-1319. doi: 10.1162/ 089976698300017467.
[6]
BUCAK S S, JIN R, and JAIN A K. Multiple kernel learning for visual object recognition: A review [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(7): 1354-1369. doi: 10.1109/TPAMI.2013.212.
[7]
BACH F R, LANCKRIET G R, and JORDAN M I. Multiple kernel learning, conic duality, and the SMO algorithm[C]. Proceedings of the Twenty-first International Conference on Machine Learning, Banff, Canada, 2004: 1-6. doi: 10.1145/1015330.1015424.
WU Tao, HE Hangen, and HE Mingke. Interpolation based kernel function’s construction[J]. Chinese Journal of Computers, 2003, 26(8): 990-996.
[9]
JAIN P, KULIS B, and DHILLON I S. Inductive regularized learning of kernel functions[C]. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, Canada, 2010: 946-954.
[10]
ZHANG L, ZHOU W, and JIAO L. Wavelet support vector machine[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2004, 34(1): 34-39. doi:10.1109/TSMCB.2003.811113.
[11]
NADARAYA E A. On estimating regression[J]. Theory of Probability & Its Applications, 1964, 9(1): 141-142. doi: 10.1137/1109020.
WATSON G S. Smooth regression analysis[J]. Sankhyā: The Indian Journal of Statistics, Series A, 1964, 26(4): 359-372.
[14]
CRISTIANINI N and SHAWE-TAYLOR J. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods[M]. Cambridge: Cambridge University Press, 2000: 93-112.
[15]
VLADIMIR V N and VAPNIK V. Statistical Learning Theory[M]. New York: Wiley, 1998: 293-394.
[16]
CHANG C C and LIN C J. LIBSVM: A library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 27. doi: 10.1145 /1961189.1961199.
[17]
REN Y and BAI G. Determination of optimal SVM parameters by using GA/PSO[J]. Journal of Computers, 2010, 5(8): 1160-1168. doi:10.4304/jcp.5.8.1160-1168.
[18]
SARAFIS I, DIOU C, and TSIKRIKA T. Weighted SVM from click through data for image retrieval[C]. 2014 IEEE International Conference on Image Processing (ICIP 2014), Paris, 2014: 3013-3017. doi: 10.1109/ICIP.2014.7025609.
[19]
SONKA M, HLAVAC V, and BOYLE R. Image Processing, Analysis, and Machine Vision[M]. Kentucky: Cengage Learning, 2014: 257-749. doi: 10.1007/978-1-4899-3216-7.
[20]
SELLAM V and JAGADEESAN J. Classification of normal and pathological voice using SVM and RBFNN[J]. Journal of Signal and Information Processing, 2014, 5(1): 1-7. doi: 10. 4236/jsip.2014.51001.
[21]
高晋占. 微弱信号检测[M]. 北京: 清华大学出版社有限公司, 2004: 154-299.
GAO Jinzhan. Weak Signal Detection[M]. Beijing: Tsinghua University Press Ltd., 2004: 154-299.
[22]
MERCER J. Functions of positive and negative type, and their connection with the theory of integral equations[J]. Philosophical Transactions of the Royal Society of London Series A, Containing Papers of a Mathematical or Physical Character, 1909, 209(456): 415-446. doi: 10.1098/rsta.1909. 0016.
[23]
MARRON J and CHUNG S. Presentation of smoothers: The family approach[J]. Computational Statistics, 2001, 16(1): 195-207. doi: 10.1007/s001800100059.
[24]
LICHMAN M. UCI machine learning repository[OL]. http://archive.ics.uci.edu/ml. 2013.
