大数据中一种基于语义特征阈值的层次聚类方法

doi:10.11999/JEIT150422

Abstract
Figure/Table
References (16)
Related Citation (15)

Download: PDF (2192 KB)
Export: BibTeX | EndNote (RIS)

Abstract

The type and scale of data has been promoted with a hitherto unknown speed by the emerging services including cloud computing, health care, street view services recommendation system and so on. However, the surge in the volume of data may lead to many common problems, such as the representability, reliability and handlability of data. Therefore, how to effectively handle the relationship between the data and the analysis to improve the efficiency of classification of the data and establish the data clustering analysis model has become an academic and business problem, which needs to be solved urgently. A hierarchical clustering method based on semantic feature is proposed. Firstly, the data should be trained according to the semantic features of data, and then is used the training result to process hierarchical clustering in each subset; finally, the density center point is produced. This method can improve the efficiency and accuracy of data clustering. This algorithm is of low complexity about sampling, high accuracy of data analysis and good judgment. Furthermore, the algorithm is easy to realize.

Key words： Big data Data extraction Hierarchical clustering Clustering analysis

Received: 10 April 2015 Published: 01 November 2015

PACS:

TP393

Fund:

The National Natural Science Foundation of China (60173037, 6272496, 61272151); The Hunan Provincial Education Departmant of China (2015C0589); Key Discipline Project of Hunan University of Science and Engineering

Corresponding Authors: Luo En-tao E-mail: cs_entaoluo@csu.edu.cn

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Luo En-tao
	Wang Guo-jun

Cite this article:

Luo En-tao,Wang Guo-jun. A Hierarchical Clustering Method Based on the Threshold of Semantic Feature in Big Data[J]. JEIT, 2015, 37(12): 2795-2801.

URL:

http://jeit.ie.ac.cn/EN/10.11999/JEIT150422 OR http://jeit.ie.ac.cn/EN/Y2015/V37/I12/2795

[1]	程学旗, 靳小龙, 王元卓, 等. 大数据系统和分析技术综述[J]. 软件学报, 2014, 25(9): 1889-1909.
	Cheng Xue-qi, Jin Xiao-long, Wang Yuan-zhuo, et al.. Survey on big data system and analytic technology[J]. Journal of Software, 2014, 25(9): 1889-1909.
[2]	Du Y, He Y, Tian Y, et al.. Microblog bursty topic detection based on user relationship[C]. IEEE 6th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, China, 2011, 1: 260-263.
[3]	孙吉贵, 刘杰, 赵连宇. 聚类算法研究[J]. 软件学报, 2008, 19(1): 48-61.
	Sun Ji-gui, Liu Jie, and Zhao Lian-yu. Clustering algorithms research[J]. Journal of Software, 2008, 19(1): 48-61.
[4]	Choromanska A, Jebara T, Kim H, et al.. Fast spectral clustering via the nystr?m method[C]. Proceedings of the 24th International Conference, Algorithmic Learning Theory 2013, Singapore, 2013: 367-381.
[5]	Hearn T A and Reichel L. Fast computation of convolution operations via low-rank approximation[J]. Applied Numerical Mathematics, 2014, (75): 136-153.
[6]	Gajjar M R, Sreenivas T V, and Govindarajan R. Fast computation of Gaussian likelihoods using low-rank matrix approximations[C]. 2011 IEEE Workshop on Signal Processing Systems (SiPS), Beirut, Lebanon, 2011: 322-327.
[7]	崔颖安, 李雪, 王志晓, 等. 社会化媒体大数据多阶段整群抽样方法[J]. 软件学报, 2014, 25(4): 781-796.
	Cui Ying-an, Li Xue, Wang Zhi-xiao, et al.. Sampling online social media big data based multi stage cluster method[J]. Journal of Software, 2014, 25(4): 781-796.
[8]	Chen W Y, Song Y, Bai H, et al.. Parallel spectral clustering in distributed systems[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(3): 568-586.
[9]	丁世飞, 贾洪杰, 史忠植. 基于自适应 Nyström采样的大数据谱聚类算法[J]. 软件学报, 2014, 25(9): 2037-2049.
	Ding Shi-fei, Jia Hong-jie, and Shi Zhong-zhi. Spectral clustering algorithm based on adaptive Nyström sampling for big data analysis[J]. Journal of Software, 2014, 25(9): 2037-2049.
[10]	Chen X and Cai D. Large scale spectral clustering with landmark-based representation[C]. Proceedings of the 25th AAAI Conference on Artificaial Inteligence, San Francisco, USA, 2011: 313-318.
[11]	慈祥, 马友忠, 孟小峰. 一种云环境下的大数据Top-K查询方法[J]. 软件学报, 2014, 25(4): 813-825.
	Ci Xiang, Ma You-zhong, and Meng Xiao-feng. Method for Top-K query on big data in cloud[J]. Journal of Software, 2014, 25(4): 813-825.
[12]	Horng S J, Su M Y, Chen Y H, et al.. A novel intrusion detection system based on hierarchical clustering and support vector machines[J]. Expert Systems with Applications, 2011, 38(1): 306-313.
[13]	Bahmani B, Moseley B, Vattani A, et al.. Scalable k- means++[J]. Proceedings of the VLDB Endowment, 2012, 5(7): 622-633.
[14]	Zhang X and You Q. Clusterability analysis and incremental sampling for Nyström extension based spectral clustering[C]. IEEE 11th International Conference on Data Mining (ICDM) , Vancouver, Canada, 2011: 942-951.
[15]	Zhang K and Kwok J T. Clustered Nyström method for large scale manifold learning and dimension reduction[J]. IEEE Transactions on Neural Networks, 2010, 21(10): 1576-1587.
[16]	Vlachou A, Doulkeridis C, Kotidis Y, et al.. Monochromatic and bichromatic reverse top-k queries[J]. IEEE Transactions on Knowledge and Data Engineering, 2011, 23(8): 1215-1229.