A Hierarchical Clustering Method Based on the Threshold of Semantic Feature in Big Data
Luo En-tao①② Wang Guo-jun①
①(School of Information Science and Engineering, Central South University, Changsha 410083, China) ②(School of Electronics and Information Engineering, Hunan University of Science and Engineering, Yongzhou 425006, China)
The type and scale of data has been promoted with a hitherto unknown speed by the emerging services including cloud computing, health care, street view services recommendation system and so on. However, the surge in the volume of data may lead to many common problems, such as the representability, reliability and handlability of data. Therefore, how to effectively handle the relationship between the data and the analysis to improve the efficiency of classification of the data and establish the data clustering analysis model has become an academic and business problem, which needs to be solved urgently. A hierarchical clustering method based on semantic feature is proposed. Firstly, the data should be trained according to the semantic features of data, and then is used the training result to process hierarchical clustering in each subset; finally, the density center point is produced. This method can improve the efficiency and accuracy of data clustering. This algorithm is of low complexity about sampling, high accuracy of data analysis and good judgment. Furthermore, the algorithm is easy to realize.
罗恩韬,王国军. 大数据中一种基于语义特征阈值的层次聚类方法[J]. 电子与信息学报, 2015, 37(12): 2795-2801.
Luo En-tao, Wang Guo-jun. A Hierarchical Clustering Method Based on the Threshold of Semantic Feature in Big Data. JEIT, 2015, 37(12): 2795-2801.
Cheng Xue-qi, Jin Xiao-long, Wang Yuan-zhuo, et al.. Survey on big data system and analytic technology[J]. Journal of Software, 2014, 25(9): 1889-1909.
[2]
Du Y, He Y, Tian Y, et al.. Microblog bursty topic detection based on user relationship[C]. IEEE 6th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, China, 2011, 1: 260-263.
Sun Ji-gui, Liu Jie, and Zhao Lian-yu. Clustering algorithms research[J]. Journal of Software, 2008, 19(1): 48-61.
[4]
Choromanska A, Jebara T, Kim H, et al.. Fast spectral clustering via the nystr?m method[C]. Proceedings of the 24th International Conference, Algorithmic Learning Theory 2013, Singapore, 2013: 367-381.
[5]
Hearn T A and Reichel L. Fast computation of convolution operations via low-rank approximation[J]. Applied Numerical Mathematics, 2014, (75): 136-153.
[6]
Gajjar M R, Sreenivas T V, and Govindarajan R. Fast computation of Gaussian likelihoods using low-rank matrix approximations[C]. 2011 IEEE Workshop on Signal Processing Systems (SiPS), Beirut, Lebanon, 2011: 322-327.
Cui Ying-an, Li Xue, Wang Zhi-xiao, et al.. Sampling online social media big data based multi stage cluster method[J]. Journal of Software, 2014, 25(4): 781-796.
[8]
Chen W Y, Song Y, Bai H, et al.. Parallel spectral clustering in distributed systems[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(3): 568-586.
Ding Shi-fei, Jia Hong-jie, and Shi Zhong-zhi. Spectral clustering algorithm based on adaptive Nyström sampling for big data analysis[J]. Journal of Software, 2014, 25(9): 2037-2049.
[10]
Chen X and Cai D. Large scale spectral clustering with landmark-based representation[C]. Proceedings of the 25th AAAI Conference on Artificaial Inteligence, San Francisco, USA, 2011: 313-318.
Ci Xiang, Ma You-zhong, and Meng Xiao-feng. Method for Top-K query on big data in cloud[J]. Journal of Software, 2014, 25(4): 813-825.
[12]
Horng S J, Su M Y, Chen Y H, et al.. A novel intrusion detection system based on hierarchical clustering and support vector machines[J]. Expert Systems with Applications, 2011, 38(1): 306-313.
[13]
Bahmani B, Moseley B, Vattani A, et al.. Scalable k- means++[J]. Proceedings of the VLDB Endowment, 2012, 5(7): 622-633.
[14]
Zhang X and You Q. Clusterability analysis and incremental sampling for Nyström extension based spectral clustering[C]. IEEE 11th International Conference on Data Mining (ICDM) , Vancouver, Canada, 2011: 942-951.
[15]
Zhang K and Kwok J T. Clustered Nyström method for large scale manifold learning and dimension reduction[J]. IEEE Transactions on Neural Networks, 2010, 21(10): 1576-1587.
[16]
Vlachou A, Doulkeridis C, Kotidis Y, et al.. Monochromatic and bichromatic reverse top-k queries[J]. IEEE Transactions on Knowledge and Data Engineering, 2011, 23(8): 1215-1229.