|
|
|
| Multi-feature Fusion Based on Semantic Understanding Attention Neural Network for Chinese Text Categorization |
| XIE Jinbao HOU Yongjin KANG Shouqiang LI Baiwei ZHANG Xiao |
| (School of Electrical and Electronic Engineering, Harbin University of Science and Technology, Harbin 150080, China) |
|
|
|
|
Abstract In Chinese text categorization tasks, the locations of the important features in the Chinese texts are disperse and sparse, and the different characteristics of Chinese texts contributes differently for the recognition of their categories. In order to solve the above problems, this paper proposes a multi-feature fusion model Three Convolutional neural network paths and Long short term memory path fused with Attention neural network path (3CLA) for Chinese text categorization, which is based on Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) and semantic understanding attention neural networks. The model first uses text preprocessing to finish the segmentation and vectorization of the Chinese text. Then, through the embedding layer, the input data are sent to the CNN path, the LSTM path and the attention path respectively to extract text features of different levels and different characteristics. Finally, the text features are fused by the fusion layer and classified by the classifier. Based on the Chinese corpus, the text classification experiment is carried out. The results of the experiments show that compared with the CNN structure model and the LSTM structure model, the proposed algorithm model improves the recognition ability of Chinese text categories by up to about 8%.
|
|
Received: 17 August 2017
Published: 15 March 2018
|
|
|
| Fund:The Overseas Scholars Fund Project of Heilongjiang Province (1253HQ019) |
|
Corresponding Authors:
HOU Yongjin
E-mail: hyj1516@163.com
|
|
|
|
| [1] |
孙晓, 彭晓琪, 胡敏, 等. 基于多维扩展特征与深度学习的微博短文本情感分析[J]. 电子与信息学报, 2017, 39(9): 2048-2055. doi: 10.11999/JEIT160975.
|
|
SUN Xiao, PENG Xiaoqi, HU Min, et al. Extended multi- modality features and deep learning based Microblog short text sentiment analysis[J]. Journal of Electronics & Information Technology, 2017, 39(9): 2048-2055. doi: 10.11999/JEIT160975.
|
| [2] |
KIM Yoon. Convolutional neural networks for sentence classification[C]. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014: 1746-1751.
|
| [3] |
PHAM Ngocquan, KRUSZEWSKI German, and BOLEDA Gemma. Convolutional neural network language models[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1153-1162.
|
| [4] |
陈钊, 徐睿峰, 桂林, 等. 结合卷积神经网络和词语情感序列特征的中文情感分析[J]. 中文信息学报, 2015, 29(6): 172-178. doi: 10.3969/j.issn.1003-0077.2015.06.023.
|
|
CHEN Zhao, XU Ruifeng, GUI Lin, et al. Combining convolutional neural networks and word sentiment sequence features for Chinese text sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 172-178. doi: 10.3969/j.issn.1003-0077.2015.06.023.
|
| [5] |
刘龙飞, 杨亮, 张绍武, 等. 基于卷积神经网络的微博情感倾向性分析[J]. 中文信息学报, 2015, 29(6): 159-165. doi: 10.3969/j.issn.1003-0077.2015.06.021.
|
|
LIU Longfei, YANG Liang, ZHANG Shaowu, et al. Convolutional neural networks for Chinese micro-blog sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 159-165. doi: 10.3969/j.issn.1003- 0077.2015.06.021.
|
| [6] |
ZHANG Ye, MARSHALL Iain, and WALLACE B C. Rational-augmented convolutional neural networks for text classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 759-804.
|
| [7] |
MIYAMOTO Yasumasa and CHO Kyunghyun. Gated word-character recurrent language model[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1992-1997.
|
| [8] |
TANG Duyu, QIN Bing, LIU Ting, et al. Document modeling with gated recurrent neural network for sentiment classification[C]. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2015: 1422-1432.
|
| [9] |
梁军, 柴玉梅, 原慧斌, 等. 基于极性转移和LSTM递归网络的情感分析[J]. 中文信息学报, 2015, 29(5): 160-167. doi: 10.3969/j.issn.1003-0077.2015.05.020.
|
|
LIANG Jun, CHAI Yumei, YUAN Huibin, et al. Polarity shifting and LSTM based recursive networks for sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(5): 160-167. doi: 10.3969/j.issn.1003-0077.2015.05.020.
|
| [10] |
刘飞龙, 郝文宁, 陈刚, 等. 基于双线性函数注意力Bi-LSTM模型的机器阅读理解[J]. 计算机科学, 2017, 44(6A): 92-96. doi: 10.11896/j.issn.1002-137X.2017.6A.019.
|
|
LIU Feilong, HAO Wenning, CHEN Gang, et al. Attention of bilinear function based Bi-LSTM model for machine reading comprehension[J]. Computer Science, 2017, 44(6A): 92-96. doi: 10.11896/j.issn.1002-137X.2017.6A.019.
|
| [11] |
CHENG Jianpeng, DONG Li, and LAPATA Mirella. Long short-term memory-networks for machine reading[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 551-561.
|
| [12] |
WANG Yequan, HUANG Minlie, ZHAO Li, et al. Attention- based LSTM for aspect-level sentiment classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 606-615.
|
| [13] |
PARIKH Ankur P, TAKSTROM Oscar, DAS Dipanjan, et al. A decomposable attention model for natural language inference[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2249-2255.
|
| [14] |
GOLUB David and HE Xiaodong. Character-level question answering with attention[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1598-1607.
|
| [15] |
MI Haitao, WANG Zhiguo, and ITTYCHERIAH Abe. Supervised attentions for neural machine translation[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2283-2288.
|
|
|
|
