Abstract:In Chinese text categorization tasks, the locations of the important features in the Chinese texts are disperse and sparse, and the different characteristics of Chinese texts contributes differently for the recognition of their categories. In order to solve the above problems, this paper proposes a multi-feature fusion model Three Convolutional neural network paths and Long short term memory path fused with Attention neural network path (3CLA) for Chinese text categorization, which is based on Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) and semantic understanding attention neural networks. The model first uses text preprocessing to finish the segmentation and vectorization of the Chinese text. Then, through the embedding layer, the input data are sent to the CNN path, the LSTM path and the attention path respectively to extract text features of different levels and different characteristics. Finally, the text features are fused by the fusion layer and classified by the classifier. Based on the Chinese corpus, the text classification experiment is carried out. The results of the experiments show that compared with the CNN structure model and the LSTM structure model, the proposed algorithm model improves the recognition ability of Chinese text categories by up to about 8%.
谢金宝,侯永进,康守强,李佰蔚,张霄. 基于语义理解注意力神经网络的多元特征融合中文文本分类[J]. 电子与信息学报, 2018, 40(5): 1258-1265.
XIE Jinbao, HOU Yongjin, KANG Shouqiang, LI Baiwei, ZHANG Xiao. Multi-feature Fusion Based on Semantic Understanding Attention Neural Network for Chinese Text Categorization. JEIT, 2018, 40(5): 1258-1265.
SUN Xiao, PENG Xiaoqi, HU Min, et al. Extended multi- modality features and deep learning based Microblog short text sentiment analysis[J]. Journal of Electronics & Information Technology, 2017, 39(9): 2048-2055. doi: 10.11999/JEIT160975.
[2]
KIM Yoon. Convolutional neural networks for sentence classification[C]. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014: 1746-1751.
[3]
PHAM Ngocquan, KRUSZEWSKI German, and BOLEDA Gemma. Convolutional neural network language models[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1153-1162.
CHEN Zhao, XU Ruifeng, GUI Lin, et al. Combining convolutional neural networks and word sentiment sequence features for Chinese text sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 172-178. doi: 10.3969/j.issn.1003-0077.2015.06.023.
LIU Longfei, YANG Liang, ZHANG Shaowu, et al. Convolutional neural networks for Chinese micro-blog sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 159-165. doi: 10.3969/j.issn.1003- 0077.2015.06.021.
[6]
ZHANG Ye, MARSHALL Iain, and WALLACE B C. Rational-augmented convolutional neural networks for text classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 759-804.
[7]
MIYAMOTO Yasumasa and CHO Kyunghyun. Gated word-character recurrent language model[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1992-1997.
[8]
TANG Duyu, QIN Bing, LIU Ting, et al. Document modeling with gated recurrent neural network for sentiment classification[C]. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2015: 1422-1432.
LIANG Jun, CHAI Yumei, YUAN Huibin, et al. Polarity shifting and LSTM based recursive networks for sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(5): 160-167. doi: 10.3969/j.issn.1003-0077.2015.05.020.
LIU Feilong, HAO Wenning, CHEN Gang, et al. Attention of bilinear function based Bi-LSTM model for machine reading comprehension[J]. Computer Science, 2017, 44(6A): 92-96. doi: 10.11896/j.issn.1002-137X.2017.6A.019.
[11]
CHENG Jianpeng, DONG Li, and LAPATA Mirella. Long short-term memory-networks for machine reading[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 551-561.
[12]
WANG Yequan, HUANG Minlie, ZHAO Li, et al. Attention- based LSTM for aspect-level sentiment classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 606-615.
[13]
PARIKH Ankur P, TAKSTROM Oscar, DAS Dipanjan, et al. A decomposable attention model for natural language inference[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2249-2255.
[14]
GOLUB David and HE Xiaodong. Character-level question answering with attention[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1598-1607.
[15]
MI Haitao, WANG Zhiguo, and ITTYCHERIAH Abe. Supervised attentions for neural machine translation[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2283-2288.