Weibo Text Sentiment Classification Model Based on FastText-BERT-Attention
DOI: https://doi.org/10.62517/jbdc.202401104
Author(s)
Jianming Zhang*
Affiliation(s)
School of Management, Xi’an Polytechnic University, Xi’an, Shaanxi, China
*Corresponding Author.
Abstract
This paper introduces a Weibo text sentiment classification model that integrates FastText, BERT, and an attention mechanism, aiming to overcome the limitations of traditional models in processing social media data. By leveraging FastText's efficient word-level feature extraction capabilities, BERT's deep semantic understanding, and the feature fusion advantage of the attention mechanism, this model significantly enhances the accuracy of Weibo text classification. Experimental results show that compared to Word2Vec, FastText, and BERT models, the FastText-BERT-Attention model proposed in this paper demonstrates higher precision, recall, and F1 scores in the sentiment binary classification task, proving its effectiveness and superiority in handling large-scale short text data from Weibo comments. This study not only presents theoretical innovation but also exhibits superior performance in practical applications, making it particularly suitable for processing short text data from social media platforms like Weibo.
Keywords
FastText, BERT, Attention, Text Sentiment Classification
References
[1]Wan X. Using bilingual knowledge and ensemble techniques for unsupervised Chinese sentiment analysis. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. 2008.553-561.
[2]Yang Chao, Feng Shi, Wang Daling, et al. Analysis of Online Public Opinion Orientation Based on Sentiment Dictionary Expansion Technology. Journal of Mini & Micro Computer Systems, 2010, 31(04): 691-695.
[3]Yang A M, Lin J H, Zhou Y M, et al. Research on Building a Chinese Sentiment Lexicon Based on SO-PMI. Applied Mechanics & Materials, 2013, 263-266:1688-1693.
[4]Pang B, Lee L, Vaithyanathan S .Thumbs up? Sentiment Classification using Machine Learning Techniques.2002 [2024-03-13] ．
[5]Kang H, Yoo S J, Han D. Senti-lexicon and improved Nave Bayes algorithms for sentiment analysis of restaurant reviews. Expert Systems with Applications, 2012, 39(5): 6000-6010.
[6]Yang Shuang, Chen Fen. Research on Multi-Level Sentiment Classification of Weibo Based on SVM and Multi-Feature Fusion. Data Analysis and Knowledge Discovery, 2017, 1(02): 73-79.
[7]Daeli N O F, Adiwijaya A. Sentiment analysis on movie reviews using information gain and K-nearest neighbor. Journal of Data Science and Its Applications, 2020, 3(1): 1-7.
[8]Chen Tao, An Junxiu. Research on Weibo Short Text Sentiment Classification Based on Feature Fusion. Frontiers of Data and Computing, 2020, 2(06): 21-29.
[9]Chen Zhiqun, Ju Ting. Research on the Orientation Analysis of Weibo Comments Based on BERT and Bidirectional LSTM. Information Theory and Practice, 2020, 43(08): 173-177.
[10]Singh M, Jakhar A K, Pandey S. Sentiment analysis on the impact of coronavirus in social life using the BERT model. Social Network Analysis and Mining, 2021, 11(1): 1-11.
[11]Basiri M E,Nemati S, Abdar M et al. ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis. Future Generation Computer Systems, 2021, 115: 279-294.
[12]K. Sharmila, N. S. Devi and R. Devi, "Survey on Sentiment Analysis using Deep Learning," 2022 11th International Conference on System Modeling & Advancement in Research Trends (SMART), Moradabad, India, 2022, pp. 1434-1438
[13]Chandrasekaran G, Hemanth J .Deep Learning and TextBlob Based Sentiment Analysis for Coronavirus (COVID-19) Using Twitter Data. International Journal of Artificial Intelligence Tools: Architectures, Languages, Algorithms, 2022(1):31.
[14]S. Chen and G. Fnu, "Deep Learning Techniques for Aspect Based Sentiment Analysis," 2022 14th International Conference on Computer Research and Development (ICCRD), Shenzhen, China, 2022, pp. 69-73
[15]Zhao, Y. (2023). Overview of Deep Learning Methods for Sentiment Analysis. Advances in Engineering Technology Research
