XELSA: Sentiment Analysis Using Transformer Model
DOI:
https://doi.org/10.47839/ijc.25.2.4655Keywords:
XELSA, BERT, Election campaigns, Permutation language modeling, AccuracyAbstract
Sentiment analysis is an invaluable tool in modern election campaigns, which provides profound insights into voter preferences, behavior, and opinions, enabling campaigns to make informed decisions and ultimately increase their chances of success in an election. Many existing approaches use transformer models like BERT, which uses a masked language model technique, to predict these masked tokens by substituting a [MASK] token for certain input tokens. A mismatch between pre-training and fine-tuning may arise from this method since BERT does not view the actual sequence when training. The proposed XELSA model leverages the advanced capabilities of XLNet’s permutation language modeling to achieve superior sentiment analysis performance compared to traditional models like BERT. The proposed work shows that XLNet can model longer-term dependencies and capture more context compared to BERT, often leading to improved performance in understanding complex sentiment expressions. The results show that XELSA shows a better classification of negative, positive, and neutral tweets and outperforms BERT in terms of providing an accuracy of 96%. A dataset of 10,000 tweets pertaining to the US 2020 election was used in the study, and XELSA demonstrated improved precision, recall, and F1score across all sentiment categories. These results highlight the effectiveness of XELSA in providing more accurate sentiment analysis, which is crucial for election forecasting.
References
M. S. Lewis-Beck, “Election forecasting: Principles and practice,” Brit.J. Politics Int. Relations, vol. 7, pp. 145-164, 2005. https://doi.org/10.1111/j.1467-856X.2005.00178.x.
W. Ascher, “Political forecasting: The missing link,” J. Forecasting, vol. 1, pp. 227-239, 1982. https://doi.org/10.1002/for.3980010303.
K. L. Remmer, “The political economy of elections in Latin America, 1980-1991,” Amer. Political Sci. Rev., vol. 87, pp. 393-407, 1993. https://doi.org/10.2307/2939049.
S. Ansolabehere and J. M. Snyder, Jr., “The incumbency advantage in US elections: An analysis of state and federal offices, 1942-2000,” Election Law J., vol. 1, pp. 315-338, 2002. https://doi.org/10.1089/153312902760137578.
G. Terhanian, J. Bremer, R. Smith, and R. Thomas, “Correcting data from online surveys for the effects of non-random selection and non-random assignment,” Harris Interactive, Chicago, IL, USA, White Paper, pp. 1-13, 2000.
G. Enli, “Twitter as arena for the authentic outsider: Exploring the social media campaigns of Trump and Clinton in the 2016 US presidential election,” Eur. J. Commun., vol. 32, pp. 50-61, 2017. https://doi.org/10.1177/0267323116682802.
H. Wang, D. Can, A. Kazemzadeh, F. Bar, and S. Narayanan, “A system for real-time Twitter sentiment analysis of 2012 US presidential election cycle,” Proc. ACL Syst. Demonstrations, 2012, pp. 115-120.
A. Giachanou and F. Crestani, “Like it or not: A survey of Twitter sentiment analysis methods,” ACM Comput. Surv., vol. 49, pp. 1-41, 2016. https://doi.org/10.1145/2938640.
A. Agarwal, D. Toshniwal, and J. Bedi, “Can Twitter help to predict outcome of 2019 Indian general election: A deep learning based study,” Proc. Joint Eur. Conf. Mach. Learn. Knowl. Discovery Databases, India, Springer, pp. 38-53, 2019. https://doi.org/10.1007/978-3-030-43887-6_4.
H. T. Phan, N. T. Nguyen and D. Hwang, “Aspect-level sentiment analysis using CNN over BERT-GCN,” IEEE Access, vol. 10, pp. 110402-110409, 2022. https://doi.org/10.1109/ACCESS.2022.3214233.
K. A. Alshaikh, O. A. Almatrafi and Y. B. Abushark, “BERT-based model for aspect-based sentiment analysis for analyzing Arabic open-ended survey responses: A case study,” IEEE Access, vol. 12, pp. 2288-2302, 2024. https://doi.org/10.1109/ACCESS.2023.3348342.
R. Chandra and R. Saini, “Biden vs Trump: Modeling US general elections using BERT language model,” IEEE Access, vol. 9, pp. 128494-128505, 2021. https://doi.org/10.1109/ACCESS.2021.3111035.
N. Liu, J. Zhao, “A BERT-based aspect-level sentiment analysis algorithm for cross-domain text,” Comput Intell Neurosci., Art. no. 8726621, 2022. https://doi.org/10.1155/2022/8726621.
I. Üveges and O. Ring, “HunEmBERT: A fine-tuned BERT-model for classifying sentiment and emotion in political communication,” IEEE Access, vol. 11, pp. 60267-60278, 2023. https://doi.org/10.1109/ACCESS.2023.3285536.
X. Yan, F. Jian and B. Sun, “SAKG-BERT: Enabling language representation with knowledge graphs for Chinese sentiment analysis,” IEEE Access, vol. 9, pp. 101695-101701, 2021. https://doi.org/10.1109/ACCESS.2021.3098180.
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv Preprint, arXiv:1907.11692, 2019.
P. He, X. Liu, J. Gao, and W. Chen, “DeBERTa: Decoding-enhanced BERT with disentangled attention,” arXiv Preprint, arXiv:2006.03654, 2020.
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. Salakhutdinov, and Q. V. Le, “XLNet: Generalized autoregressive pretraining for language understanding,” arXiv Preprint, arXiv:1906.08237, 2019.
A. Graefe et al., “Comparing election forecasting methods: Bayesian and frequentist approaches,” Int. J. Forecasting, vol. 33, no. 2, pp. 391–400, 2017.
S. Devlin et al., “BERT for targeted sentiment analysis of political tweets,” Proc. IEEE Int. Conf. Data Mining, 2020, pp. 1122–1127.
L. Derczynski et al., “Political sentiment analysis with BERT: A case study of UK elections,” IEEE Trans. Comput. Soc. Syst., vol. 8, no. 4, pp. 1023–1032, 2021.
R. K. Koul et al., “Transformer models for election prediction: BERT vs. RoBERTa,” IEEE Access, vol. 10, pp. 34567–34578, 2022.
R. Cai et al., “Sentiment analysis about investors and consumers in energy market based on BERT-BiLSTM,” IEEE Access, vol. 8, pp. 171408-171415, 2020. https://doi.org/10.1109/ACCESS.2020.3024750.
Z. Gao, A. Feng, X. Song and X. Wu, “Target-dependent sentiment classification with BERT,” IEEE Access, vol. 7, pp. 154290-154299, 2019. https://doi.org/10.1109/ACCESS.2019.2946594.
C. Huang, “Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China,” Lancet, vol. 395, pp. 497-506, 2020. https://doi.org/10.1016/S0140-6736(20)30183-5.
C. Wang, P. W. Horby, F. G. Hayden, and G. F. Gao, “A novel coronavirus outbreak of global health concern,” Lancet, vol. 395, pp. 473-490, 2020. https://doi.org/10.1016/S0140-6736(20)30185-9.
J. Zhou, J. X. Huang, Q. V. Hu, and L. He, “SK-GCN: Modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification,” Knowl.-Based Syst., vol. 205, Art. no. 106292, 2020. https://doi.org/10.1016/j.knosys.2020.106292.
K. Bijari, H. Zare, E. Kebriaei, and H. Veisi, “Leveraging deep graph-based text representation for sentiment polarity applications,” Expert Syst. with Appl., vol. 144, Art. no. 113090, 2020. https://doi.org/10.1016/j.eswa.2019.113090.
H. Cai, Y. Tu, X. Zhou, J. Yu, and R. Xia, “Aspect-category based sentiment analysis with hierarchical graph convolutional network,” Proc. 28th Int. Conf. Comput. Linguistics, 2020, pp. 833-843. https://doi.org/10.18653/v1/2020.coling-main.72.
J. van der Geer, J.A.J. Hanraads, R.A. Lupton, “The art of writing a scientific article,” Journal of Science Communication, vol. 16, issue 3, pp. 51-59, 2000.
J. E. Campbell, The American Campaign: U.S. Presidential Campaigns and the National Vote, 2nd ed. Texas A&M University Press, 2008.
M. A. Al-Garadi et al., “Analyzing voter behavior via BERT-based topic modeling,” Proc. ACM Web Sci. Conf., 2021, pp. 1–9.
P. G. Ferreira et al., “Cross-domain sentiment analysis for political campaigns using adversarial BERT,” Knowl.-Based Syst., vol. 238, Art. no. 107842, 2022.
T. Chen and D. Wijaya, “Multilingual election sentiment analysis with BERT: A comparative study,” Proc. EMNLP, 2021, pp. 456–470.
Downloads
Published
How to Cite
Issue
Section
License
International Journal of Computing is an open access journal. Authors who publish with this journal agree to the following terms:• Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
• Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
• Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
