Optimism/pessimism prediction of Twitter messages and users using BERT with soft label assignment Conference Paper
| Authors: | Alshahrani, A.; Ghaffari, M.; Amirizirtol, K.; Liu, X. |
| Title: | Optimism/pessimism prediction of Twitter messages and users using BERT with soft label assignment |
| Conference Title: | 2021 International Joint Conference on Neural Networks (IJCNN) |
| Abstract: | Being able to accurately predict users' outlooks on social media platforms is important for developing educational public health interventions. In this paper, utilizing the contextualized representations provided by BERT, we propose new models to predict optimism/pessimism by fine-tuning BERT. By paying attention to the negation and other syntactic patterns, the self-attention mechanism via the transformer in BERT leads to more accurate models. For example, using the commonly used dataset for optimism/pessimism prediction with the proposed Soft Label Assignment (SLA), we have achieved 100% prediction accuracy at the user level and 97.10% at the tweet message level on the test set excluding the neutral ones. Furthermore, utilizing available training sample scores, we assign labels softly, which improves the generalization of the models and therefore their performance further. We additionally analyze the attention heads to illustrate the mechanisms of our models to classify different messages and demonstrate the connections between emotions and optimism/pessimism prediction. © 2021 IEEE. |
| Keywords: | forecasting; accurate modeling; fine tuning; statistical tests; social networking (online); prediction accuracy; attention mechanisms; bert; fine-tuning; soft label; transformer; health interventions; social media platforms; soft labels; syntactic patterns |
| Journal Title | Proceedings of the International Joint Conference on Neural Networks |
| Conference Dates: | 2021 Jul 18-22 |
| Conference Location: | Virtual |
| ISBN: | 978-1-6654-4597-8 |
| Publisher: | IEEE |
| Date Published: | 2021-09-01 |
| Language: | English |
| DOI: | 10.1109/ijcnn52387.2021.9534100 |
| PROVIDER: | scopus |
| DOI/URL: | |
| Notes: | Conference Paper -- Source: Scopus |
Altmetric
Citation Impact
BMJ Impact Analytics
MSK Authors
Related MSK Work