Enhancing Twitter Sentiment Analysis Using Hybrid Transformer and Sequence Models
Bouassida Yosra, Mezali Hakim
Sentiment analysis on social media platforms such as Twitter is crucial for market analysis, public
opinion monitoring, and social media management. However, the complex and evolving language on
Twitter poses challenges for traditional models. Transformer models (e.g., BERT, RoBERTa) excel
in contextual understanding but struggle with sequential dependencies, while sequence models (e.g.,
BiLSTM) capture these dependencies but lack deep contextual insights. This research addresses this
gap by developing and evaluating three novel hybrid models: BERT-BiLSTM, Roberta-CNN-BiLSTM,
and DistilBERT-BiLSTM. These models combine transformer-based embeddings with sequence-based
processing to enhance sentiment classification. Using the Sentiment140 dataset, results indicate that the
proposed hybrid models improve sentiment analysis accuracy, with DistilBERT-BiLSTM achieving the
highest accuracy of 81%, compared to BERT-BiLSTM’s 79% and Roberta-CNN-BiLSTM’s 77%. These
innovative models provide a more nuanced sentiment analysis, although future research is needed to
explore real-time and multi-language datasets for further model optimization.
