RoBERTa, ResNeXt and BiLSTM with self-attention: The ultimate trio for customer sentiment analysis

Sentiment analysis of customer feedback is a pivotal component of Natural Language Processing (NLP), enabling businesses to gauge consumer emotions towards their products and services. The inherent variability of language introduces substantial challenges in the analysis of unstructured customer data. Sentiment analysis techniques generally fall into two categories: traditional methods, which involve the extraction of hand-crafted features and the application of machine learning algorithms, and end-to-end deep learning approaches that process raw data, transforming them layer by layer into high-level textual features. Each method presents its own balance between simplicity and speed versus analytical power and flexibility, and they all grapple with the need for extensive labeled data and computational resources. To address these challenges, this paper presents an innovative hybrid model that amalgamates RoBERTa, ResNeXt, BiLSTM, and self-attention mechanisms. This integration capitalizes on the collective strengths of these models to surmount the limitations inherent in each. The proposed model proves to be a powerful and efficient tool for sentiment analysis, demonstrating proficiency across various data types and tasks. We undertake a comprehensive evaluation of the proposed model’s accuracy and training efficiency using four benchmark datasets. Our investigation also explores the impact of different similarity measures and convolutional neural network architectures on the model’s performance. The results affirm that our model not only achieves high accuracy but also significantly reduces training time compared to RoBERTa’s fine-tuning. Furthermore, the model exhibits exceptional domain adaptability, particularly when fine-tuned on the IMDb dataset following initial training on the Yelp Review Full dataset. The practicality of the proposed model is underscored by its reduced computational complexity and its adeptness at navigating semantic and syntactic nuances. •A novel hybrid sentiment model using RoBERTa, ResNeXt, BiLSTM, and self-attention.•Outperforms other methods on two datasets; matches RoBERTa base on two others.•Fewer parameters and faster fine-tuning compared to RoBERTa base.•Analysis of document similarity methods and text feature extractors.•TS-SS is viable for low-dimensional features but not high-dimensional ones.