Multimodal sentiment analysis using deep learning and fuzzy logic: A comprehensive survey

Multimodal sentiment analysis (MSA) is the process of identifying sentiment polarities that users may simultaneously display in text, audio, and video data. Sentiment analysis methods based on a single data type are becoming increasingly unsuitable. Therefore, MSA has emerged, developed, and become increasingly popular today. MSA plays an important role in many practical applications, particularly in decision-making, recommendation systems, and fake news detection systems. Therefore, new proposals and performance improvements in multimodal sentiment-analysis methods are of great interest to scientists. Many multimodal sentiment-analysis methods have been proposed and developed, including those based on deep learning (DL) models, which have achieved significant prospects. This study comprehensively surveys the aspects related to MSA methods based on deep learning (MSA-based on DL) techniques. MSA-based on DL is the process of identifying sentiment polarities using DL, such as CNN, RNN, LSTM, GAN, and DBN, to create hidden layers for the automation of time-consuming stages, such as feature selection, feature extraction, parameter optimization, feature vector processing, and prediction generation on various data types simultaneously. In this paper, we propose a new taxonomy for MSA methods based on DL, and evaluate and compare method groups using commonly used data types. This article presents the advantages, disadvantages, and challenges that must be addressed in the future. Unlike previous survey works, this study proposes a particular classification by adding a group of multimodal sentiment-analysis methods based on a combination of DL techniques and fuzzy logic. The results of this study are expected to provide essential guidance to beginners, practitioners, and researchers regarding building and improving the performance of multimodal sentiment-analysis methods. •A comprehensive review of multimodal sentiment analysis (MSA) methods.•A new taxonomy of MSA methods with six main groups.•Introduce, categorize, and compare MSA models in detail.•Highlight challenges and future opportunities of MSA.