A survey on multimodal data-driven smart healthcare systems: approaches and applications

Multimodal data-driven approach has emerged as an important driving force for smart healthcare systems with applications ranging from disease analysis to triage, diagnosis and treatment. Smart healthcare system necessitates new demands for data management and decision-making, which has inspired the rapid development of medical services using artificial intelligence and new transformations in the healthcare industry. In this paper, we provide a comprehensive survey of existing techniques which include not only state-of-the-art methods but also the most recent trends in the field. In particular, this review focuses on the types of decision-making processes used in smart healthcare systems. Firstly, approaches that utilize multimodal association mining with fine-grained data semantics in smart healthcare systems are introduced. We review the smart healthcare-oriented semantic perception, semantic alignment, entity association mining, and discuss the pros and cons of these approaches. Secondly, we discuss approaches for multimodal data fusion and cross-border association that have been employed in developing smart healthcare systems. Finally, we focus specifically on the use of the panoramic decision framework, interactive decision making, and intelligent decision support systems. We introduce how smart healthcare systems can be applied to and benefit a wide variety of fields, including knowledge discovery and privacy protection.