Comprehensive Review of Multimodal Medical Data Analysis: Open Issues and Future Research Directions

Abstract

Over the past few decades, the enormous expansion of medical data has led to searching for ways of data analysis in smart healthcare systems. Acquisition of data from pictures, archives, communication systems, electronic health records, online documents, radiology reports and clinical records of different styles with specific numerical information has given rise to the concept of multimodality and the need for machine learning and deep learning techniques in the analysis of the healthcare system. Medical data play a vital role in medical education and diagnosis; determining dependency between distinct modalities is essential. This paper gives a gist of current radiology medical data analysis techniques and their various approaches and frameworks for representation and classification. A brief outline of the existing medical multimodal data processing work is presented. The main objective of this study is to spot gaps in the surveyed area and list future tasks and challenges in radiology. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (or PRISMA) guidelines were incorporated in this study for effective article search and to investigate several relevant scientific publications. The systematic review was carried out on multimodal medical data analysis and highlighted advantages, limitations and strategies. The inherent benefit of multimodality in the medical domain powered with artificial intelligence has a significant impact on the performance of the disease diagnosis frameworks.

Dynamic data‐driven railway bridge construction knowledge graph update method

“…Some works integrating the knowledge resources in the enterprise based on the KG help to optimize the knowledge management of enterprise risk control, credit investigation, auditing, etc., and improve the intelligent level of enterprise management (Khan et al., 2016; Lai et al., 2023). In the medical field, medical KG can advance the intelligent processing of massive medical data, which is important for assisting disease diagnosis and drug recommendation (Shetty et al., 2022; Zeng et al., 2022). However, in the category of bridge construction, due to issues such as data privacy policies and data quality, fewer study on KG for bridge construction have been conducted, and the applicability of KG in the bridge field is still to be further investigated.…”

Cross-modal Deep Learning-based Clinical Recommendation System for Radiology Report Generation from Chest X-rays

“…Hospitals around the world heavily rely on medical imaging, which provides valuable insights for disease diagnosis and treatment planning [1,2]. However, it is crucial for the radiologist to thoroughly examine the medical images in order to provide comprehensive findings and interpretations [3].…”

Diagnostic Performance Evaluation of Deep Learning-Based Medical Text Modelling to Predict Pulmonary Diseases from Unstructured Radiology Free-Text Reports

“…The radiologist interprets a radiology imaging examination such as a chest X-ray to diagnose the conditions affecting the lungs. The radiologist analyses the chest X-ray and records their observations in descriptive reports to validate the prognosis (Shetty et al, 2022). Figure 1 shows an example of a radiology report prepared by a radiologist after analysing a raw chest X-ray image.…”

“…Available unstructured text in clinical practice is scarce as most radiology reports are restricted to the private institution or are domain-specific (Shetty et al, 2022). DL models produce better results with a large cohort size.…”