A cognitive IoT-based framework for effective diagnosis of COVID-19 using multimodal data

Abstract

The COVID-19 emerged at the end of 2019 and has become a global pandemic. There are many methods for COVID-19 prediction using a single modality. However, none of them predicts with 100% accuracy, as each individual exhibits varied symptoms for the disease. To decrease the rate of misdiagnosis, multiple modalities can be used for prediction. Besides, there is also a need for a self-diagnosis system to narrow down the risk of virus spread in testing centres. Therefore, we propose a robust IoT and deep learning-based multi-modal data classification method for the accurate prediction of COVID-19. Generally, highly accurate models require deep architectures. In this work, we introduce two lightweight models, namely CovParaNet for audio (cough, speech, breathing) classification and CovTinyNet for image (X-rays, CT scans) classification. These two models were identified as the best unimodal models after comparative analysis with the existing benchmark models. Finally, the obtained results of the five independently trained unimodal models are integrated by a novel dynamic multimodal Random Forest classifier. The lightweight CovParaNet and CovTinyNet models attain a maximum accuracy of 97.45% and 99.19% respectively even with a small dataset. The proposed dynamic multimodal fusion model predicts the final result with 100% accuracy, precision, and recall, and the online retraining mechanism enables it to extend its support even in a noisy environment. Furthermore, the computational complexity of all the unimodal models is minimized tremendously and the system functions effectively with 100% reliability even in the absence of any one of the input modalities during testing.

Challenges and Opportunities of Deep Learning for Cough-Based COVID-19 Diagnosis: A Scoping Review

“…In addition, the FNR can be straightforwardly computed as . We note that the confusion matrix, a matrix that counts the TN, FN, TP, and FP from actual target and predicted values, is often available, which can provide the reader with all the necessary tools to make a judgment [ 24 , 40 , 41 , 42 ].…”

Ensemble Deep Learning for IoT Based COVID 19 Health Care Pollution Monitor

“…Single-objective optimization, on the other side, can reduce the cloud-based data center's efficiency and effectiveness. To execute such kinds of applications or tasks i.e., prediction of COVID-19 with respect to healthcare pollution monitoring aspects with minimum prediction time and error and higher prediction accurateness remains the exciting issue.A robust IoT and deep learning-based multi-modal data classification method was designed in [20] for accurate prediction of COVID-19. It consist of two lightweight models, namely CovParaNet for audio (cough, speech, breathing) classification and CovTinyNet for image (X-rays, CT scans) classification.…”

“…As given in the above Cloud-based Framingham Feature Extraction algorithm with COVID -19 dataset [20] provided as input, the objective here remains in extracting the most relevant and robust features according to structured and categorical features obtained via patient's sensors. Next, Gini indexed factor is evaluated separately for both structured and categorical factors, therefore removing irrelevant and redundant features.…”

A novel deep fusion strategy for COVID-19 prediction using multimodality approach

“…To determine the Cepstral Coefficients of sound signals, a cepstral investigation is carried out on the Mel Spectrum. As a result, these numbers are referred to as MFCCs and multi-dimensional MFCC feature extraction is done from the acoustic signals [25] . Firstly, the acquired acoustic signals are converted into the Mel scale to calculate higher acoustic resolution in the case of lower audio frequencies which helps to enhance symptom determination stability.…”