A Multimodal, Multimedia Point-of-Care Deep Learning Framework for COVID-19 Diagnosis

Abstract

In this article, we share our experiences in designing and developing a suite of deep neural network–(DNN) based COVID-19 case detection and recognition framework. Existing pathological tests such as RT-PCR-based pathogen RNA detection from nasal swabbing seem to display low detection rates during the early stages of virus contraction. Moreover, the reliance on a few overburdened laboratories based around an epicenter capable of supplying large numbers of RT-PCR tests makes this testing method non-scalable when the rate of infections is high. Similarly, finding an effective drug or vaccine with which to combat COVID-19 requires a long time and many clinical trials. The development of pathological COVID-19 tests is hindered by shortages in the supply chain of chemical reagents necessary for testing on a large scale. This diminishes the speed of diagnosis and the ability to filter out COVID-19 positive patients from uninfected patients on a national level. Existing research has shown that DNN has been successful in identifying COVID-19 from radiological media such as CT scans and X-ray images, audio media such as cough sounds, optical coherence tomography to identify conjunctivitis and pink eye symptoms on the ocular surface, body temperature measurement using smartphone fingerprint sensors or thermal cameras, the use of live facial detection to identify safe social distancing practices from camera images, and face mask detection from camera images. We also investigate the utility of federated learning in diagnosis cases where private data can be trained via edge learning. These point-of-care modalities can be integrated with DNN-based RT-PCR laboratory test results to assimilate multiple modalities of COVID-19 detection and thereby provide more dimensions of diagnosis. Finally, we will present our initial test results, which are encouraging.

Res-CovNet: an internet of medical health things driven COVID-19 framework using transfer learning

“…Further, they utilized the device on a collected dataset of chest X-ray and CT photos as well as experienced enhanced results. Mohammed et al [ 7 ] proposed a methodology for automatic recognition of COVID-19 infected person from the healthy person using X-ray images. This methodology was built using two reliable technologies they include conventional machine learning methodologies and deep learning frameworks [ 26 , 27 ].…”

“…Another diagnostic technique to detect COVID-19 is Reverse Transcription polymerase chain reaction (RT-PCR), which is a more commonly used diagnostic technique. Still, it has low sensitivity problems in initial phases with lengthy assessment facilitating additional transmission [ 7 ]. Thus, most people with suspected pneumonia symptoms are recommended to scan the chest by utilizing the techniques, X-rays, and Computer Tomography (CT) scans for faster diagnosis and isolation of the affected people [ 8 ].…”

Review of Recent Technologies for Tackling COVID-19

“…In [8], the authors presented a DNN model that can be leveraged by three stakeholders (patients, doctors, and decisionmaker authority). The model connects patients or suspects with various biometric sensors that help to quickly collect relevant samples and then process them locally.…”

Diagnosing COVID-19 from CT Image of Lung Segmentation & Classification with Deep Learning Based on Convolutional Neural Networks

“…The diagnosis of COVID-19 is accomplished by the multimodal and deep learning approach. In this work X-Ray and CT scans are utilized in the disease identification and the augmentation based identification is also accomplished by DNN [ 18 , 19 ]. The information transmitted over the internet or stored in the cloud can be secured with Blockchain technology and other cryptography mechanism [ 20 , 21 ].…”