Rapidly deploying a COVID-19 decision support system in one of the largest Brazilian hospitals

Carmo, Diedre; Campiotti, Israel; Rodrigues, Lívia; Fantini, Irene; Pinheiro, Gustavo Retuci; Moraes, Daniel de Sousa; Nogueira, Rodrigo; Rittner, Letícia; Lotufo, Roberto de Alencar

doi:10.1177/14604582211033017

Cited by 4 publications

(16 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…At the beginning of the COVID-19 pandemic, we and others sought to generate AI models to successfully predict COVID-19 from biomedical imaging ( 19 , 20 ). Unfortunately, 2 years into the pandemic, no such generalizable model exists, and few models have been investigated in real time ( 9 , 21 ). There may be several reasons for the lack of any successfully deployed diagnostic model.…”

Section: Discussionmentioning

confidence: 99%

Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study

Sun¹,

Peng²,

Li³

et al. 2022

Radiology: Artificial Intelligence

View full text Add to dashboard Cite

Purpose To conduct a prospective observational study across 12 U.S. hospitals to evaluate real-time performance of an interpretable artificial intelligence (AI) model to detect COVID-19 on chest radiographs. Materials and Methods A total of 95 363 chest radiographs were included in model training, external validation, and real-time validation. The model was deployed as a clinical decision support system, and performance was prospectively evaluated. There were 5335 total real-time predictions and a COVID-19 prevalence of 4.8% (258 of 5335). Model performance was assessed with use of receiver operating characteristic analysis, precision-recall curves, and F1 score. Logistic regression was used to evaluate the association of race and sex with AI model diagnostic accuracy. To compare model accuracy with the performance of board-certified radiologists, a third dataset of 1638 images was read independently by two radiologists. Results Participants positive for COVID-19 had higher COVID-19 diagnostic scores than participants negative for COVID-19 (median, 0.1 [IQR, 0.0–0.8] vs 0.0 [IQR, 0.0–0.1], respectively; P < .001). Real-time model performance was unchanged over 19 weeks of implementation (area under the receiver operating characteristic curve, 0.70; 95% CI: 0.66, 0.73). Model sensitivity was higher in men than women ( P = .01), whereas model specificity was higher in women ( P = .001). Sensitivity was higher for Asian ( P = .002) and Black ( P = .046) participants compared with White participants. The COVID-19 AI diagnostic system had worse accuracy (63.5% correct) compared with radiologist predictions (radiologist 1 = 67.8% correct, radiologist 2 = 68.6% correct; McNemar P < .001 for both). Conclusion AI-based tools have not yet reached full diagnostic potential for COVID-19 and underperform compared with radiologist prediction. Keywords: Diagnosis, Classification, Application Domain, Infection, Lung Supplemental material is available for this article. . © RSNA, 2022

show abstract

Section: Discussionmentioning

confidence: 99%

Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study

Sun¹,

Peng²,

Li³

et al. 2022

Radiology: Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…These studies also recommended the use of CDSS in future research. Finally, 68 articles met all the inclusion criteria 5,17,34–99 . The flowchart of the selection process is shown in Figure 1.…”

Section: Resultsmentioning

confidence: 99%

“…Types of CDSS to assist in diagnosing COVID‐19 are shown in Figure 4. Most of the studies used ICDSS based on ML (nonknowledge‐based CDSS) ( n = 52 [76.5%]) 34–85 . In these studies, the most common methods for designing CDSS were CNN ( n = 33), 38,40–42,45–47,49–52,54,56–69,71,72,78,82–85 SVM ( n = 8), 35,36,39,43,44,54,56,57 RF ( n = 7), 34,35,37,39,42,44,55 and KNN ( n = 7) 36,37,39,42,43,55,56 (Table 1 and Appendix ).…”

Section: Resultsmentioning

confidence: 99%

“…Most studies used a combination of performance metrics to evaluate CDSS ( n = 40) (Appendix ). In most of these studies, accuracy ( n = 43), 35–40,42–47,49–69,71,72,76–78,80–83,85 sensitivity (recall) ( n = 35) 35–37,39,41–44,46,47,49–54,56,57,61,63–69,71,72,76,78,81–85 and F1‐score ( n = 26) 34–36,39,41–43,46,47,50,52–54,56,57,60,61,63,64,66,68,69,81,82,84,85 were used (Table 2). Other metrics for evaluating ICDSS based on ML were the Matthews correlation coefficient (MCC), 52,56,58 receiver operating characteristic (ROC) curve, 34,36,70 and so on (Appendix ).…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Clinical decision support systems (CDSS) in assistance to COVID‐19 diagnosis: A scoping review on types and evaluation methods

Ameri,

Salmanizadeh

et al. 2024

Health Science Reports

View full text Add to dashboard Cite

Background and AimsDue to the COVID‐19 pandemic, a precise and reliable diagnosis of this disease is critical. The use of clinical decision support systems (CDSS) can help facilitate the diagnosis of COVID‐19. This scoping review aimed to investigate the role of CDSS in diagnosing COVID‐19.MethodsWe searched four databases (Web of Science, PubMed, Scopus, and Embase) using three groups of keywords related to CDSS, COVID‐19, and diagnosis. To collect data from studies, we utilized a data extraction form that consisted of eight fields. Three researchers selected relevant articles and extracted data using a data collection form. To resolve any disagreements, we consulted with a fourth researcher.ResultsA search of the databases retrieved 2199 articles, of which 68 were included in this review after removing duplicates and irrelevant articles. The studies used nonknowledge‐based CDSS (n = 52) and knowledge‐based CDSS (n = 16). Convolutional Neural Networks (CNN) (n = 33) and Support Vector Machine (SVM) (n = 8) were employed to design the CDSS in most of the studies. Accuracy (n = 43) and sensitivity (n = 35) were the most common metrics for evaluating CDSS.ConclusionCDSS for COVID‐19 diagnosis have been developed mainly through machine learning (ML) methods. The greater use of these techniques can be due to their availability of public data sets about chest imaging. Although these studies indicate high accuracy for CDSS based on ML, their novelty and data set biases raise questions about replacing these systems as clinician assistants in decision‐making. Further studies are needed to improve and compare the robustness and reliability of nonknowledge‐based and knowledge‐based CDSS in COVID‐19 diagnosis.

show abstract

“…Outros estudos nessa mesma linha são os de Munõz-Jariloa et al (13) e Carmo et al (17) . O primeiro realiza revisão da literatura em busca dos achados de imagem causados pela infecção do Sars-Cov-2 e esses auxiliam o profissional e facilitam o acompanhamento durante o curso de infecção aguda nos diferentes estágios da patologia.…”

Section: Estudo Transversal Do Tipo Ecológicounclassified

Inteligência Artificial Como Ferramenta De Redimensionamento Durante a Pandemia De Covid-19: Revisão Narrativa

Miranda¹,

Menezes²,

Saquetto³

et al. 2022

Rev. Bras. Saúde Funcional

View full text Add to dashboard Cite

Objetivo: Descrever a Inteligência Artificial (IA) comociência dirigida à área da saúde, como estratégia deredimensionamento de recursos humanos e financeirosdurante a pandemia do COVID-19. Métodos: Tratase de uma revisão narrativa. A pesquisa foi realizadaentre outubro e novembro de 2021 nas bases de dadosPubmed, Scielo, Lilacs e Medline utilizando descritores“Artificial Intelligence”; “coronavírus”. Foram incluídosEnsaios Clínicos e Estudos de Caso-Controle encontradosnas bases de dados dentro do período de março/2020 aoutubro/2021. Desenvolvimento: Foram encontrados9 (nove) artigos na busca bibliográfica. Após leitura naíntegra foi percebido uma diversidade de ferramentasque já estão sendo utilizadas. Tais ferramentascontemplam desde instrumentos para o controle desinais e diagnóstico, até serviços de geolocalização deindivíduos infectados, planejamento vacinal de acordocom a área de residência e instrumentos para produçãoepidemiológica visando medidas comportamentais eassistenciais. Considerações finais: a IA é uma possívelestratégia para redimensionamento dos recursoshumanos e financeiros durante esse período, seja peladiversidade da aplicabilidade, seja pela efetividade dasua utilização favorecendo o tempo de qualidade doprofissional de saúde, assim como do indivíduo, com omelhor direcionamento do seu diagnóstico e possíveltratamento.

show abstract

Rapidly deploying a COVID-19 decision support system in one of the largest Brazilian hospitals

Cited by 4 publications

References 34 publications

Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study

Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study

Clinical decision support systems (CDSS) in assistance to COVID‐19 diagnosis: A scoping review on types and evaluation methods

Inteligência Artificial Como Ferramenta De Redimensionamento Durante a Pandemia De Covid-19: Revisão Narrativa

Contact Info

Product

Resources

About