Potential limitations in COVID-19 machine learning due to data source variability: A case study in the nCov2019 dataset

Sáez, Carlos; Romero, Nekane; Conejero, J. Alberto; García‐Gómez, Juan Miguel

doi:10.1093/jamia/ocaa258

Cited by 64 publications

(46 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There were virtually no measures to prevent the virus from spreading worldwide even when the hazard was well known to everyone, and there was an inability for detecting cases in time to prevent further contagion. Even at the time of counting the fatalities, the data collection systems have shown major discrepancies [ 3 ], evidencing a total lack of basic data management strategies. Moreover, the very fact that the pandemic was unexpected, despite warnings by some studies [ 4 ], shows the unpreparedness of public administrations and society in general.…”

Section: Introductionmentioning

confidence: 99%

Mass Tracking in Cellular Networks for the COVID-19 Pandemic Monitoring

Khatib

Roselló

Páez

et al. 2021

Sensors

View full text Add to dashboard Cite

The year 2020 was marked by the emergence of the COVID-19 pandemic. After months of uncontrolled spread worldwide, a clear conclusion is that controlling the mobility of the general population can slow down the propagation of the pandemic. Tracking the location of the population enables better use of mobility limitation policies and the prediction of potential hotspots, as well as improved alert services to individuals that may have been exposed to the virus. With mobility in their core functionality and a high degree of penetration of mobile devices within the general population, cellular networks are an invaluable asset for this purpose. This paper shows an overview of the possibilities offered by cellular networks for the massive tacking of the population at different levels. The major privacy concerns are also reviewed and a specific use case is shown, correlating mobility and number of cases in the province of Málaga (Spain).

show abstract

Section: Introductionmentioning

confidence: 99%

Mass Tracking in Cellular Networks for the COVID-19 Pandemic Monitoring

Khatib

Roselló

Páez

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The Cox model allows the analysis of the simultaneous effect of a set of covariables in the survival expressed by the hazard ratio. The Cox model implemented in lifelines [43] also allows to obtain a prediction of survival by calculating the survival expected time.…”

Section: Discussionmentioning

confidence: 99%

“…The main limitation in our study resides in the use of data from only one hospital, an internal validation only assures the performance of the models with similar data. We cannot ensure the reported efficiency in other hospitals and/or with other patient populations [43]. Also, data from the same centres can change over time due a wide variety of reasons such as change in protocols or external agents such a pandemic [44, 45].…”

Section: Discussionmentioning

confidence: 99%

Complementary frailty and mortality prediction models on older patients as a tool for assessing palliative care needs

Blanes-Selva

Doñate-Martínez

Linklater

et al. 2021

Preprint

View full text Add to dashboard Cite

BackgroundPalliative care (PC) has demonstrated benefits for life-limiting illnesses. Cancer patients have mainly accessed these services, but there is growing consensus about the importance of promoting access for patients with non-malignant disease. Bad survival prognosis and patient’s frailty are usual dimensions to decide PC inclusion.ObjectivesThe main aim of this work is to design and evaluate three quantitative models based on machine learning approaches to predict frailty and mortality on older patients in the context of supporting PC decision making: one-year mortality, survival regression and one-year frailty classification.MethodsThe dataset used in this study is composed of 39,310 hospital admissions for 19,753 older patients (age >= 65) from January 1st, 2011 to December 30th, 2018. All prediction models were based on Gradient Boosting Machines. From the initial pool of variables at hospital admission, 20 were selected by a recursive feature elimination algorithm based on the random forest’s GINI importance criterion. Besides, we run an independent grid search to find the best hyperparameters in each model. The evaluation was performed by 10-fold cross-validation and area under the receiver operating characteristic curve and mean absolute error were reported. The Cox proportional-hazards model was used to compare our proposed approach with classical survival methods.ResultsThe one-year mortality model achieved an AUC ROC of 0.87 ± 0.01; the mortality regression model achieved an MAE of 329.97 ± 5.24 days. The one-year frailty classification reported an AUC ROC of 0.9 ± 0.01. The Spearman’s correlation between the admission frailty index and the survival time was –0.1, while the point-biserial correlation between one-year frailty index and survival time was –0.16.ConclusionsOne-year mortality model performance is at a state-of-the-art level. Frailty Index used in this study behaves coherently with other works in the literature. One-year frailty classifier demonstrated that frailty status within the year could be predicted accurately. To our knowledge, this is the first study predicting one-year frailty status based on a frailty index. We found mortality and frailty as two weakly correlated and complementary PC needs assessment criteria. Predictive models are available online at http://demoiapc.upv.es.

show abstract

“…We firstly selected the group of clusters that showed relatively better Silhouette Coefficient values, then chose the number of clusters from these which provided the most reasonable and clinically distinguishable classification regarding clinical phenotypes and demographic features. This process was supported by the COVID-19 pipelines and exploratory tool we developed in previous work 21 .…”

Section: Methodsmentioning

confidence: 99%

Subphenotyping of COVID-19 patients at pre-admission towards anticipated severity stratification: an analysis of 778 692 Mexican patients through an age-gender unbiased meta-clustering technique

Zhou¹,

Romero²,

Martı́nez-Miranda

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Objective To describe COVID-19 subphenotypes regarding severity patterns including prognostic, ICU and morbimortality outcomes, through stratification based on gender and age groups, as described by inter-patient variability patterns in clinical phenotypes and demographic features. Materials and methods We used the COVID-19 open data from the Mexican Government including patient-level epidemiological and clinical data from 778 692 SARS-CoV-2 patients from January 13, 2020 to September 30, 2020. Inter-patient variability was analyzed by combining dimensionality reduction and hierarchical clustering methods. We produced cluster analyses for all combinations of gender and age groups (<18, 18-49, 50-64, and >64). For each group, the optimum number of clusters was selected combining a quantitative approach using the Silhouette coefficient, and a qualitative approach through a subgroup expert inspection via visual analytics. Using the features of the resultant age-gender clusters, we performed a meta-clustering analysis to provide an overall description of the population. Results We observed a total of 56 age-gender clusters, grouped in 11 clinically distinguishable meta-clusters with different outcomes. Meta-clusters 1 to 3 showed the highest recovery rates (90.27-95.22%). These clusters include: (1) healthy patients of all ages, (2) children with comorbidities who had priority in medical resources, (3) young patients with obesity and smoking habit. Meta-clusters 4 and 5 showed moderate recovery rates (81.3-82.81%): (4) patients with hypertension or diabetes of all ages, (5) typical obese patients with three highly correlated conditions, namely, pneumonia, hypertension and diabetes. Meta-clusters 6 to 11 had very low recovery rates (53.96-66.94%) which include: (6) immunosuppressed patients with the highest comorbidity rate in many diseases, (7) CKD patients with the worse survival length and recovery, (8) elderly smoker with mild COPD, (9) severe diabetic elderly with hypertension, (10, 11) oldest obese smokers with severe COPD and mild cardiovascular disease with the latter (11) showing a relatively higher age and smoke rate, severe COPD and shorter survival length, reinforcing a high correlation between smoking habit and COPD among elderly. Additionally, the source Mexican state and type of clinical institution proved to be an important factor for heterogeneity in severity. Discussion The proposed unsupervised learning approach successfully uncovered discriminative COVID-19 severity patterns for both genders and all age groups from clinical phenotypes and demographic features. A careful read of group outcomes showed consistent results regarding recent literature. Regarding the Mexican population, our results suggest that habits and comorbidities may play a key role in predicting mortality in older patients. Centenarians tended to fall in the groups with better outcomes repeatedly. Additionally, immunosuppression was not found as a relevant factor for severity alone but did when present along with chronic kidney disease. Further useful correlations could be found by evaluating the duration of unhealthy habits, demographic features, comorbidities, the time since diagnosis, recovery progress, readmission record, and the effect of source variability. Conclusion The resultant eleven meta-clusters provide bases to comprehend the classification of patients with COVID-19 based on comorbidities, habits, demographic characteristics, geographic data and type of clinical institutions, as well as revealing the correlations between the above characteristics thereby help to anticipate the possible clinical outcomes for every specifically characterized patient. These subphenotypes can establish target groups for automated stratification or triage systems to provide personalized therapies or treatments. Code available at: https://github.com/bdslab-upv/covid19-metaclustering Dynamic results visualization at: http://covid19sdetool.upv.es/?tab=mexicoGov

show abstract

Potential limitations in COVID-19 machine learning due to data source variability: A case study in the nCov2019 dataset

Cited by 64 publications

References 22 publications

Mass Tracking in Cellular Networks for the COVID-19 Pandemic Monitoring

Mass Tracking in Cellular Networks for the COVID-19 Pandemic Monitoring

Complementary frailty and mortality prediction models on older patients as a tool for assessing palliative care needs

Subphenotyping of COVID-19 patients at pre-admission towards anticipated severity stratification: an analysis of 778 692 Mexican patients through an age-gender unbiased meta-clustering technique

Contact Info

Product

Resources

About