Grouped feature importance and combined features effect plot

Au, Quay; Herbinger, Julia; Stachl, Clemens; Bischl, Bernd; Casalicchio, Giuseppe

doi:10.1007/s10618-022-00840-5

Cited by 15 publications

(14 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the models were only trained on the MIT-BIH database, which was obtained by ambulatory EKG recordings and contains minimal noise, real-world data with more noise may present additional challenges. As aforementioned, the eight transition matrix features could be combined together and considered as one feature, using grouped feature importances [ 61 ]. This would allow for the cumulative importance of the transition matrix to be evaluated, potentially resulting in a higher importance than those of the individual transitions.…”

Section: Discussionmentioning

confidence: 99%

A Study of R-R Interval Transition Matrix Features for Machine Learning Algorithms in AFib Detection

Patel

Wang

Guo

et al. 2023

Sensors

View full text Add to dashboard Cite

Atrial Fibrillation (AFib) is a heart condition that occurs when electrophysiological malformations within heart tissues cause the atria to lose coordination with the ventricles, resulting in “irregularly irregular” heartbeats. Because symptoms are subtle and unpredictable, AFib diagnosis is often difficult or delayed. One possible solution is to build a system which predicts AFib based on the variability of R-R intervals (the distances between two R-peaks). This research aims to incorporate the transition matrix as a novel measure of R-R variability, while combining three segmentation schemes and two feature importance measures to systematically analyze the significance of individual features. The MIT-BIH dataset was first divided into three segmentation schemes, consisting of 5-s, 10-s, and 25-s subsets. In total, 21 various features, including the transition matrix features, were extracted from these subsets and used for the training of 11 machine learning classifiers. Next, permutation importance and tree-based feature importance calculations determined the most predictive features for each model. In summary, with Leave-One-Person-Out Cross Validation, classifiers under the 25-s segmentation scheme produced the best accuracies; specifically, Gradient Boosting (96.08%), Light Gradient Boosting (96.11%), and Extreme Gradient Boosting (96.30%). Among eleven classifiers, the three gradient boosting models and Random Forest exhibited the highest overall performance across all segmentation schemes. Moreover, the permutation and tree-based importance results demonstrated that the transition matrix features were most significant with longer subset lengths.

show abstract

Section: Discussionmentioning

confidence: 99%

A Study of R-R Interval Transition Matrix Features for Machine Learning Algorithms in AFib Detection

Patel

Wang

Guo

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Although the RF generated feature importance and the permutation importance converged in the upper ranking, delayed recall was rated differently between the two estimates. This suggests that possible mutual information in memory tests should be controlled for when running such random shuffling 28 . Overall, these results confirm a close relation between memory function and hippocampus 29 , and that both tend to be affected already in an early stage of AD 4 .…”

Section: Discussionmentioning

confidence: 99%

Predicting conversion to Alzheimer’s disease in individuals with Mild Cognitive Impairment using clinically transferable features

Rye

Vik

Kocinski

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Patients with Mild Cognitive Impairment (MCI) have an increased risk of Alzheimer’s disease (AD). Early identification of underlying neurodegenerative processes is essential to provide treatment before the disease is well established in the brain. Here we used longitudinal data from the ADNI database to investigate prediction of a trajectory towards AD in a group of patients defined as MCI at a baseline examination. One group remained stable over time (sMCI, n = 357) and one converted to AD (cAD, n = 321). By running two independent classification methods within a machine learning framework, with cognitive function, hippocampal volume and genetic APOE status as features, we obtained a cross-validation classification accuracy of about 70%. This level of accuracy was confirmed across different classification methods and validation procedures. Moreover, the sets of misclassified subjects had a large overlap between the two models. Impaired memory function was consistently found to be one of the core symptoms of MCI patients on a trajectory towards AD. The prediction above chance level shown in the present study should inspire further work to develop tools that can aid clinicians in making prognostic decisions.

show abstract

“…Further, if correlations or other complex dependencies are present within the data, LOCO might give misleading results if only one covariate at a time is excluded as this neglects potential interaction effects between groups of variables. In the presence of such group-wise structures, the exclusion of multiple covariates at a time is advisable (Au et al, 2022;Rinaldo et al, 2016).…”

Section: Leave-one-covariate-out (Loco)mentioning

confidence: 99%

Conditional Feature Importance for Mixed Data

Blesch¹,

Watson²,

Wright³

2022

Preprint

View full text Add to dashboard Cite

Despite the popularity of feature importance measures in interpretable machine learning, the statistical adequacy of these methods is rarely discussed. From a statistical perspective, a major distinction is between analyzing a variable's importance before and after adjusting for covariates -i.e., between marginal and conditional measures. Our work draws attention to this rarely acknowledged, yet crucial distinction and showcases its implications. Further, we reveal that for testing conditional feature importance (CFI), only few methods are available and practitioners have hitherto been severely restricted in method application due to mismatching data requirements. Most real-world data exhibits complex feature dependencies and incorporates both continuous and categorical data (mixed data). Both properties are oftentimes neglected by CFI measures. To fill this gap, we propose to combine the conditional predictive impact (CPI) framework (Watson and Wright, 2021) with sequential knockoff sampling (Kormaksson et al., 2021). The CPI enables CFI measurement that controls for any feature dependencies by sampling valid knockoffs -hence, generating synthetic data with similar statistical properties -for the data to be analyzed. Sequential knockoffs were deliberately designed to handle mixed data and thus allow us to extend the CPI approach to such datasets. We demonstrate through numerous simulations and a real-world example that our proposed workflow controls type I error, achieves high power and is in line with results given by other CFI measures, whereas marginal feature importance metrics result in misleading interpretations. Our findings highlight the necessity of developing statistically adequate, specialized methods for mixed data.

show abstract

Grouped feature importance and combined features effect plot

Cited by 15 publications

References 63 publications

A Study of R-R Interval Transition Matrix Features for Machine Learning Algorithms in AFib Detection

A Study of R-R Interval Transition Matrix Features for Machine Learning Algorithms in AFib Detection

Predicting conversion to Alzheimer’s disease in individuals with Mild Cognitive Impairment using clinically transferable features

Conditional Feature Importance for Mixed Data

Contact Info

Product

Resources

About