An Explainable Multimodal Neural Network Architecture for Predicting Epilepsy Comorbidities Based on Administrative Claims Data

Lindén, Thomas; Jong, Johann de; Lu, Chao; Kiri, Victor A.; Haeffs, Kathrin; Fröhlich, Holger

doi:10.3389/frai.2021.610197

Cited by 12 publications

(6 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In multimodal deep learning, neural networks are used to integrate, fuse, and learn complementary representations from multiple input domains (Ngiam et al, 2011 ). Recent work has successfully fused images and text (Abavisani et al, 2020 ), detected adverse weather by combining different types of sensor information (Bijelic et al, 2020 ), estimated the 3-D surface of faces (Abrevaya et al, 2020 ), and combined information from multiple drug and diagnosis domains (Linden et al, 2021 ).…”

Section: Methodsmentioning

confidence: 99%

Multimodal explainable AI predicts upcoming speech behavior in adults who stutter

et al. 2022

View full text Add to dashboard Cite

A key goal of cognitive neuroscience is to better understand how dynamic brain activity relates to behavior. Such dynamics, in terms of spatial and temporal patterns of brain activity, are directly measured with neurophysiological methods such as EEG, but can also be indirectly expressed by the body. Autonomic nervous system activity is the best-known example, but, muscles in the eyes and face can also index brain activity. Mostly parallel lines of artificial intelligence research show that EEG and facial muscles both encode information about emotion, pain, attention, and social interactions, among other topics. In this study, we examined adults who stutter (AWS) to understand the relations between dynamic brain and facial muscle activity and predictions about future behavior (fluent or stuttered speech). AWS can provide insight into brain-behavior dynamics because they naturally fluctuate between episodes of fluent and stuttered speech behavior. We focused on the period when speech preparation occurs, and used EEG and facial muscle activity measured from video to predict whether the upcoming speech would be fluent or stuttered. An explainable self-supervised multimodal architecture learned the temporal dynamics of both EEG and facial muscle movements during speech preparation in AWS, and predicted fluent or stuttered speech at 80.8% accuracy (chance=50%). Specific EEG and facial muscle signals distinguished fluent and stuttered trials, and systematically varied from early to late speech preparation time periods. The self-supervised architecture successfully identified multimodal activity that predicted upcoming behavior on a trial-by-trial basis. This approach could be applied to understanding the neural mechanisms driving variable behavior and symptoms in a wide range of neurological and psychiatric disorders. The combination of direct measures of neural activity and simple video data may be applied to developing technologies that estimate brain state from subtle bodily signals.

show abstract

Section: Methodsmentioning

confidence: 99%

Multimodal explainable AI predicts upcoming speech behavior in adults who stutter

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Consequently, ML models can discard features which are irrelevant for the prediction (sparse models). The set of features selected by the algorithm can then establish a precise biomarker signature which is characteristic for the medical condition under consideration [ 185 – 187 ]. However, any ML model is capable to capture a pattern involving the data which the algorithm was originally trained on.…”

Section: Expert Recommendations For Multi-professional Considerationmentioning

confidence: 99%

The paradigm change from reactive medical services to 3PM in ischemic stroke: a holistic approach utilising tear fluid multi-omics, mitochondria as a vital biosensor and AI-based multi-professional data interpretation

Golubnitschaja,

Polivka,

Potuznik

et al. 2024

EPMA Journal

Self Cite

View full text Add to dashboard Cite

Worldwide stroke is the second leading cause of death and the third leading cause of death and disability combined. The estimated global economic burden by stroke is over US$891 billion per year. Within three decades (1990–2019), the incidence increased by 70%, deaths by 43%, prevalence by 102%, and DALYs by 143%. Of over 100 million people affected by stroke, about 76% are ischemic stroke (IS) patients recorded worldwide. Contextually, ischemic stroke moves into particular focus of multi-professional groups including researchers, healthcare industry, economists, and policy-makers. Risk factors of ischemic stroke demonstrate sufficient space for cost-effective prevention interventions in primary (suboptimal health) and secondary (clinically manifested collateral disorders contributing to stroke risks) care. These risks are interrelated. For example, sedentary lifestyle and toxic environment both cause mitochondrial stress, systemic low-grade inflammation and accelerated ageing; inflammageing is a low-grade inflammation associated with accelerated ageing and poor stroke outcomes. Stress overload, decreased mitochondrial bioenergetics and hypomagnesaemia are associated with systemic vasospasm and ischemic lesions in heart and brain of all age groups including teenagers. Imbalanced dietary patterns poor in folate but rich in red and processed meat, refined grains, and sugary beverages are associated with hyperhomocysteinaemia, systemic inflammation, small vessel disease, and increased IS risks. Ongoing 3PM research towards vulnerable groups in the population promoted by the European Association for Predictive, Preventive and Personalised Medicine (EPMA) demonstrates promising results for the holistic patient-friendly non-invasive approach utilising tear fluid-based health risk assessment, mitochondria as a vital biosensor and AI-based multi-professional data interpretation as reported here by the EPMA expert group. Collected data demonstrate that IS-relevant risks and corresponding molecular pathways are interrelated. For examples, there is an evident overlap between molecular patterns involved in IS and diabetic retinopathy as an early indicator of IS risk in diabetic patients. Just to exemplify some of them such as the 5-aminolevulinic acid/pathway, which are also characteristic for an altered mitophagy patterns, insomnia, stress regulation and modulation of microbiota-gut-brain crosstalk. Further, ceramides are considered mediators of oxidative stress and inflammation in cardiometabolic disease, negatively affecting mitochondrial respiratory chain function and fission/fusion activity, altered sleep–wake behaviour, vascular stiffness and remodelling. Xanthine/pathway regulation is involved in mitochondrial homeostasis and stress-driven anxiety-like behaviour as well as molecular mechanisms of arterial stiffness. In order to assess individual health risks, an application of machine learning (AI tool) is essential for an accurate data interpretation performed by the multiparametric analysis. Aspects presented in the paper include the needs of young populations and elderly, personalised risk assessment in primary and secondary care, cost-efficacy, application of innovative technologies and screening programmes, advanced education measures for professionals and general population—all are essential pillars for the paradigm change from reactive medical services to 3PM in the overall IS management promoted by the EPMA.

show abstract

“…The learned weight models aided in understanding the significance of selected attributes and demonstrated that they represent cross-patient data and open the way to future studies for seizure analysis. In [33], inpatient and outpatient administrative health claims data were used for epilepsy patients. To predict the time-dependent risk of prevalent comorbidities in epilepsy patients, the work in [33] presented a specialized multimodal neural network architecture (Deep personalized LOngitudinal convolutional RIsk model-DeepLORI).…”

Section: Explainable Ai-based Epileptic Seizure Detection and Predictionmentioning

confidence: 99%

Performance Analysis of Deep-Learning and Explainable AI Techniques for Detecting and Predicting Epileptic Seizures

Patil,

Patil

2023

IJRITCC

View full text Add to dashboard Cite

Epilepsy is one of the most common neurological diseases globally. Notably, people in low to middle-income nations could not get proper epilepsy treatment due to the cost and availability of medical infrastructure. The risk of sudden unpredicted death in Epilepsy is considerably high. Medical statistics reveal that people with Epilepsy die more prematurely than those without the disease. Early and accurately diagnosing diseases in the medical field is challenging due to the complex disease patterns and the need for time-sensitive medical responses to the patients. Even though numerous machine learning and advanced deep learning techniques have been employed for the seizure stages classification and prediction, understanding the causes behind the decision is difficult, termed a black box problem. Hence, doctors and patients are confronted with the black box decision-making to initiate the appropriate treatment and understand the disease patterns respectively. Owing to the scarcity of epileptic Electroencephalography (EEG) data, training the deep learning model with diversified epilepsy knowledge is still critical. Explainable Artificial intelligence has become a potential solution to provide the explanation and result interpretation of the learning models. By applying the explainable AI, there is a higher possibility of examining the features that influence the decision-making that either the patient recorded from epileptic or non-epileptic EEG signals. This paper reviews the various deep learning and Explainable AI techniques used for detecting and predicting epileptic seizures using EEG data. It provides a comparative analysis of the different techniques based on their performance.

show abstract

An Explainable Multimodal Neural Network Architecture for Predicting Epilepsy Comorbidities Based on Administrative Claims Data

Cited by 12 publications

References 53 publications

Multimodal explainable AI predicts upcoming speech behavior in adults who stutter

Multimodal explainable AI predicts upcoming speech behavior in adults who stutter

The paradigm change from reactive medical services to 3PM in ischemic stroke: a holistic approach utilising tear fluid multi-omics, mitochondria as a vital biosensor and AI-based multi-professional data interpretation

Performance Analysis of Deep-Learning and Explainable AI Techniques for Detecting and Predicting Epileptic Seizures

Contact Info

Product

Resources

About