Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Nodes

Cho, Sohee; Lee, Ginkyeng; Chang, Wonjoon; Choi, Jaesik

doi:10.48550/arxiv.2004.12538

Cited by 3 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They observe that even if both [3] and [17] correctly identify the discriminative time window, reference [17] provides more precise attribution maps and thus a more informative explanation. Inspired by the approach of Network Dissection, namely a method showing the spatial locations that each unit in the CNN is looking at [28], Siddiqui et al [18], and Cho et al [19] rely on the neuron and filter activation of a CNN with the aim of identifying the contribution of the raw input data when performing MTS classification. The former [18] creates a dummy dataset for time series anomaly detection with three features, that is, pressure, temperature and torque.…”

Section: Related Workmentioning

confidence: 99%

“…The former [18] creates a dummy dataset for time series anomaly detection with three features, that is, pressure, temperature and torque. The latter [19] interprets deep temporal representations using two open-source MTS datasets. A final interesting contribution is by Schegel et al [13], which evaluates 5 explainability methods, considering both agnostic and model-specific approaches previously used for image and text-domain.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Translating Image XAI to Multivariate Time Series

Tronchin,

Cordelli,

Celsi

et al. 2024

IEEE Access

View full text Add to dashboard Cite

As Artificial Intelligence (AI) is becoming part of our daily lives, the need to understand and trust its decisions is becoming a pressing issue. EXplainable AI (XAI) aims at answering this demand, providing tools to get insights into the models' behaviour and reasoning. Following this trend, our research paper explores the explainability of a deployed multimodal architecture applied to a real-world dataset of multivariate time series. The study aims to enhance the trustworthiness of an AI agent responsible for crash detection in an insurance company's automatic assistance service. By introducing an XAI layer, we provide insights into the AI agent's decision-making process, enabling the optimization of emergency medical services allocation. The dataset consists of real-world telematics data collected from vehicles equipped with black box technology. The challenge lies in explaining the complex interactions within the multivariate time series data to accurately understand the forces applied to vehicles during accidents. To this end, we adapt to this context two state-of-the-art XAI model-specific approaches, originally designed for images. We offer a qualitative and a quantitative evaluation, also comparing with a well-known agnostic method, and further validating our findings on an external dataset. The results show that Integrated Gradients, among the methodologies examined, is the most effective approach. Its ability to handle the complexity of the data provides the most comprehensive and insightful explanations for the considered use case. The findings emphasize the potential of XAI to enhance the trustworthiness of AI systems and optimize emergency response in the insurance industry. Code is available at https://github.com/ltronchin/translating-xai-mts.git.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Translating Image XAI to Multivariate Time Series

Tronchin,

Cordelli,

Celsi

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Siddiqui et al (2019) clustered CNN filters according to their activation pattern, based on the assumption that filters with similar activation patterns basically detect the same content. Cho et al (2020) proposed an alternative clustering approach by grouping input sub-sequences that activate the same nodes, and associating each cluster to a representative example. These methods represent preliminary works to achieve some global insights, but much more effort is still required to generate human-consumable explanations, and especially to explain the latent space of CNN.…”

Section: Understanding Dnn Modelsmentioning

confidence: 99%

Explainable AI for clinical and remote health applications: a survey on tabular and time series data

Martino

Delmastro

2022

Artif Intell Rev

View full text Add to dashboard Cite

Nowadays Artificial Intelligence (AI) has become a fundamental component of healthcare applications, both clinical and remote, but the best performing AI systems are often too complex to be self-explaining. Explainable AI (XAI) techniques are defined to unveil the reasoning behind the system’s predictions and decisions, and they become even more critical when dealing with sensitive and personal health data. It is worth noting that XAI has not gathered the same attention across different research areas and data types, especially in healthcare. In particular, many clinical and remote health applications are based on tabular and time series data, respectively, and XAI is not commonly analysed on these data types, while computer vision and Natural Language Processing (NLP) are the reference applications. To provide an overview of XAI methods that are most suitable for tabular and time series data in the healthcare domain, this paper provides a review of the literature in the last 5 years, illustrating the type of generated explanations and the efforts provided to evaluate their relevance and quality. Specifically, we identify clinical validation, consistency assessment, objective and standardised quality evaluation, and human-centered quality assessment as key features to ensure effective explanations for the end users. Finally, we highlight the main research challenges in the field as well as the limitations of existing XAI methods.

show abstract

“…Wang et al adapted the CAM method to time series data to visualize the region of raw input data that activates the network's neurons for a given label [42]. In another work on univariate time series data, Clustered Pattern of Highly Activated Period (CPHAP) [10,9] extracts those representative patterns from the input data that highly activate neurons in a CNN channel. However, all these methods can only handle univariate time series data.…”

Section: Interpretable Modelsmentioning

confidence: 99%

“…These neurons are activated based on the features captured by the training data. Cho et al [10] proposed a method to extract the highly activated periods from univariate time series data, while [46] proposed a similar method for multivariate time series data. The unique features of multivariate time series data, i.e., unknown dependencies among multiple channels (dimensions) and through time, pose challenges.…”

Section: Multivariate Highly Activated Period Extractionmentioning

confidence: 99%

Multivariate Time Series Analysis: An Interpretable CNN-based Model

Younis

Zerr

Ahmadi

2022

2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA)

View full text Add to dashboard Cite

Conventional time series classification approaches based on bags of patterns or shapelets face significant challenges in dealing with a vast amount of feature candidates from high-dimensional multivariate data. In contrast, deep neural networks can learn low-dimensional features efficiently, and in particular, Convolutional Neural Networks (CNN) have shown promising results in classifying Multivariate Time Series (MTS) data. A key factor in the success of deep neural networks is this astonishing expressive power. However, this power comes at the cost of complex, black-boxed models, conflicting with the goals of building reliable and human-understandable models. An essential criterion in understanding such predictive deep models involves quantifying the contribution of time-varying input variables to the classification. Hence, in this work, we introduce a new framework for interpreting multivariate time series data by extracting and clustering the input representative patterns that highly activate CNN neurons. This way, we identify each signal's role and dependencies, considering all possible combinations of signals in the MTS input. Then, we construct a graph that captures the temporal relationship between the extracted patterns for each layer. An effective graph merging strategy finds the connection of each node to the previous layer's nodes. Finally, a graph embedding algorithm generates new representations of the created interpretable time-series features. To evaluate the performance of our proposed framework, we run extensive experiments on eight datasets of UCR/UEA archive, along with HAR and PAM datasets. The experiments indicate the benefit of our timeaware graph-based representation in MTS classification while enriching them with more interpretability.

show abstract

Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Nodes

Cited by 3 publications

References 19 publications

Translating Image XAI to Multivariate Time Series

Translating Image XAI to Multivariate Time Series

Explainable AI for clinical and remote health applications: a survey on tabular and time series data

Multivariate Time Series Analysis: An Interpretable CNN-based Model

Contact Info

Product

Resources

About