AffectiveTDA: Using Topological Data Analysis to Improve Analysis and Explainability in Affective Computing

Elhamdadi, Hamza; Canavan, Shaun; Rosen, Paul

doi:10.1109/tvcg.2021.3114784

Cited by 8 publications

(4 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Carbonell et al [60] use unsupervised methods to augment the supervised analysis of emotion expressions, revealing additional insights into how emotion expressions differ by individual and gender. Also aimed at comparing and contrasting data across emotions and individuals, the authors of AffectiveTDA [61] propose to use topological data analysis (TDA) to build an explainable visual data representation of facial expressions using facial landmarks.…”

Section: A Eda: Understanding Multimodal Affective Datasetsmentioning

confidence: 99%

Toward Explainable Affective Computing: A Review

Cortiñas-Lorenzo,

Lacey

2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Affective computing has an unprecedented potential to change the way humans interact with technology. While the last decades have witnessed vast progress in the field, multimodal affective computing systems are generally black box by design. As affective systems start to be deployed in real-world scenarios, such as education or healthcare, a shift of focus toward improved transparency and interpretability is needed. In this context, how do we explain the output of affective computing models? and how to do so without limiting predictive performance? In this article, we review affective computing work from an explainable AI (XAI) perspective, collecting and synthesizing relevant papers into three major XAI approaches: premodel (applied before training), in-model (applied during training), and postmodel (applied after training). We present and discuss the most fundamental challenges in the field, namely, how to relate explanations back to multimodal and time-dependent data, how to integrate context and inductive biases into explanations using mechanisms such as attention, generative modeling, or graph-based methods, and how to capture intramodal and cross-modal interactions in post hoc explanations. While explainable affective computing is still nascent, existing methods are promising, contributing not only toward improved transparency but, in many cases, surpassing state-of-the-art results. Based on these findings, we explore directions for future research and discuss the importance of data-driven XAI and explanation goals, and explainee needs definition, as well as causability or the extent to which a given method leads to human understanding.

show abstract

Section: A Eda: Understanding Multimodal Affective Datasetsmentioning

confidence: 99%

Toward Explainable Affective Computing: A Review

Cortiñas-Lorenzo,

Lacey

2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…BubblEX framework consists of two stages: Visualization Module and Interpretability Module. Firstly, t-SNE [23] and UMAP [24] are used as they attempt to preserve the clustering structure by considering a local neighbor [12]. Both t-SNE and UMAP contain hyperparameters that can impact the structures visible to the operator.…”

Section: A Nature and Scopementioning

confidence: 99%

“…DNNs architectures are increasingly being adopted in several domains from medical diagnosis [8], retail [9] to autonomous driving [10] due to their competence to learn relevant abstractions from data. At first, these models were considered as "black box" operators, but as their popularity has grown they need to be interpretable and explainable [11], [12], [13]. The perception of DNNs as "black box" algorithms makes difficult to ethically justify their use in high-stake decisions, especially in case of failure [14].…”

Section: Introductionmentioning

confidence: 99%

BubblEX: An Explainable Deep Learning Framework for Point-Cloud Classification

Matrone

Paolanti

Felicetti

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Point clouds data are nowadays one of the major data sources for describing our environment. Recently, deep architectures have been proposed as a key step in understanding and retrieving semantic information. Despite the great contribution of deep learning in this field, the explainability of these models for 3D data is still fairly unexplored. Explainability, identified as a potential weakness of Deep Neural Networks (DNNs), can help researchers against skepticism, considering that these models are far from being self-explanatory. Although literature provides many examples on the exploitation of Explainable Artificial Intelligence (XAI) approaches with 2D data, only a few studies have attempted to investigate it for 3D DNNs. To overcome these limitations, it is here proposed BubblEX, a novel multimodal fusion framework to learn the 3D point features. BubblEX framework comprises two stages: "Visualization Module" for the visualisation of features learned from the network in its hidden layers and "Interpretability Module", which aims at describing how the neighbour points are involved in the features extraction. For our experiments, it has been used Dynamic Graph CNN (DGCNN), trained on Modelnet40 dataset. The developed framework extends a method for obtaining saliency maps from image data, to deal with 3D point cloud data, allowing the analysis, comparison and contrasting of multiple features. Besides, it permits the generation of visual explanations from any DNN-based network for 3D point cloud classification without requiring architectural changes or re-training. Our findings will be extremely useful for both scientists and non-experts in understanding and improving future AI-based models.

show abstract

“…DNN ar-chitectures are increasingly being adopted in geomatics due to their competence to learn relevant abstractions from data. At first, these models were considered "black box" operators, but as their popularity has grown they need to be interpretable and explainable (Xiao et al, 2018;Elhamdadi et al, 2021;Fuhrman et al, 2021). Moreover, deep learning methods are needed to cope with complex statistics, multiple outputs, different noise sources, and high-dimensional spaces.…”

Section: Algorithms and Models For Geoaimentioning

confidence: 99%

GeoAI: a review of artificial intelligence approaches for the interpretation of complex geomatics data

Pierdicca

Paolanti

2022

Geosci. Instrum. Method. Data Syst.

View full text Add to dashboard Cite

Abstract. Researchers have explored the benefits and applications of modern artificial intelligence (AI) algorithms in different scenarios. For the processing of geomatics data, AI offers overwhelming opportunities. Fundamental questions include how AI can be specifically applied to or must be specifically created for geomatics data. This change is also having a significant impact on geospatial data. The integration of AI approaches in geomatics has developed into the concept of geospatial artificial intelligence (GeoAI), which is a new paradigm for geographic knowledge discovery and beyond. However, little systematic work currently exists on how researchers have applied AI for geospatial domains. Hence, this contribution outlines AI-based techniques for analysing and interpreting complex geomatics data. Our analysis has covered several gaps, for instance defining relationships between AI-based approaches and geomatics data. First, technologies and tools used for data acquisition are outlined, with a particular focus on red–green–blue (RGB) images, thermal images, 3D point clouds, trajectories, and hyperspectral–multispectral images. Then, how AI approaches have been exploited for the interpretation of geomatic data is explained. Finally, a broad set of examples of applications is given, together with the specific method applied. Limitations point towards unexplored areas for future investigations, serving as useful guidelines for future research directions.

show abstract

AffectiveTDA: Using Topological Data Analysis to Improve Analysis and Explainability in Affective Computing

Cited by 8 publications

References 70 publications

Toward Explainable Affective Computing: A Review

Toward Explainable Affective Computing: A Review

BubblEX: An Explainable Deep Learning Framework for Point-Cloud Classification

GeoAI: a review of artificial intelligence approaches for the interpretation of complex geomatics data

Contact Info

Product

Resources

About