On the Explainability of Natural Language Processing Deep Models

Abstract

Despite their success, deep networks are used as black-box models with outputs that are not easily explainable during the learning and the prediction phases. This lack of interpretability is significantly limiting the adoption of such models in domains where decisions are critical such as the medical and legal fields. Recently, researchers have been interested in developing methods that help explain individual decisions and decipher the hidden representations of machine learning models in general and deep networks specifically. While there has been a recent explosion of work on Explainable Artificial Intelligence (ExAI) on deep models that operate on imagery and tabular data, textual datasets present new challenges to the ExAI community. Such challenges can be attributed to the lack of input structure in textual data, the use of word embeddings that add to the opacity of the models and the difficulty of the visualization of the inner workings of deep models when they are trained on textual data. Lately, methods have been developed to address the aforementioned challenges and present satisfactory explanations on Natural Language Processing (NLP) models. However, such methods are yet to be studied in a comprehensive framework where common challenges are properly stated and rigorous evaluation practices and metrics are proposed. Motivated to democratize ExAI methods in the NLP field, we present in this work a survey that studies model-agnostic as well as model-specific explainability methods on NLP models. Such methods can either develop inherently interpretable NLP models or operate on pre-trained models in a post hoc manner. We make this distinction and we further decompose the methods into three categories according to what they explain: (1) word embeddings (input level), (2) inner workings of NLP models (processing level), and (3) models’ decisions (output level). We also detail the different evaluation approaches interpretability methods in the NLP field. Finally, we present a case-study on the well-known neural machine translation in an appendix, and we propose promising future research directions for ExAI in the NLP field.

Neural dynamics of semantic control underlying generative storytelling

“…We started our analysis plan by firstly adopting a computational approach to operationalize the abstract features we required participants to manipulate during the generation of the stories. Our approach relied on recent advances in natural language processing based on deep neural network models that learn statistical patterns from large text corpora 50,51,58,66 and allowed us to successfully differentiate the novel and appropriate components of complex textual data such as narratives. Note that, although the use of semantic distance as a measure of novelty or originality has been extensively used in previous research 42,47,61 , our measure of appropriateness, derived from the BERT model’s surprise, has not been used yet and in our opinion can be a valid measure for further studies attempting at quantifying the usefulness of a narrative.…”

Applications of artificial intelligence-enabled robots and chatbots in ophthalmology: recent advances and future trends

“…The healthcare industry is currently undergoing a transition from small-scale AI models to large-scale foundation AI models. Notably, in 2022, LLMs such as Chatbot Generative Pretrained Transformer (ChatGPT) [45] have exhibited reasonable performance in various domains [46,47 ▪▪ ]. Typically, small-scale AI systems are engineered to execute precise and limited-scope operations, such as health data screening or medical image analysis [48].…”

Explaining video summarization based on the focus of attention

“…Nowadays there is a growing interest on methods providing explanations about the working mechanism or the predictions of neural networks. A lot of progress has been made in the domains of pattern recognition [9] and natural language processing [10]. However, only a few works deal with network architectures processing video data.…”