Using reading behavior to predict grammatical functions

Barrett, Maria; Søgaard, Anders

doi:10.18653/v1/w15-2401

Cited by 25 publications

(21 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Positive results have emerged from a range of diverging representations. In some cases, including tens of gaze features show a benefit (Mishra et al, 2017a;Barrett and Søgaard, 2015b) while other studies report successful experiments using a single gaze feature (Barrett et al, 2018a;Klerke et al, 2016).…”

Section: Background and Motivationmentioning

confidence: 99%

Proceedings of the Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN)

2019

View full text Add to dashboard Cite

Neural Module Networks, originally proposed for the task of visual question answering, are a class of neural network architectures that involve human-specified neural modules, each designed for a specific form of reasoning. In current formulations of such networks only the parameters of the neural modules and/or the order of their execution is learned. In this work, we further expand this approach and also learn the underlying internal structure of modules in terms of the ordering and combination of simple and elementary arithmetic operators. We utilize a minimum amount of prior knowledge from the human-specified neural modules in the form of different input types and arithmetic operators used in these modules. Our results show that one is indeed able to simultaneously learn both internal module structure and module sequencing without extra supervisory signals for module execution sequencing. With this approach, we report performance comparable to models using hand-designed modules. In addition, we do a analysis of sensitivity of the learned modules w.r.t. the arithmetic operations and infer the analytical expressions of the learned modules.

show abstract

Section: Background and Motivationmentioning

confidence: 99%

Proceedings of the Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN)

2019

View full text Add to dashboard Cite

show abstract

“…Eye-tracking technology is a relatively new NLP, with very few systems directly making use of gaze data in prediction frameworks. Klerke et al (2016) present a novel multi-task learning approach for sentence compression using labeled data, while, Barrett and Søgaard (2015) discriminate between grammatical functions using gaze features. The closest works to ours are by Mishra et al (2016b) and Mishra et al (2016c) that introduce feature engineering based on both gaze and text data for sentiment and sarcasm detection tasks.…”

Section: Related Workmentioning

confidence: 99%

Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network

Mishra¹,

Dey²,

Bhattacharyya³

2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

100

View full text Add to dashboard Cite

Cognitive NLP systems-i.e., NLP systems that make use of behavioral data -augment traditional text-based features with cognitive features extracted from eye-movement patterns, EEG signals, brain-imaging etc.. Such extraction of features is typically manual. We contend that manual extraction of features may not be the best way to tackle text subtleties that characteristically prevail in complex classification tasks like sentiment analysis and sarcasm detection, and that even the extraction and choice of features should be delegated to the learning system. We introduce a framework to automatically extract cognitive features from the eye-movement / gaze data of human readers reading the text and use them as features along with textual features for the tasks of sentiment polarity and sarcasm detection. Our proposed framework is based on Convolutional Neural Network (CNN). The CNN learns features from both gaze and text and uses them to classify the input text. We test our technique on published sentiment and sarcasm labeled datasets, enriched with gaze information, to show that using a combination of automatically learned text and gaze features often yields better classification performance over (i) CNN based systems that rely on text input alone and (ii) existing systems that rely on handcrafted gaze and textual features.

show abstract

“…Mishra et al (2013) measure translation annotation difficulty of a given sentence based on gaze input of translators used to label training data. Klerke et al (2016) present a novel multi-task learning approach for sentence compression using labelled data, while, Barrett and Søgaard (2015) discriminate between grammatical functions using gaze features. The recent advancements in the literature discussed above, motivate us to explore gaze-based cognition for sentiment analysis.…”

Section: Related Workmentioning

confidence: 99%

Leveraging Cognitive Features for Sentiment Analysis

Mishra

Kanojia

Nagar

et al. 2016

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning

View full text Add to dashboard Cite

Sentiments expressed in user-generated short text and sentences are nuanced by subtleties at lexical, syntactic, semantic and pragmatic levels. To address this, we propose to augment traditional features used for sentiment analysis and sarcasm detection, with cognitive features derived from the eye-movement patterns of readers. Statistical classification using our enhanced feature set improves the performance (F-score) of polarity detection by a maximum of 3.7% and 9.3% on two datasets, over the systems that use only traditional features. We perform feature significance analysis, and experiment on a held-out dataset, showing that cognitive features indeed empower sentiment analyzers to handle complex constructs.

show abstract

Using reading behavior to predict grammatical functions

Cited by 25 publications

References 8 publications

Proceedings of the Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN)

Proceedings of the Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN)

Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network

Leveraging Cognitive Features for Sentiment Analysis

Contact Info

Product

Resources

About