The ZuCo Benchmark on Cross-Subject Reading Task Classification with EEG and Eye-Tracking Data

Abstract

We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com.HighlightsWe present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research.We provide an interface to evaluate new approaches with an accompanying public leaderboard.The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading.The data is based on the Zurich Cognitive Language Processing Corpus of simultaneous eye-tracking and EEG signals from natural reading.

Comparing Human Text Classification Performance and Explainability with Large Language and Machine Learning Models Using Eye-Tracking

“…In addition to reading and language comprehension, eye-tracking has also been used in understanding how humans perform other linguistic tasks such as annotation [18,19]. Previous studies [20,21,22] have analyzed eye tracking data such as number of xations, search time, and xation duration recorded during manual annotation of named-entities in texts to a) estimate the di culty level based on cognitive load assessed using gaze data eye-tracking data, and, b) identify prominent features used by humans for named-entity recognition.…”