Missing-Stimulus Potentials Associated With a Disruption of Human-Computer Interaction

Abstract

Cognitive processing associated with the recognition of computer malfunction was examined by recording event-related brain potentials time-locked to voluntary mouse button clicks that did not produce feedback from the computer. Sixteen university students performed a simple stimulus discrimination (oddball) task in auditory and visual modalities. Each button press produced either a target or a standard stimulus, or no stimulus response in some trials (missing stimulus, p=0.2). Biphasic missing-stimulus potentials appeared after the stimulus omission within a latency range of 150 to 550 ms. For both auditory and visual modalities, the initial negative component was dominant at the right temporal site, whereas the following positive component was at the central site. The peak latencies of these potentials were about 30 ms shorter in the auditory than in the visual modality, though the eliciting event was identical (i.e., stimulus omission). Brain potentials after users' operations may provide an objective and unobtrusive index of users' expectations in human-computer interaction.

Attention and prediction in human audition: a lesson from cognitive psychophysiology

“…Omission responses were only obtained when the specific physical characteristics of the sound could be predicted, and not when only the expectation of a sound was possible, but its identity was unknown. To our knowledge, ERP responses elicited by omissions of the expected sensory consequences of a button press have been reported in another two studies, also when the missing stimulus was in the visual modality (Nittono, ; Nittono & Sakata, ). Unfortunately these studies focused their analysis only on late responses (~200 ms and later).…”

Event-related potentials elicited by unexpected visual stimuli after voluntary actions

“…Event-related brain potentials (ERPs) have been proposed to be a tool of assessing the state of attention in human-computer interaction (Nittono, 2005;Nittono et al, 2003). Previous studies have shown that the frontocentral P3 (P3a) elicited by rare deviant stimuli is enhanced when a participant controls the initiation of stimulus presentation by pressing a single trigger button, compared to when a computer controls the stimulus initiation without a participant's action (Nittono, 2004;Nittono and Ullsperger, 2000).…”

Omitted Stimulus Potential depends on the sensory modality

“…The OSP reflects a type of expectation that is quickly generated with each stimulus (Sutton et al1965, 1967, Karamürsel and Bullock 2000, Janata 2001, Jongsma et al 2004, 2005, 2006. It requires few conditioning stimuli but it is essential that subjects be focused because the OSP is not observed in inattentive or drowsy subjects (Takasaka 1985, Jongsma et al 2004.…”

“…It has also been suggested that the OSP may be sensitive to uncertainty and might be delayed until the uncertainty is resolved (Sutton et al 1965(Sutton et al , 1967. The OSP appears after slow trains (0.3-1 Hz) of stimuli and has a slow time course and long duration with a positive peak approximately 400-1100 ms after the due time of the first omitted stimulus (Simson et al 1976, Stapleton et al 1987, Hamon et al 1989, Tarkka and Stokic 1998, Karamürsel and Bullock 2000, Jongsma et al 2004, 2005, 2006, Nakano et al 2014). The OSP is totally attention-dependent and belongs to the "slow" type seen in humans and other vertebrates , Takasaka 1985, Bullock et al1994, Jongsma et al 2004).…”