Using binocular rivalry to tag foreground sounds: Towards an objective visual measure for auditory multistability

Abstract

In binocular rivalry, paradigms have been proposed for unobtrusive moment-by-moment readout of observers' perceptual experience ("no-report paradigms"). Here, we take a first step to extend this concept to auditory multistability. Observers continuously reported which of two concurrent tone sequences they perceived in the foreground: high-pitch (1008 Hz) or low-pitch (400 Hz) tones. Interstimulus intervals were either fixed per sequence (Experiments 1 and 2) or random with tones alternating (Experiment 3). A horizontally drifting grating was presented to each eye; to induce binocular rivalry, gratings had distinct colors and motion directions. To associate each grating with one tone sequence, a pattern on the grating jumped vertically whenever the respective tone occurred. We found that the direction of the optokinetic nystagmus (OKN)-induced by the visually dominant grating-could be used to decode the tone (high/low) that was perceived in the foreground well above chance. This OKN-based readout improved after observers had gained experience with the auditory task (Experiments 1 and 2) and for simpler auditory tasks (Experiment 3). We found no evidence that the visual stimulus affected auditory multistability. Although decoding performance is still far from perfect, our paradigm may eventually provide a continuous estimate of the currently dominant percept in auditory multistability.

Visual Awareness in Binocular Rivalry Modulates Induced Pupil Fluctuations

“…One objective of applying luminance modulation to high-level stimuli is the development of a no-report paradigm for such complex stimuli in BR. Although moment-by-moment decoding in the present paradigm is far below optimal and also clearly below other oculomotor measures such as the optokinetic nystagmus (OKN; Einhäuser, Thomassen & Bendixen, 2017; Fox, Todd, & Bettinger, 1975; Frässle et al, 2014; Wilbertz, Ketkar, Guggenmos, & Sterzer, 2017; Marx & Einhäuser, 2015; Naber et al, 2011; Soltész, Pastukhov, Braun, & Kovács, 2016), decoding performance is significantly above chance for all experimental conditions. While further optimization is necessary, it is conceivable that the notion of no-report paradigms in BR can be extended to complex stimuli and used in combination with other techniques, in particular fMRI.…”

Compact, modular and in-plane AOSLO for high-resolution retinal imaging

“…Further developments have opened up the possibility of studying other retinal structures such as inner segments of cones, imaged via split-detection [ 8 ], the retinal pigment epithelium, imaged via dark-field techniques [ 9 ] and retinal ganglion cells imaged via offset-aperture imaging [ 10 ]. AOSLO imaging is increasingly being used to study the function of the retina, such as using single-cell stimulation to understand the neural wiring in the retina [ 11 ], monitoring blood flow [ 12 ] and metabolic activity [ 13 ], and analyzing motion distortions within the images for eye tracking at high spatial and temporal resolution [ 14 – 16 ].…”

Interactions between the visual and motor systems during optokinetic stimulation and simultaneous dynamic handgrip exercise: an EEG study

“…Whole scalp EEG holds great potential as a valuable tool in research fields involving nystagmus, such as vestibular research, since it can compensate for the lack of temporal resolution in other techniques currently used in the field (Lopez and Blanke, 2011). OKS and OKN are also used in other fields such as in binocular rivalry where it has been previously suggested that simultaneous EEG should be possible using ICA (Einhäuser et al 2017). Our whole-scalp recording during OKS and use of ICA and Surface Laplacian supports the successful use of EEG when nystagmus artefacts are unavoidable.…”

“…This absence of evidence may also be due to the problem of the unavoidable eye-movement artefacts in the EEG induced by OKN (Gulyás et al, 2007), which are stronger at anterior regions. However, recent advances allow circumventing this problem through techniques such as independent component analysis (ICA), used to separate out eyemovement components (Bell and Sejnowski, 1995, Makeig et al, 1995) and speculated to be potentially as applicable to OKN EEG artefacts as to saccades (Einhäuser et al 2017). Spatial filters like the surface Laplacian further reduce volume conduction and help to specify the location of experimental effects (Kayser and Tenke, 2015, Tenke and Kayser, 2005).…”