Examining the Role of Spatial Changes in Bimodal and Uni-Modal To-Be-Ignored Stimuli and How They Affect Short-Term Memory Processes

Marsja, Erik; Marsh, John E.; Hansson, Patrik; Neely, Gregory

doi:10.3389/fpsyg.2019.00299

Cited by 5 publications

(5 citation statements)

References 46 publications

(88 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…can result in higher accuracy and new findings. For instance, Marsja et al (2019) found that changes in bimodal stimuli (both visual and auditory) conveyed a shift in the performance of spatial and verbal short-term memory tasks while changes in visual or auditory stimuli individually did not lead to a significant shift in the performance of the mentioned tasks [27]. This can be easily achieved by the aid of our proposed system, by sending the markers indicating the onset, offset, and other information related to multiple stimuli in different streams simultaneously.…”

Section: Discussionmentioning

confidence: 99%

Multimodal-Multisensory Experiments

Razavi¹,

Yamauchi²,

Janfaza³

et al. 2020

Preprint

View full text Add to dashboard Cite

The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior—task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals—gaze motion, electroencephalogram (EEG), galvanic skin response (GSR), mouse/cursor movement, and body motion—with low-cost consumer-grade devices in a simple behavioral task designed by PsychoPy and a virtual reality environment designed by Unity. This tutorial shows a step-by-step process by which a complex multimodal-multisensory experiment can be designed and implemented in a few hours. When conducting the experiment, all of the data synchronization and recoding of the data to disk will be done automatically.

show abstract

Section: Discussionmentioning

confidence: 99%

Multimodal-Multisensory Experiments

Razavi¹,

Yamauchi²,

Janfaza³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Multimodal-Multisensory Experiments: Design and Implementation

Razavi

Yamauchi

Janfaza

et al. 2020

Preprint

View full text Add to dashboard Cite

The human mind is multimodal. Yet most behavioral studies rely on century-old measures of behavior - task accuracy and latency (response time). Multimodal and multisensory analysis of human behavior creates a better understanding of how the mind works. The problem is that designing and implementing these experiments is technically complex and costly. This paper introduces versatile and economical means of developing multimodal-multisensory human experiments. We provide an experimental design framework that automatically integrates and synchronizes measures including electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, virtual reality (VR), body movement, mouse/cursor motion and response time. Unlike proprietary systems (e.g., iMotions), our system is free and open-source; it integrates PsychoPy, Unity and Lab Streaming Layer (LSL). The system embeds LSL inside PsychoPy/Unity for the synchronization of multiple sensory signals - gaze motion, electroencephalogram (EEG), galvanic skin response (GSR), mouse/cursor movement, and body motion - with low-cost consumer-grade devices in a simple behavioral task designed by PsychoPy and a virtual reality environment designed by Unity. This tutorial shows a step-by-step process by which a complex multimodal-multisensory experiment can be designed and implemented in a few hours. When conducting the experiment, all of the data synchronization and recoding of the data to disk will be done automatically.

show abstract

“…In future, this platform can be expanded for multimodal-multisensory experiments that involve different human senses (e.g., tactile, hear, smell, taste, etc.). Applying OpenSync to different human senses will improve accuracy and is likely to advance our knowledge of human behavior [35]. For this purpose, the state-of-the-art deep learning models are powerful tools that have recently been used for combining and analyzing data from multiple sources together [36].…”

Section: Summary and Future Workmentioning

confidence: 99%

Opensync: An Opensource Platform for Synchronizing Multiple Measures in Neuroscience Experiments

Ghods¹,

Janfaza²,

Yamauchi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: The human mind is multimodal. Yet most behavioral studies rely on century-old measures such as task accuracy and latency. To create a better understanding of human behavior and brain functionality, we should introduce other measures and analyze behavior from various aspects. However, it is technically complex and costly to design and implement the experiments that record multiple measures. To address this issue, a platform that allows synchronizing multiple measures from human behavior is needed. Method: This paper introduces an opensource platform named OpenSync, which can be used to synchronize multiple measures in neuroscience experiments. This platform helps to automatically integrate, synchronize and record physiological measures (e.g., electroencephalogram (EEG), galvanic skin response (GSR), eye-tracking, body motion, etc.), user input response (e.g., from mouse, keyboard, joystick, etc.), and task-related information (stimulus markers). In this paper, we explain the structure and details of OpenSync, provide two case studies in PsychoPy and Unity. Comparison with existing tools: Unlike proprietary systems (e.g., iMotions), OpenSync is free and it can be used inside any opensource experiment design software (e.g., PsychoPy, OpenSesame, Unity, etc., https://pypi.org/project/OpenSync/ and https://github.com/moeinrazavi/OpenSync_Unity). Results: Our experimental results show that the OpenSync platform is able to synchronize multiple measures with microsecond resolution.

show abstract

Examining the Role of Spatial Changes in Bimodal and Uni-Modal To-Be-Ignored Stimuli and How They Affect Short-Term Memory Processes

Cited by 5 publications

References 46 publications

Multimodal-Multisensory Experiments

Multimodal-Multisensory Experiments

Multimodal-Multisensory Experiments: Design and Implementation

Opensync: An Opensource Platform for Synchronizing Multiple Measures in Neuroscience Experiments

Contact Info

Product

Resources

About