A SLAM-based augmented reality app for the assessment of spatial short-term memory using visual and auditory stimuli

Abstract

A SLAM-based Augmented Reality (AR) app has been designed, developed, and validated to assess spatial short-term memory. Our app can be used with visual and auditory stimuli and can run on mobile devices. It can be used in any indoor environment. The anchors and data of the app are persistently stored in the cloud. As an authoring tool, the type of stimulus, its number, and specific positions in the real environment can be customized for each session. A study involving 48 participants was carried out to analyze the performance outcomes comparing the location and remembering of stimuli in a real environment using visual versus auditory stimuli. The number of objects placed correctly was similar for the two different stimuli used. However, the group that used the auditory stimulus spent significantly more time completing the task and required significantly more attempts. The performance outcomes were independent of age and gender. For the auditory stimuli, correlations among all of the variables of the AR app and the variables of two other tasks (object-recall and map-pointing) were found. We also found that the greater the number of correctly placed auditory stimuli, the greater the perceived competence and the less mental effort required. The greater the number of errors, the less the perceived competence. Finally, the auditory stimuli are valid stimuli that may benefit the assessment of the memorization of spatial-auditory associations, but the memorization of spatial-visual associations is dominant, as our results suggest.

Augmented Reality to Assess Short-Term Spatial Memory: A Comparative Study of Olfactory, Visual, and Tactile Stimuli

“…There are previous works on the use of AR for the assessment of spatial memory. Most of these works were presented by our research group [1], [35]- [39]. The first works [35], [36] developed an AR application that used images as targets.…”

“…The study concluded that the distance estimations were more accurate when displaying a grid but that object location memory performance was worse when displaying a grid. Auditory stimuli have also been studied for the assessment of spatial memory [39]. This work presented an AR application that included visual and auditory stimuli for the assessment of spatial memory.…”

“…This work presented an AR application that included visual and auditory stimuli for the assessment of spatial memory. The study included in [39] compared the performance of participants when receiving visual or auditory stimuli. The study concluded that the number of successes was similar for the two stimuli, but the memorization of spatial-visual associations was dominant.…”

PLPF‐VSLAM: An indoor visual SLAM with adaptive fusion of point‐line‐plane features

“…SLAM has been widely used in various fields, such as indoor autonomous navigation (Wen et al, 2021), virtual reality (VR) (Jiang et al, 2022), and augmented reality (AR) (Juan et al, 2022). Conventional VSLAM typically relies on point features to track the movements of agents and build maps, as this approach is simple and effective.…”

“…SLAM has been widely used in various fields, such as indoor autonomous navigation (Wen et al, 2021), virtual reality (VR) (Jiang et al, 2022), and augmented reality (AR) (Juan et al, 2022).…”